Multimodality in Architecture: Collaboration, Technology and Education [1st ed. 2024] 3031495101, 9783031495106

Citation preview

Ju Hyun Lee Michael J. Ostwald Mi Jeong Kim   Editors

Multimodality in Architecture Collaboration, Technology and Education

Multimodality in Architecture

Ju Hyun Lee · Michael J. Ostwald · Mi Jeong Kim Editors

Multimodality in Architecture Collaboration, Technology and Education

Editors Ju Hyun Lee School of Built Environment, Faculty of Arts, Design and Architecture The University of New South Wales Sydney, NSW, Australia

Michael J. Ostwald School of Built Environment, Faculty of Arts, Design and Architecture The University of New South Wales Sydney, NSW, Australia

Mi Jeong Kim School of Architecture Hanyang University Seoul, Republic of Korea

ISBN 978-3-031-49510-6 ISBN 978-3-031-49511-3 (eBook) https://doi.org/10.1007/978-3-031-49511-3 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland Paper in this product is recyclable.

Preface

This book is about multimodality in architecture, a topic which has become increasingly significant in recent years. Multimodality is central to collaboration and is core to understanding the technical systems that support architectural practice and education. Despite its importance, it is a topic that needs to be better understood in the architectural field. This preface introduces the concept of multimodality and explains why it is significant for architectural scholars, practitioners, and educators. After that, the preface provides an overview of the book’s scope. The word “multimodal” refers to the presence of more than one type of information, data, or activity in a process or system. For example, in a communication process, two standard modes of information transmission that often occur concurrently are “verbal” and “visual”. Thus, a simple multimodal presentation might combine speech and images. In a contemporary team meeting or classroom, the speech is often “live”, although the speaker may not be in the same physical space as the participants, and the images will typically be presented using software (PowerPoint, Keynote or equivalent). In such an example, the modes are no longer limited to “verbal” and “visual”. The verbal might be divided into “physical” (in the same room) or “virtual” (not in the same room) modes, and the visual might comprise a range of different modes, from still images to video and animations of data trends. Such data types, sources, and operations can all be conceptualised as modes in a larger communication process or inputs in a communication system. In the latter case, if conceptualised as a system, the various verbal and visual modes are “inputs” in the process, and the “output” is knowledge or skill development (often divided into three levels, an “awareness of”, an “understanding of”, and a “capacity to”). Thus, multimodality offers a way of understanding a system in terms of the various sources of data or information available and its potential outcomes. Depending on the process or system, the nature of the modality can also shift. For example, while the communication mode “speech” may be helpful for a presentation, it is less effective in a collaborative setting. In contrast, the “discussion” mode is more valuable for this purpose. Similarly, the “image” mode may be ideal for depicting a fixed or static piece of information. However, having the capacity to modify that image, to add notes or graphic information, is vital in a team process where collective v

vi

Preface

intelligence must be developed and applied. Thus, an “interactive visualisation” mode using, for example, collaborative sketching, is more effective for a design team than the presentation of a fixed image. Different collaborative learning approaches can also be treated as modes in a larger pedagogical process. Thus, the combination of “peer-to-peer learning” and “self-directed learning” modes is thought to be superior, in many circumstances, to the “didactic learning” mode, often called “teacher-centred instruction”. These examples of collaborative, technical, and educational processes or systems emphasise that there are different types of modes. The most common tend to be linked to communication and cognition and typically rely on the senses for inputs (sight, sound, touch, taste, and smell). But modes can also be defined by their purpose in the system, with some, for example, providing data and others analysing data. Similarly, modes may be classified in terms of their types of interaction, while others may be defined by the extent to which their operations are structured. Thus, multimodality can be used to conceptualise and optimise the relationships between complex inputs, operations, and outputs in a larger process. Multimodality is an important concept in contemporary research because many processes are thought to benefit from additional sources of information, data, or activities. This is partly because these additional inputs provide a basis for triangulating results. Thus, multimodality can be linked to increased confidence in the validity or effectiveness of a process. Of equal importance, multimodality may support equity and access in a process. People in teams and classrooms will have different social, cultural, and professional backgrounds, skills, and experiences, some of which may hinder their capacity to engage with a process. For example, people from different linguistic backgrounds typically require additional modes to support effective communication in both learning environments and design teams. Thus, whereas speech and image modes may be enough for some design team members to communicate effectively, the inclusion of physical models and text may assist others to gain a similar level of understanding. Multimodality is, therefore, valued for its capacity to improve transparency (resulting in more effective communication) and accessibility (supporting enhanced interaction and engagement from people with diverse backgrounds). Architectural practice, especially the design process, is innately multimodal, although this property is often underestimated or overlooked. For example, the history of architecture, which is how many people understand the profession, tends to group buildings, people, and technology into “movements”, “eras”, or “styles” and then arrange these in order from oldest to newest. Thus, Western architectural histories typically commence with descriptions of Ancient Greece before moving on to Medieval, Romanesque, Gothic, Renaissance, and Modern movements. Such a formulation of history is necessarily reductive, partly because its purpose is not to explain the mechanics of architecture, just one aspect of its evolution. However, such accounts also generally lack any reference to the processes or systems used to translate design ideas into reality. For example, the methods architects use to communicate their design intentions with their clients (or patrons) and builders (guilds or artisans) are rarely mentioned. Discussion of collaboration in design is also largely

Preface

vii

missing from architectural histories, even though common sense will tell people that design teams must have existed before the late twentieth century. The technologies used to enable design communication and the education of journeyman architects are also largely absent in architectural histories. While there are exceptions, a reader of the history of architecture could be forgiven for thinking that buildings manifest out of thin air. The collaborative, technical, and educational modes are the missing processes and systems that transform ideas into reality. The topic of multimodality in architecture is especially pertinent today in a world where architects’ systems and processes are increasingly reliant on technology. Architectural meetings regularly occur online, bringing together teams from diverse cultural and linguistic backgrounds to produce large, complex designs. Architectural teams share multiple sources of data, often embedded in computer-aided design (CAD) or building information model (BIM) files, which are digital representations of buildings combined with archives of information about their performance, construction, operations, and maintenance. The COVID-19 pandemic accelerated this shift in reliance on technical modes, with both large and small architectural firms being forced to adopt digital modes to transform their communication and collaboration practices. A consequence is that the types of critical interaction that occur in international, multicultural design teams are now largely reliant on technology. The research collected in this book was developed during this time of rapid transformation when architectural firms and schools shifted their operations and embraced new modes of communication, technology, and knowledge transfer in an increasingly global workplace. The challenges addressed in this book coalesce around three themes—collaboration, technology, and education—all of which can be conceptualised as multimodal systems or processes. Part I (Collaboration) of this book examines multimodality and architectural teams. Some topics covered include design processes and communication in remote teamwork, digital design collaboration, co-design in virtual environments, and the complexities of cross-cultural and multilingual design practice. Part II (Technology) is focused on multimodality and technology in architecture and design. The content includes research about the operations of design teams in immersive virtual environments, advanced scanning and three-dimensional printing technology, AIgenerated images for architects, and collective intelligence models incorporating human and digital inputs. Part III (Education) examines problem-based learning in a virtual environment, modelling stress in educational settings, learner experiences of computer-mediated environments, and pattern languages to support educational design. This book has been written for practitioners, teachers, students, and scholars interested in understanding and optimising processes in architecture and design. It grew out of research presented at the first (2022) and second (2023) symposia, “Multiculturalism and Multimodality in Architecture”, of the Australia-Korea Architecture Network (AKA.N or https://auskorarch.net/). This research explored the communication and collaboration challenges faced by architectural teams working remotely in diverse environments. These symposia grew out of the project “Supporting Exports of International Creative Team’s Services: Australia-Korea Remote Teamwork”, which

viii

Preface

was funded by the Australian Department of Foreign Affairs and Trade (DFAT) and its Australia-Korea Foundation (AKF). The UNSW Scientia Program and the ARC (DP230100605) also supported the symposia and completion of this work. The support from all of these sources is gratefully acknowledged. Sydney, Australia Sydney, Australia Seoul, Republic of Korea (South)

Ju Hyun Lee Michael J. Ostwald Mi Jeong Kim

Contents

Part I

Collaboration

1

Multimodality and Architectural Collaboration . . . . . . . . . . . . . . . . . . Ju Hyun Lee

2

Online Design Processes and Communication Across Cultures: A Cognitive-Social-Technical (C-S-T) System Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ju Hyun Lee and Michael J. Ostwald

3

Evaluating the Use of Digital Technologies to Support Design Collaboration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ning Gu, Rongrong Yu, Kerry London, Zelinna Pablo, and Maria Roberts

4

Multimodality in Virtual Co-urban Design . . . . . . . . . . . . . . . . . . . . . . Shuva Chowdhury and Marc Aurel Schnabel

5

Hybrid Practice: Exploring the Complexities of Cross-Cultural Collaboration Through the Dialogue of Two International Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dijana Alic, Mladen Jadric, and Geun Ju Yoon

Part II

3

15

39

65

79

Technology

6

Multimodality and Architectural Technology . . . . . . . . . . . . . . . . . . . . Michael J. Ostwald

99

7

Multimodal Collaborative Design in an Immersive Virtual Environment: Opera Staging and Performance Design Using iModel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 Michael J. Ostwald, Susanne Thurow, Dennis Del Favero, and Michael Scott-Mitchell

ix

x

Contents

8

The Museum of Touch: Tangible Models for Blind and Low Vision Audiences in Museums . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 Dagmar Reinhardt, Leona Holloway, Jane Thogersen, Eve Guerry, Claudio Andres Corvalan Diaz, William Havellas, and Philip Poronnik

9

How to Enhance Architectural Visualisation Using Image Gen AI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157 Jin-Kook Lee, Hyun Jeong, Youngchae Kim, Suhyung Choi, Hayoung Jo, Sumin Chae, and Youngjin Yoo

10 Development of Collective Intelligence for Building Energy Efficiency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175 Peichun Xiao, Lan Ding, and Deo Prasad Part III Education 11 Multimodality and Architectural Education . . . . . . . . . . . . . . . . . . . . . 199 Mi Jeong Kim, Ju Hyun Lee, and Michael J. Ostwald 12 Urban and Architectural Design Education Using the IC-PBL Model on a Virtual Platform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 211 Hyungmo Yang and Mi Jeong Kim 13 The Impact of Visual Character on Perceived Stress Levels: An Intelligent Approach Applied to University Campus Design . . . . 229 Zhixian Li, Xiaoyi Zu, Ju Hyun Lee, and Michael J. Ostwald 14 Design Education and Learner Experiences in a Computer-Mediated Environment . . . . . . . . . . . . . . . . . . . . . . . . . . 245 Myung Eun Cho and Mi Jeong Kim 15 Pattern Languages: Concepts and Applications in Design for Ageing and Design Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263 Michael J. Dawes

About the Editors

Dr. Ju Hyun Lee is Associate Professor of Architecture and Computational Design at UNSW, Sydney. He has made significant contributions towards research in architectural computing and cognition. As a senior lecturer, he completed a five-year postdoctoral fellowship at the University of Newcastle and has held multiple academic roles in Australia and South Korea. He was a senior research fellow at UNISA in 2018. He is co-author with Michael J. Ostwald of Grammatical and Syntactical Approaches in Architecture (IGI Global 2020) and co-author with Michael J. Ostwald and Ning Gu of Design Thinking: Creativity, Collaboration and Culture (Springer 2020). Lee has been awarded eleven competitive research grants with a total value of over $11M (AUD) in Australia and South Korea, including two ARC Discovery projects. Prof. Michael J. Ostwald is Professor of Architectural Analytics at UNSW, Sydney. He has a Ph.D. in architectural history and theory and a D.Sc. in design mathematics and computing. Michael has previously been a Professorial Research Fellow at Victoria University Wellington, a visiting professor and research fellow at RMIT University, an ARC Future Fellow at Newcastle and a visiting fellow at ANU, MIT, HKU, and UCLA. He completed postdoctoral research on geometry at UCLA (Calif.), CCA (McGill, Montreal), and Harvard (Mass.). Michael is Editor-in-Chief of the Nexus Network Journal: Architecture and Mathematics (Springer) and on the editorial boards of ARQ (Cambridge) and Architectural Theory Review (Taylor and Francis). Prof. Mi Jeong Kim is Professor of the School of Architecture at Hanyang University in Korea. She received her Ph.D. from the Key Centre of Design Computing and Cognition at the University of Sydney and worked as a postdoc fellow at UC Berkeley before joining Kyung Hee University. She was previously a visiting fellow at NYU, MIT, and Curtin University. She is Editor-in-Chief of the Journal of the Korean

xi

xii

About the Editors

Institute of Interior Design and on the editorial boards of the International Journal of Architectural Research. Her research interests include sensing architecture, human– building interaction, design education, and strategies for creativity, smart homes, and communities.

Abbreviations

AEC AI ANOVA AR ARC BIM BLV CAD CAVE CI CoI CSCW C-S-T CT CTA CVE CWA EEG FCN FDM FMH GA Gen AI GIS H HCI HMD IC-PBL ICT IVE LIWC

Architecture, Engineering, and Construction Artificial intelligence Analysis of variance Augmented reality Australian Research Council Building information modelling Blind and low vision Computer-aided design Cave Automatic Virtual Environment Collective intelligence Community of Inquiry Computer-supported cooperative work Cognitive-social-technical Communication technology Cognitive task analysis Collaborative virtual environments Cognitive work analysis Electroencephalography Fully convolutional network Fused deposition modelling Faculty of Medicine & Health Genetic algorithm Generative artificial intelligence Geographic Information System Entropy Human–computer interaction Head-mounted display Industry-coupled problem-based learning Information and Communications Technology Immersive virtual environment Linguistic inquiry and word count xiii

xiv

LLM LMS LoRA LSM M ML MVCSD MXD NAS OBL PBL PSS RF SD SD SegNet SLA SLS SMM STS SVI TMM TUI UML VEP VR

Abbreviations

Large language model Learning Management System Low-rank adaptation Language style matching Mean Machine learning Multimodal Virtual Communication System for Co-Urban Design Mixed-use development Network-attached storage Object-based learning Problem-based learning Perceived stress score Random forest Stable diffusion Standard deviation Semantic segmentation model Stereolithography Selective laser sintering Shared mental model Sociotechnical system Street view imagery Team mental model Tangible user interface Unified modelling language Visual element proportion Virtual reality

Part I

Collaboration

Chapter 1

Multimodality and Architectural Collaboration Ju Hyun Lee

Abstract Architectural collaboration has been evolving in response to shifts in our sociotechnical systems. Particularly, multiple modes of digital communication and representation have become increasingly prevalent in recent design teamwork. Part I of this book offers a comprehensive understanding of architectural collaborative processes in the digital ecosystem as well as design discourses across diverse settings. Architectural practice is inherently collaborative, and collectively fosters creative solutions that ultimately influence both environments and society. This chapter provides a background to the multimodal aspects of “architectural collaboration” and proceeds to introduce four chapters consisting of three empirical, cognitive studies and one qualitative, practical report. Throughout Part I, multimodal architectural collaboration is closely discussed with empirical data, obtained from design experiments and practices, and their implications. This introductory chapter also discusses the four contributions in Part I with a modality–functionality model to facilitate the exploration of design collaboration and associated communications and interactions in architecture. Keywords Design collaboration · Design process · Communication · Representation

1.1 Introduction The rapid advancement in information and communication technologies is fundamentally reshaping our creative industry, redefining the way we create, access, and trade goods and services. This digital transformation also poses new challenges to our traditional methods of design collaboration in the Architecture, Construction and

J. H. Lee (B) School of Built Environment, Faculty of Arts, Design and Architecture, The University of New South Wales, Sydney, NSW 2052, Australia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 J. H. Lee et al. (eds.), Multimodality in Architecture, https://doi.org/10.1007/978-3-031-49511-3_1

3

4

J. H. Lee

Engineering (AEC) industry. Part I of this book introduces and explores the multimodal aspects of design collaboration in architecture, which not only encompasses digital design processes but also involves diverse cognitive approaches. Architectural practice revolves on the exchange of ideas among diverse stakeholders, requiring involvement in a multitude of complex processes and arrangements. Notably, emerging interactive and collective platforms in a digital ecosystem have supported these collaborative processes, in terms of digital collaboration (Elia et al. 2020), problem solving (Elia and Margherita 2018), and collective design (Stelzle et al. 2017). They may appear to replicate face-to-face design processes. However, it’s important to note that traditional sketch activities and gestures don’t function in the same way in the digital realm, especially across spaces (Lee and Ostwald 2022). Nonetheless, this shift towards digital design platforms has not only enhanced team performance and collaborative processes but has also opened up new possibilities for creativity and innovation in architecture. As such, multimodal, digital communications and interactions collectively generate creative solutions that have a profound impact on both the built environment and society. Indeed, it’s worth noting that a design outcome is a reflection of the collective vision and values of the collaborators involved in the creative processes, encapsulating their combined efforts, insights, and perspectives. In this context, ‘multimodality’ in Part I highlights cognitive, collaborative, and collective design activities in the creative, digital ecosystem. Indeed, state-of-the-art Computer Supported Cooperative Work (CSCW) creates new collaborative processes, tasks, roles, and relationships within a project team, enabling effective communications and interactions (Chiu 2002; Grudin 1994; Ibrahim and Pour Rahimian 2010, Lee et al. 2023). As such, advanced communication tools and digital design technologies has played a significant role in the evolution of architectural collaboration. In this context, the first contribution presented in Part I of this book explores cognitive, collaborative operations during remote design communication and coordination. By proposing a cognitive-social-technical (C-S-T) system, this protocol study not only captures the cognitive characteristics of collaborative design processes, but also provides a knowledge framework for teamwork in the digital era. Furthermore, a combined method of cognitive and linguistic research techniques is used to empirically examine design processes and communication across cultures. The second contribution also investigates the collaborative processes across three design environments (face-to-face, 3D modelling, and immersive). In addition to this comparative analysis, it investigates the collaborative design behaviours of two team types (experts with non-experts, and experts with other experts), identifying interesting differences in design collaborative practices related to ‘expertise’, ‘organising mechanism’, ‘problem solving’, and ‘shared goals’. The third research contribution further examines a multimodal co-design in virtual urban design, exploring the embodiment of collective intelligence (CI) in the remote design process. The participants’ collective decisions in the Multimodal Virtual Communication System for Co-Urban Design (MVCSD) are categorised by three CI requisites

1 Multimodality and Architectural Collaboration

5

such as communication, representation, and motivations. In contrast, the final contribution explores real design and communication processes between two geographically distributed architectural firms. It captures their collaborative thoughts and approaches, documented in a unique manner, and provides valuable insights into the dynamic and challenging processes faced by contemporary architectural practice as it operates in diverse and digital modes. The chapter starts by presenting an overview of multimodal architectural collaboration in the digital ecosystem and proceeds to further introduce the four contributions in Part I of this book—a series of protocol studies (Chaps. 2–4) and one practice research (Chap. 5). To facilitate comparison, the relationships between these studies are then discussed by the lens of a modality-functionality model.

1.2 Multimodality in Architectural Collaboration Architectural collaboration has traditionally revolved around creating and communicating a set of 2D drawings and specifications, often supplemented by physical models. However, contemporary practice has adapted to multimodal tools, technologies, and social dynamics, increasingly relying on 3D visualisation and digital networks. These changes have influenced the way architects and teams collaborate and create architectural designs. Furthermore, the growing adoption of emerging technologies across different disciplines has led to the increased level of complexity and automation in the AEC sector. Consequently, the need for an integrated digital platform has become imperative to facilitate enhanced collaboration and communication in the industry. For example, Building Information Modelling (BIM) not only introduces innovative ontologies and technologies, but also enhances the effectiveness of heterogeneous design communications and representations in design teamwork (Lee et al. 2023). As such, this multi-disciplinary collaboration platform highlights interactive and collective processes in dynamic design practices (Verstegen et al. 2019). In addition, it provides support for diverse stakeholders including architects, engineers, specialist consultants, contractors, project managers, policy makers, service providers, providers, and others (Gu et al. 2015). Importantly, successful design collaboration relies on effective communication and knowledge exchange among these various stakeholders, requiring multimodal tools as well as collective engagements throughout the lifecycle of a building project (Lee et al. 2023). Furthermore, this ‘multimodal collaboration’ may encompass ‘participatory design’ (Luck 2018; Smith and Iversen 2018), ‘co-creation’ (Prahalad and Ramaswamy 2004; Nonaka and Konno 1998) and ‘co-design’ (Mitchell et al. 2016; Huybrechts et al. 2017). Recent research on multimodality in architectural collaboration is closely associated with remote teamwork (Lee and Ostwald 2022). Effective collaboration without physical contact among designers working in different locations is crucial. However, the impacts of digital design modalities on this new teamwork process remain uncertain. A limited set of protocol studies have investigated and explored cognitive,

6

J. H. Lee

collaborative benefits and drawbacks in remote digital platforms. Specifically, Lee’s and Ostwald’s interactive communication and collaboration model (2022) describes the relationship between ‘communication’ and ‘collaboration’ in online digital design platforms. Obviously, communication is the foundation of collaboration, and collaboration, in turn, relies on effective communication. Communication supports ‘coideation’, ‘collectiveness’, and ‘communicative interactions’ in remote teamwork (Dorta 2008; Chowdhury and Schnabel 2020; Kim et al. 2020; Boudhraa et al. 2021). In contrast, digital design collaboration needs new types of ‘spatial cognition’, ‘design activities’, and ‘diverse interactions’ (Kim and Maher 2008; Gül and Maher 2009; Chowdhury and Schnabel 2020; Rahimian and Ibrahim 2011; Tang et al. 2011; Eris et al. 2014). Designers’ shared activities rely on the diverse interfaces of digital tools (Gül and Maher 2009), and multimodal design communication and representation (Eris et al. 2014). In contrast, collective information sharing and discussion are enhanced by problem framing and ideation (Kim et al. 2020) as well as social interactions and collaborative ideation (Boudhraa et al. 2021). In addition, ‘coordination’, which manages and develop a team cognition, stands as one of the fundamental components in multimodal architectural collaboration (Lee et al. 2023). Again, design interactions are rapidly shifting online, driven not only by the utilisation of digital design platforms but also within the context of the world’s expanding ‘digital ecology’. The notion of a digital ecology has been widely applied to many business models for the past two decades. It mirrors this omnipresence in which digital actors, such as customers, partners, and providers, along with their collective activities, are reshaping the entrepreneurial process (Elia et al. 2020). For example, in the AEC industry, BIM consisting of hundreds of participants and applications is regarded as an ecosystem where the products, processes, and people co-evolve (Gu et al. 2015). Furthermore, this interconnected digital platform empowers design teams to collaborate with enhanced effectiveness and efficiency, supported by both synchronous and asynchronous sharing of design concepts, 3D models, and project data across spaces. Thus, Part I of this book focuses on collaborative design processes in the digital ecosystem, encompassing collective and interactive platforms (Markus and Loebbecke 2013).

1.2.1 Online Design Processes and Communication Across Cultures Among various multimodal collaborative processes, communication and coordination processes in remote teamwork are the focus of Chap. 2. In “Online Design Processes and Communication across Cultures: A Cognitive-Social-Technical (CS-T) System Approach”, Lee and Ostwald present a novel C-S-T system model to precisely examine cognitive design processes as well as social and technical patterns across cultures and spaces. The knowledge model generates three combined systems,

1 Multimodality and Architectural Collaboration

7

(i) technical-cognitive system (TCS), (ii) cognitive-social system (CSS), and (iii) social-technical system (STS). As such, the core idea of Chap. 2 is that the addition of an interconnected cognitive system can overcome some limitations of the traditional STS, jointly optimising and managing complex collaboration systems in the digital era. To holistically analyse system complexities in this research, the cognitive characteristics of collaborative design processes are compared with the linguistic features of these processes. The dual viewpoints, cognitive and linguistic, not only capture communication and coordination patterns in remote teamwork, but also reveal the relationship between cognition and language use in architectural collaboration. Specifically, Lee and Ostwald use a combination of protocol analysis and Linguistic Inquiry and Word Count (LIWC). In this way, they identify four system categories (visual communication, design coordination, verbal communication, and task coordination) and highlights six linguistic categories (‘drives’, ‘cognition’, ‘affect’, ‘social’, ‘perception’, and ‘conversation’). Lee and Ostwald note that there is a possible correlation between ‘task coordination’ activities and ‘social’ terms; between ‘verbal communication’ and ‘conversation’; between ‘design coordination’ and ‘perception’. Furthermore, two indexes—cognitive complexity (H) and Language Style Matching (LSM)—are also measured to compare four teams’ design processes (two mixedcultural and two mono-cultural teams) with their communication styles. Lee and Ostwald suggest that there appears to be a negative relationship between H and LSM. The combined method offers a comprehensive insight into how a C-S-T process can be interconnected with design communication, especially in online teamwork across diverse cultures.

1.2.2 Digital Technologies and Design Collaboration Digital collaboration in the AEC industry is vital for complex design and construction projects that involve diverse stakeholders and digital technologies (Merschbrock and Munkvold 2015). Specifically, immersive design environments, including Virtual Reality (VR) and Augmented Reality (AR), offer advanced design visualisation and simulation capabilities, potentially impacting on design teamwork. The impact of various digital technologies on design collaboration is the focus of Chap. 3. To empirically address this research problem, a cognitive approach protocol analysis is employed. A coding scheme is also selectively developed from London’s and Pablo’s collaborative practice model (2021). In “Evaluating the use of digital technologies to support design collaboration”, Gu, Yu, London, Pablo, and Roberts present two sets of comparative protocol studies on digital design collaboration. The first comparison highlights collaborative processes in three design environments—(i) face-to-face sketching (baseline), (ii) 3D modelling in BIM enhanced with Zoom meetings, and (iii) immersive design in Hyve-3D. Across the three digital platforms, collaboration processes exhibit similarities in ‘organising mechanisms’, ‘problem solving’, and ‘shared goals’. In contrast, Gu

8

J. H. Lee

et al. present that there are distinct differences in ‘expertise’ and ‘shared space’. In contrast to the digital design settings, participants in the face-to-face mode more frequently rely on their expert knowledge or personal experience. The second part of Chap. 3 presents a comparative analysis of the coding results between two different team dynamics in design collaboration—(i) architects collaborating with end-users, and (ii) architects collaborating with urban planners/designers. Design collaboration between these two exhibits notable differences across the three design environments. The coding results for ‘expertise’, ‘organising mechanism’, and ‘problem solving’ are all significantly higher in the former teams as compared to the latter teams. In contrast, the teams consisting solely of designers display a higher frequency of establishing ‘shared goals’ compared to the other teams. In Chap. 3, Gu et al. highlight that the immersive design environment exhibits similarities to the 3D modelling environment.

1.2.3 Multimodality in Virtual Co-urban Design The relationship between ‘CI’ and virtual ‘co-design’ is mutually beneficial and reinforcing. Individuals from different backgrounds bring forth distinctive insights and ideas that promote and stimulate the creation of innovative solutions in the digital ecosystem. This idea forms the foundation of Chap. 4, which undertake a comparison of collaborative design processes in three collaborative design sessions—(i) face-toface with remote sketching, (ii) face-to-face with a 3D virtual world, and (iii) faceto-face with 3D virtual world sketching. Like Chap. 3, in “Multimodality in Virtual Co-Urban Design”, Chowdhury and Schnabel adopt protocol analysis to investigate how laypeople communicate and control design ideas in a virtual and immersive environment. By redefining their past coding schemes (Chowdhury and Schnabel 2020), in accordance with CI prerequisites (communication, representation and motivation) (Murty et al. 2010), Chowdhury and Schnabel examine the impact of collective intelligence on participants’ decision-making processes in the three MVCSD sessions. In Chap. 4, Chowdhury and Schnabel suggest that “communication happens through the exchange of information, ideas, knowledge, and options among individuals to achieve a shared goal or solve a problem”, revealing that communication intelligence is dominant in the three sessions. The representation of 3D artifacts establishes an association with design tasks, consequently attributing meaning to communication. The participants’ engagement with neighbourhood design tasks, coupled with immediate feedback on design actions, align with the CI requirements for motivation. The MVCSD system empowers participants to actively engage as co-designers, facilitating a substantial contribution to the collaborative, collective design processes.

1 Multimodality and Architectural Collaboration

9

1.2.4 Hybrid Architectural Design Practice Cross-cultural design processes and communication between two architectural practices are the focus of of Chap. 4. In “Hybrid Practice: Exploring the Complexities of Cross-cultural Collaboration through the Dialogue of Two International Practices”, Alic, Jadric, and Yoon report their challenges and experiences in this international collaboration for an award-winning project, ‘Seoul Photographic Art Museum’. By the use of narrative inquiry, Alic et al. provide a valuable example of the use of different modes of communication, across different cultural and linguistic groups, in a real collaborative design process. In Chap. 4, Alic et al. illustrate (i) the project brief and design response, (ii) the design process and communication, (iii) communication and engagement, (iv) cross-cultural collaborations, (v) Zoom and video communication, (vi) online design collaboration tools, and (vii) data management leveraging NAS servers and extensive documentation. They also discuss about the seamless integration of analogue and digital communication and collaboration, as well as multicultural challenges in contemporary architecture. Jadric states, “we have to accept, of course, different opinions and limits of such a kind of communication and cooperation because you have to acknowledge and accept differences”. Whilst highlighting the significance of “reflection-in-action” and “reflection-on-action”, Alic et al. offer a comprehensive insight into the complex interaction between individual experiences and the broader design processes in the cross-cultural architectural practice.

1.3 Discussion The characteristics of digital modalities in collaboration processes can be examined with key functionalities of interactive and collective platforms (Lee et al. 2020). Based on the ‘5W1H’ genome framework of CI (Malone et al. 2010), this chapter proposes the modality–functionality model of digital design collaboration that consists of three entities (collaborator, content, location and time) in Fig. 1.1. ‘Collaborator’ encompasses three types of stakeholders: hierarchy (e.g., a group of professionals within the same department), diversity (e.g., a group of professionals in diverse departments or with diverse backgrounds), and crowd (e.g., a broad range of users, some of whom may be unknown). It can be extended to have ‘artificial intelligence (AI) as a teammate’ (Seeber et al. 2020), collaborating with a person or diverse teams as well as engaging with the public. ‘Content’ considers collaborative creation and decision as well as collective creation and decision. Like AI, this axis can include “opportunistic” creation and decision, which are pervasive content using mobile tracking and sensors (Huang et al. 2017). Lastly, the ‘location and time’ axis identifies the capability to work in different locations (either local or global) and at various times (within time-bound constraints or at any time). In this way, the model

10

J. H. Lee

Fig. 1.1 The modality–functionality model of digital design collaboration

can be employed to discuss about the multimodality of architectural collaboration inherent in each contribution in Part I. As illustrated in the left chart of Fig. 1.2, participants in Lee’s and Ostwald’s research use a videoconferencing tool that enables ‘time-bound global’ and ‘collaborative decision’, but they are categorised into two types—mixed-cultural (diversity) and mono-cultural teams (hierarchy). With the increased multimodality, polyhierarchical users tend to produce more ‘visual communication’ and ‘verbal communication’ activities than hierarchical users. In contrast, hierarchical users tend to produce a relatively higher number of ‘planning’ (shared goals) and ‘explaining’ activities, which may be impacted by their assumed hierarchical roles. Likewise, Lee and Ostwald argue that mono-cultural teams tend to require a higher frequency of goaldriven language in design teamwork, whereas mixed-cultural teams can display more emotional and sentiment-based expressions. Gu et al.’s research examines the role of digital technologies in supporting design collaboration. The study not only compares collaborative processes across three design environments (Fig. 1.2) but also explores the impact of two different team dynamics. Similar to the findings in Lee’s and Ostwald’s research, it reveals that hierarchy-like teams (architects and planners) tend to generate a greater number of ‘shared goals’ activities compared to more diverse teams (architects and end-users). Interestingly, participants in the VR environment using Hyve-3D, which is limited to ‘time-bound local’ and ‘collaborative decision’ in Fig. 1.2b, produce a relatively higher number of ‘shared goals’ activities. Nonetheless, due to the increased level of multimodality, the videoconferencing is associated with the highest percentage of ‘shared space’ among the three environments in their research.

1 Multimodality and Architectural Collaboration

11

Fig. 1.2 The multimodalities of videoconferencing and VR (or face-to-face collaboration)

Like Gu et al.’s research, Chowdhury and Schnabel examine urban design collaboration in a virtual and immersive environment but emphasise laypeople’ communication, representation, and motivation in the decision-making process. Thus, their MVCSD can be categorised as ‘crowd’, ‘collective decision’, and ‘time-bound local’ in Fig. 1.3a, although it also enables ‘face-to-face with remote sketching’, which is a ‘global’ mode of collaboration. The MVCSD is primarily designed for real-time design collaboration via face-to-face communication. However, its multimodality can be expanded to support the ‘anytime global’ aspect of CI. For the ‘location and time’ category, ‘anytime global’ collaboration has already been implemented in contemporary architectural practice. Alic et al.’s research describes cross-cultural architectural collaboration, characterised by a high level of multimodality that combines ‘diversity’, ‘collaborative decision’, and ‘anytime global’ in Fig. 1.3b. The hybrid practice is based on multimodal communications (synchronous and asynchronous) and complex design processes between two

Fig. 1.3 Multimodality in the MVCSD and hybrid practice

12

J. H. Lee

geographically distributed architectural firms. It can also take into account ‘participatory design’, making it adaptable to include ‘crowd’ and ‘collective decision’. In this context, the diverse perspectives and dynamic processes of international architectural collaboration are inherently complex, as illustrated by the elevated level of multimodality in the figure.

1.4 Conclusion This chapter has elucidated the concept of multimodality and its relevance to architectural collaboration, introducing the four contributions in Part I (Collaboration) of this book. It presents the ideas, methodologies, and findings in this collection of three cognitive studies and one architectural practice, unveiling the characteristics of collaborative communications and representations in the digital ecosystem. The chapter also introduces the modality–functionality model of architectural collaboration. The model is then employed to characterise and compare the four contributions, demonstrating their potential applications in the future. Consequently, this chapter paves the way for the exploration of multimodality in architectural collaboration, which is relatively new in architectural design research. This inaugural chapter of the book plays a pivotal role in setting the groundwork for the entire volume. That is, the conceptual foundations laid out in this chapter are instrumental for examining the level of multimodality in the other research contributions in Part II (Technology) and Part III (Education).

References Boudhraa S, Dorta T, Milovanovic J, Pierini D (2021) Co-ideation critique unfolded: an exploratory study of a co-design studio ‘crit’ based on the students’ experience. CoDesign 17(2):119–138. https://doi.org/10.1080/15710882.2019.1572765 Chiu M-L (2002) An organizational view of design communication in design collaboration. Des Stud 23(2):187–210. https://doi.org/10.1016/S0142-694X(01)00019-9 Chowdhury S, Schnabel MA (2020) Virtual environments as medium for laypeople to communicate and collaborate in urban design. Archit Sci Rev 63(5):451–464. https://doi.org/10.1080/000 38628.2020.1806031 Dorta T (2008) Design flow and ideation. Int J Archit Comput 6(3):299–316. https://doi.org/10. 1260/1478-0771.6.3.299 Elia G, Margherita A (2018) Can we solve wicked problems? A conceptual framework and a collective intelligence system to support problem analysis and solution design for complex social issues. Technol Forecast Soc Chang 133:279–286. https://doi.org/10.1016/j.techfore. 2018.03.010 Elia G, Margherita A, Passiante G (2020) Digital entrepreneurship ecosystem: how digital technologies and collective intelligence are reshaping the entrepreneurial process. Technol Forecast Soc Chang 150:119791. https://doi.org/10.1016/j.techfore.2019.119791

1 Multimodality and Architectural Collaboration

13

Eris O, Martelaro N, Badke-Schaub P (2014) A comparative analysis of multimodal communication during design sketching in co-located and distributed environments. Des Stud 35(6):559–592. https://doi.org/10.1016/j.destud.2014.04.002 Grudin J (1994) Computer-supported cooperative work: history and focus. Computer 27(5):19–26. https://doi.org/10.1109/2.291294 Gu N, Singh V, London K (2015) BIM ecosystem: the coevolution of products, processes, and people. In: Building information modeling. Wiley, pp 197–210 Gül LF, Maher ML (2009) Co-creating external design representations: comparing face-to-face sketching to designing in virtual environments. CoDesign 5(2):117–138. https://doi.org/10. 1080/15710880902921422 Huang Y, Shema A, Xia H (2017) A proposed genome of mobile and situated crowdsourcing and its design implications for encouraging contributions. Int J Hum Comput Stud 102:69–80. https:// doi.org/10.1016/j.ijhcs.2016.08.004 Huybrechts L, Benesch H, Geib J (2017) Institutioning: participatory design, co-design and the public realm. CoDesign 13(3):148–159. https://doi.org/10.1080/15710882.2017.1355006 Ibrahim R, Pour Rahimian F (2010) Comparison of CAD and manual sketching tools for teaching architectural design. Autom Constr 19(8):978–987 Kim MJ, Maher ML (2008) The impact of tangible user interfaces on spatial cognition during collaborative design. Des Stud 29(3):222–253. https://doi.org/10.1016/j.destud.2007.12.006 Kim MJ, Hwang YS, Hwang HS (2020) Utilising social networking services as a collective medium to support design communication in team collaboration. Archnet-IJAR: Int J Architectural Res 14(3):409–421. https://doi.org/10.1108/ARCH-02-2020-0025 Lee JH, Ostwald MJ (2022) The impacts of digital design platforms on design cognition during remote collaboration: a systematic review of protocol studies. Heliyon 8(11). https://doi.org/10. 1016/j.heliyon.2022.e11247 Lee JH, Ostwald MJ, Ning G (2020) Design thinking: creativity, collaboration and culture. Springer International Publishing, Cham, Switzerland Lee JH, Ostwald MJ, Arasteh S, Oldfield P (2023) BIM-enabled design collaboration processes in remote architectural practice and education in Australia. J Archit Eng 29(1):05022012. https:// doi.org/10.1061/JAEIED.AEENG-1505 London KA, Pablo Z (2021) Simulation-based collaboration training: strengthening the industry’s capacity to collaborate. In: Underwood J, Shelbourn M (eds) Handbook of research on driving transformational change in the digital built environment. IGI Global Publishing, pp 404–429 Luck R (2018) Participatory design in architectural practice: changing practices in future making in uncertain times. Des Stud 59:139–157. https://doi.org/10.1016/j.destud.2018.10.003 Malone TW, Laubacher RJ, Dellarocas C (2010) The collective intelligence genome. MIT Sloan Manag Rev 51(3):21–31 Markus ML, Loebbecke C (2013) Commoditized digital processes and business community platforms: new opportunities and challenges for digital business strategies. MIS Q 37(2):649–653 Merschbrock C, Munkvold BE (2015) Effective digital collaboration in the construction industry— a case study of BIM deployment in a hospital construction project. Comput Ind 73:1–7. https:// doi.org/10.1016/j.compind.2015.07.003 Mitchell V, Ross T, May A, Sims R, Parker C (2016) Empirical investigation of the impact of using co-design methods when generating proposals for sustainable travel solutions. CoDesign 12(4):205–220. https://doi.org/10.1080/15710882.2015.1091894 Murty P, Paulini M, Maher ML (2010) Collective intelligence and design thinking. DTRS’10: design thinking research symposium, Indiana, USA Nonaka I, Konno N (1998) The concept of “Ba”: building a foundation for knowledge creation. Calif Manage Rev 40(3):40–54. https://doi.org/10.2307/41165942 Prahalad CK, Ramaswamy V (2004) Co-creating unique value with customers. Strategy Leadersh 32(3):4–9. https://doi.org/10.1108/10878570410699249

14

J. H. Lee

Rahimian FP, Ibrahim R (2011) Impacts of VR 3D sketching on novice designers’ spatial cognition in collaborative conceptual architectural design. Des Stud 32(3):255–291. https://doi.org/10. 1016/j.destud.2010.10.003 Seeber I, Bittner E, Briggs RO, de Vreede T, de Vreede G-J, Elkins A, Maier R et al (2020) Machines as teammates: a research agenda on AI in team collaboration. Inf Manag 57(2):103174. https:// doi.org/10.1016/j.im.2019.103174 Smith RC, Iversen OS (2018) Participatory design for sustainable social change. Des Stud 59:9–36. https://doi.org/10.1016/j.destud.2018.05.005 Stelzle B, Jannack A, Noennig JR (2017) Co-design and co-decision: decision making on collaborative design platforms. Procedia Comput Sci 112:2435–2444. https://doi.org/10.1016/j.procs. 2017.08.095 Tang HH, Lee YY, Gero JS (2011) Comparing collaborative co-located and distributed design processes in digital and traditional sketching environments: a protocol study using the function–behaviour–structure coding scheme. Des Stud 32(1):1–29. https://doi.org/10.1016/j.des tud.2010.06.004 Verstegen L, Houkes W, Reymen I (2019) Configuring collective digital-technology usage in dynamic and complex design practices. Res Policy 48(8):103696. https://doi.org/10.1016/j.res pol.2018.10.020

Chapter 2

Online Design Processes and Communication Across Cultures: A Cognitive-Social-Technical (C-S-T) System Approach Ju Hyun Lee

and Michael J. Ostwald

Abstract Architectural teams are increasingly reliant on remote and flexible working and, because of this, face a growing challenge around their collaborative operations across cultures and spaces. Central to this challenge is the knowledge gap between empirical evidence and practice with respect to cognitive, dynamic perspectives about online teamwork. To address this gap, this chapter proposes a new model for a cognitive-social-technical (C-S-T) system, which can be used to examine team design processes and communication in distributed teams. This model is demonstrated in this chapter using data from a protocol analysis of four design team videoconferences (two mixed-cultural and two mono-cultural teams). This data is used to examine the cognitive characteristics of collaborative design processes. In parallel, the linguistic features of these processes are measured using Linguistic Inquiry and Word Count and compared with the protocol analysis results. Specifically, a Language Style Matching (LSM) metric is used to examine the stylistic use of language among different designers. The dual cognitive and linguistic viewpoints reveal previously hidden communication and coordination patterns in online teamwork with varying cultural diversity. One finding is that LSM tends to have a negative impact on a team’s cognitive complexity. Acknowledging its limited sample size, this study contributes to knowledge about the ways architectural teams communicate their design values in online environments. Keywords Online collaboration · Protocol analysis · Sociotechnical system (STS) · Language style matching (LSM) · Design cognition · Linguistic inquiry and word count (LIWC)

J. H. Lee (B) · M. J. Ostwald School of Built Environment, Faculty of Arts, Design and Architecture, The University of New South Wales, Sydney, NSW 2052, Australia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 J. H. Lee et al. (eds.), Multimodality in Architecture, https://doi.org/10.1007/978-3-031-49511-3_2

15

16

J. H. Lee and M. J. Ostwald

2.1 Introduction As the Architecture, Engineering and Construction (AEC) industry becomes more interconnected on a global scale, effective collaboration across cultures and spaces has increasingly been identified as a challenge. The combination of rapid digitisation and remote and flexible working also poses a challenge to conventional methods of design creativity, collaboration, and cultural practices (Lee et al. 2020, 2023). Design thinking in digital environments has characteristics which differ from its traditional, pen-and-paper counterpart (Lee et al. 2015; Oxman 2006). Likewise, collaborative design thinking in online environments differs from collaborative design thinking in a conventional, co-located, face-to-face context (Lee and Ostwald 2022; Lee et al. 2020). Co-located design processes, like collaborative sketching, writing, and modelmaking, have also been relocated to online platforms in recent years. Moreover, many architectural firms have adopted a full or partial transition to remote working for the foreseeable future. These developments in online, multi-cultural design teams are the catalyst for this chapter. While previous empirical studies have developed new knowledge about online design teamwork, a significant knowledge gap is associated with understanding team cognition and communication in remote teams (Lee and Ostwald 2022). Furthermore, past research has been limited to a context- or domain-specific approach (Gu et al. 2011; Kim et al. 2020; Stempfle and Badke-Schaub 2002; Tang et al. 2011; Valkenburg and Dorst 1998). Complicating this situation, cross-cultural design collaboration is rarely discussed in past research (Lee et al. 2020). This is partly because research about online design collaboration involving multidisciplinary or multicultural teams requires a holistic approach that can be applied across diverse scenarios. In response, this chapter presents a new approach that can holistically examine complex interactions between cognitive, social, and technical factors within a dynamic and evolving system. This new approach builds on an accepted sociotechnical system (STS) model. Architects can access online information at any time and from any location in a technology-rich, dynamic design environment. ‘Mobile work’ (Chen and Nath 2008), ‘remote, flexible, agile, and smart working’ (Bednar and Welch 2020; Cuel et al. 2022), and ‘artificial intelligence (AI) as a teammate’ (Seeber et al. 2020) are core to changing workplace culture and global workforce in the digital era. Likewise, ‘smart working’ (Bednar and Welch 2020) has already been integrated into the AEC industry. To ensure the quality and productivity of teamwork using new technologies, a range of STS approaches have been adopted in the fields of computer science, human–computer interaction (HCI), and management information systems (Baxter and Sommerville 2010; Chen and Nath 2008; Cuel et al. 2022; Herrmann et al. 2022). An STS model addresses the mutual dependency of social (structure and people) and technical (technology and tasks) systems (Bostrom and Stephen Heinen 1977). It can also identify ‘sensitisation and awareness activities and constructive engagement’ (Baxter and Sommerville 2010), which are fundamental interactive and collective design collaboration processes.

2 Online Design Processes and Communication Across Cultures …

17

Although the STS theory has been widely used for examining technology-driven transformations in hundreds of organisations over the past seven decades, its use in exploring complex, online collaboration systems has been limited. This is why this chapter begins by proposing a novel cognitive-social-technical (C-S-T) system model to investigate the complex interactions between humans, society and machines during remote design communication and coordination. In this chapter, the C-S-T system model is applied to develop a coding scheme for protocol analysis, which is the first component of this research. This new coding scheme allows for a systematic examination of cognitive processes and social and technical patterns, providing valuable insights into the dynamics of remote design collaboration across cultures. As a counterpart to the new C-S-T system, psychological, social and semiotic researchers repeatedly note that ideas are intrinsically tied to the language in which they are constructed and expressed (Bonvillain 2010; Lewis 2012). However, the realisation that design language might shape design itself, and vice versa, has been less commonly observed (Dong 2009; Lee et al. 2016, 2019, 2020). There is still a gap in disciplinary knowledge about the linguistic attributes of collaborative design language and how they shape the design process and communication. The problem is not only that the globalised design teamwork is so reliant on linguistic skills, but also that language as a system is a reflection of the way people think and of their sociocultural values (Gleitman and Papafragou 2005), both of which are central to the process of design collaboration. Thus, to support the understanding developed in the C-S-T model, there also needs to be a way of quantitively examining the relationship between team cognition and language in distributed teams. This chapter uses a Linguistic Inquiry and Word Count (LIWC) method and software implementation to address this issue. Initially designed for investigating cognitive and emotional processes in disclosure (Francis and Pennebaker 1992; Francis and Pennebaker 1993), LIWC (pronounced ‘Luke’) is a text analysis application used in various domains such as psychology, social science, HCI, healthcare, and communication (Boyd et al. 2022; Tausczik and Pennebaker 2010). Empirical studies using LIWC have revealed attentional focus (pronouns and verb tense), emotionality (positive and negative emotions), social relationships, thinking styles (conjunctions, nouns, verbs, and cognitive mechanisms), and individual differences in a variety of experiments (Tausczik and Pennebaker 2010). Recently, several design studies have used LIWC to examine linguistic features in spoken or written design discourse. For example, Paletz et al. (2018) investigate the different linguistic features of mixedcultural and mono-cultural teams and their impacts on creativity, promotion and prevention. Salah et al. (2022) also uses LIWC to analyse storytelling in design talks, identifying psychologically meaningful language categories. In addition, LIWC software automatically provides quantitative data about the cognitive and social aspects of communication. This chapter analyses data from a protocol study of four design team videoconferences (two mixed-cultural and two mono-cultural teams) to demonstrate the C-S-T model. The findings from the protocol data were then compared with the results of the LIWC analysis. As such, this chapter examines the relationship between team

18

J. H. Lee and M. J. Ostwald

design processes (identified by the C-S-T system) and linguistic patterns (captured by LIWC) in online design collaboration across cultures. The following section provides a comprehensive explanation of the C-S-T system model. The methodology section elaborates on cognitive and linguistic research techniques, each of which underscores a key index, cognitive complexity (H) and Language Style Matching (LSM), respectively. The results of this chapter highlight team C-S-T characteristics and language use in the context of remote design collaboration. Finally, the chapter discusses the implications of these findings and potential further applications of the C-S-T model.

2.2 Cognitive-Social-Technical (C-S-T) System Model Since Trist’s, Bamforth’s, and Emery’s pioneering studies in the 1950s (Emery 1959; Trist 1981; Trist and Bamforth 1951), the sociotechnical system (STS) theory has evolved to reflect dynamic shifts in work, technology, and design methods (Davis et al. 2014). Today STS is regarded as providing a ‘sociotechnical view of man– machine systems’ (Lundberg and Johansson 2021), or systems where humans interact with technology (Bennett et al. 2018). Oosthuizen and Pretorius (2016) argue that humans, as both cognitive and social creatures, use technology to comprehend situations and facilitate decision-making processes. This cognitive process is critical to understanding complex STS in the digital era. However, past research that treats the cognitive process as a subsystem of the social system often overlooks the technical system (Kelly 1978; Pasmore et al. 1982). This trend may result in an independent or biased application of socio-technical principles, potentially missing the holistic view of the interplay between social and technical factors. Furthermore, the recent evolution of organisations introduces a novel dimension where machines function as teammates and actively engage in decision-making processes (Seeber et al. 2020). This added complexity necessitates the development of a new model that effectively incorporates AI into the STS framework (Makarius et al. 2020). This progressive step also aligns with the ongoing transformation in socio-technical thinking, as highlighted by Davis et al. (2014). In this context, a cognitive system is added as a pivotal component of both social and technical systems. As shown in Fig. 2.1, the C-S-T model consists of three key systems, cognitive, social, and technical, which are closely interconnected. Applying the C-S-T system requires joint optimisation of the three interdependent systems. In addition to the factors captured in STS models (Bostrom and Stephen Heinen 1977; Chen and Nath 2008; Leavitt 1964; Sackey et al. 2015; Seeber et al. 2020), the C-S-T model accommodates important insights about team cognition. Team cognition refers to “the set of team members’ cognitive operations, perceptions, reasoning, conscious thought and mental models, amongst other things” (Lee et al. 2020, p. 116). Team cognition in digital design environments can be explored using various models, including shared mental models (SMM) and team mental models (TMMs) (Badke-Schaub et al. 2007; Bierhals et al. 2007; DeChurch and

2 Online Design Processes and Communication Across Cultures …

19

Fig. 2.1 A cognitive-social-technical (C-S-T) system model

Mesmer-Magnus 2010; Stempfle and Badke-Schaub 2002). The cognitive system embodies the emergent and collective phenomena of these team mental models. For example, “emergence and sharedness” as well as “distributed knowledge” can have an impact on team coherence and performance (Banks and Millward 2009; DeChurch and Mesmer-Magnus 2010; Lee et al. 2020). In this way, team cognitive processes can be used to examine the relationship between a complex system’s social and technical aspects. The C-S-T system in Fig. 2.1 consists of six significant interacting variables: communication and process (cognitive), culture and people (social), and task and technology (technical). Through this network of sub-systems, the C-S-T model not only facilitates exploration of the complex interactions between people and technology in workplaces, ensuring ‘productivity’—like existing STS models (Sackey et al. 2015)—but also an examination of two more cognition-driven systems: cognitive and social systems (CSS) and technical and cognitive systems (TCS). That is, in addition to this conventional knowledge base, the C-S-T system model encodes, stores, and retrieves information to/from the two cognitive knowledge bases, CSS and TCS, which evolve during team ‘process’ and ‘communication’. The CSS is closely related to the emergent constructs of the collection of team members’ cognitions. Thus, it can provide a collective understanding of problem-solving in multicultural teams, highlighting ‘creativity’ and co-ideation. In contrast, the TCS for shared and distributed knowledge in the various working environments deals with the relationships between cognitions and digital technologies, addressing ‘performance’. In past STS research, cognitive work analysis (CWA) with a focus on ‘designing for adaptation’ has been widely used for implementing computer-based information

20

J. H. Lee and M. J. Ostwald

systems in organisations (Vicente 1999) and analysing activity in complex systems (Naikar et al. 2003, 2006). CWA comprises “models of the work domain, control tasks, strategies, social-organisational factors, and worker competencies in a single, integrated framework” (Vicente 1999, p. 5). Similarly, cognitive task analysis (CTA) is focused on “cognition in real-world contexts and professional practice at work” (Crandall et al. 2006, p. vii) in terms of STS. However, these past cognitive approaches have mainly been limited to modelling and analysing social or technical systems. For example, both CWA and CTA are largely focused on capturing knowledge about ‘work domain analysis’ in the traditional STS. Furthermore, CWA and CTA typically employ interview and observation techniques (Clark et al. 2008). The use of protocol analysis is less frequent in the cognitive approaches of STS because it demands significant training and commitment (Clark et al. 2008). However, when the focus shifts to examining in-depth cognitive aspects like thought processes and goal structures in a complex STS, think-aloud protocol analysis is referred for rigorously examining participants’ task performances (Pirolli and Card 2005; Schraagen et al. 2000; Seamster et al. 1993). Like ‘process tracing’ (Cooke 1994), the knowledge elicitation method can capture intricate cognitive interactions within the C-S-T system. In summary, this chapter uses protocol analysis, a rigorous empirical method for researching cognitive systems in the C-S-T system model.

2.3 Methodology Two research techniques—protocol analysis and LIWC—are employed in this chapter to understand and analyse team design process and communication during online design collaboration. Design experiments, undertaken by two teams with diverse cultural backgrounds and two without, provide the data for this chapter. Based on the C-S-T system model, protocol analysis identifies communication and coordination processes, while LIWC measures categorical word counts. Two indexes— cognitive complexity (H) and LSM—are also used to compare team design processes with communication styles. In this way, the combined method provides an in-depth understanding of how a C-S-T process may be related to design communication.

2.3.1 Data Collection Following a formal Human Ethics application and approval process (HC200352), collaborative design protocols were collected from consenting teams undertaking a masters-level architecture course at UNSW, Sydney. Participants used Building Information Modelling (BIM) and recorded at least four 30-min team meetings via videoconferencing tools such as MS Teams or Zoom. Figure 2.2 illustrates examples captured during the collaborative design meetings.

Fig. 2.2 Example materials captured during the design team meetings

2 Online Design Processes and Communication Across Cultures … 21

22

J. H. Lee and M. J. Ostwald

As a trigger for their design collaboration, participants were provided with a design brief focusing on ‘mixed-use development (MXD)’. While no specific site or client requirements were stipulated, team members were encouraged to select an appropriate site and develop a realistic proposal. In addition, each MXD complex was expected to possess a coherent design concept and/or specific design goals such as regeneration or sustainability. Teams were composed of four members each, with each member assuming responsibility for one of the functional building blocks of the MXD (convention centre, residential, commercial, and hotel). This chapter reports data for four teams’ remote collaborations (two mixedcultural and two mono-cultural teams). The mixed-cultural teams (T1 and T2) comprise a blend of Australian and international students, while the mono-cultural teams (T3 and T4) are exclusively composed of Indian students. The focus of the present chapter is the 2nd (T1, 3, 4) or 3rd (T2) meeting of each group, depending on the stage of the process and the quality of the recording. These meetings were typically concerned with the conceptual design stage, involving design activities such as site analysis, massing study, and allocating individual roles in the team.

2.3.2 Process–Protocol Analysis Protocol analysis is one of the most rigorous empirical methods used to examine cognitive processes in design teams (Cross and Clayburn Cross 1995; Dong 2005; Goldschmidt 1995; Lee and Ostwald 2022; Lee et al. 2020; Stempfle and BadkeSchaub 2002; Valkenburg and Dorst 1998). For example, Cross and Clayburn Cross (1995) highlight that the social process of design is closely interconnected with its cognitive and technical operations. Conceptually, their finding resonates with the way cognitive, social, and technical systems interact within the C-S-T system model depicted in Fig. 2.1. Recent protocol studies explore shared activities, design communication, social interaction, and ideation in remote, digital design environments (Boudhraa et al. 2021; Eris et al. 2014; Kim et al. 2020; Lee and Ostwald 2022; Tang et al. 2011). Many protocol studies follow five methodological mechanisms (Lee et al. 2020): 1. Experimental settings: Designing controlled environments to achieve research goals and objectives. 2. Verbalisation: Encouraging participants to express their thoughts and processes either concurrently (think-aloud) or retrospectively. 3. Transcription and segmentation: Transforming verbalised content into a written form, often organised into distinct segments (or episodes) for analysis, such as syntactic markers and conversational turns. 4. Encoding with coding schemes: Categorising and encoding the transcribed and segmented data using predefined coding schemes.

2 Online Design Processes and Communication Across Cultures …

23

5. Arbitration and intercoder reliability: Ensuring consistency and accuracy by involving multiple coders and assessing inter-coder reliability such as Krippendorff’s alpha (α) and Cohen’s kappa (κ). In this protocol study, participants were asked to use a videoconferencing tool to record their teamwork sessions, during which they naturally verbalised their thoughts and interactions. The researchers refined the protocols following automated transcription and segmentation of the collected protocol data. Throughout this refining process, various factors were considered, including individual speaker turns, extended pauses, and even visual communication cues. To encode the final protocols, a coding scheme was selectively drawn from Maher et al.’s work (2006) and Dorta et al.’s design conversation framework (2016). In addition, casual conversation (chat) is considered to capture trivial discussions. Following several iterations of the encoding process, the researchers refined the codes and introduced several additional ones, such as ‘discussing’, ‘reacting’ and ‘planning’. After this process, the final C-S-T coding scheme was developed (Table 2.1). It consists of four system categories (visual communication, design coordination, verbal communication, and task coordination) developed from the C-S-T system model. In TCS, ‘visual communication’ addresses a shared visual representation in design teamwork, while ‘design coordination’ involves the management of drawings or models in a team space. This visual, technological system facilitates the examination of cognitive system behaviours using digital tools. On the other hand, CSS highlights ‘verbal communication’ that identifies collaborative conversations during the design process. Lastly, STS deals with ‘task coordination’ that organises and assigns tasks to achieve shared team goals. During the arbitration processes, the first author and another coder collaborated to review the data jointly and address any conflicts identified between them. Finally, after a period of more than three months, the first author undertook another round of encoding, in accordance with Table 2.1 and generated the definitive coding data. During this final round, two codes (‘setting’ and ‘clarifying’) were added to the coding scheme. The average Krippendorff’s alpha (α) value for the last two coding Table 2.1 C-S-T coding scheme System

System category

Code

Technical-cognitive system (TCS) Visual communication Creating, modifying, inspecting, analysing, evaluating Design coordination

Setting, positioning, examining, resolving

Cognitive-social system (CSS)

Verbal communication Proposing, constraining, questioning, discussing, clarifying, reacting, deciding, moving

Social-technical system (STS)

Task coordination

Planning, explaining, reminding assigning, negotiating, accepting

Casual conversation

Chat

24

J. H. Lee and M. J. Ostwald

results is 0.58 (T1: 0.53, T2: 0.59, T3: 0.54, T4: 0.65), which is below the lower cut-off value of 0.67 for reliability (Krippendorff 2018). Nevertheless, considering the iterative revisions during the encoding process and the fact that lower α values are to be expected in complex social interactions, this intercoder reliability is deemed acceptable.

2.3.3 Cognitive Complexity Cognitive complexity (H) as a team cognition index is measured by a degree of randomness exhibited by system categories in a C-S-T system. These system categories can be understood as the set of all possible states of the C-S-T system. That is, the probabilities of system categories can be used to measure the uncertainty of cognitive interactions in a system. In terms of Shannon’s information theory (1948), system categories in Table 2.1 are the “variables” or “events” that form a team’s C-ST system. The mathematical formula for computing entropy (H) follows Shannon’s (1948) equation: H =−

n 

pi log pi

(2.1)

i=1

where pi is the probability of a complex system being in state i. The selection of a logarithmic base establishes the unit used to quantify information. In the case of a base of 2, the unit is termed binary digits (bits), which is adopted in this chapter. For example, in a team design protocol, each segment can be encoded using the initials of three codes of a C-S-T system—Cognitive (C), Social (S), and Technical (T). An example of a sequence of C-S-T codes in a teamwork process is: C S T S C S T C S C S C S C T C T C S T. In this example, the possibilities of C, S and T are 8/20, 7/20 and 5/20, respectively. Collectively, the cognitive complexity (H) of the given design process is: H = −HC − HS − HT 7 5 8 7 5 8 − log2 − log2 H = − log2 20 20 20 20 20 20 H = −(−0.5288) − (−0.5301) − (−0.5000) H = 1.5589 (bits)

2 Online Design Processes and Communication Across Cultures …

25

Table 2.2 Language dimensions and categories of LIWC (Boyd et al. 2022) Dimension

Category (sub-category)

Word count

Analytical thinking, clout, authentic, emotional tone, word per sentence, big words, dictionary word

Linguistic dimension

Total function words (total pronouns, total function words), determiners (articles, numbers), prepositions, auxiliary verbs, adverbs, conjunctions, negations, common verbs, common adjectives, quantities

Psychological process

Drives (affiliation, achieve, power), cognition (all-or-none, cognition process, memory), affect (positive tone, negative tone, emotion, swear word), social process (social behaviour, social referents)

Expended dictionary

Culture (politics, ethnicity, technology), lifestyle (leisure, home, work, money, religion), physical (health, substances, sexual, food, death), states (need, want, acquire, lack, fulfil, fatigue), motives (reward, risk, curiosity, allure), perception (attention, motion, space, visual, auditory, feeling), time orientation (time, past focus, present focus, future focus), conversation (netspeak, assent, nonfluencies, fillers)

2.3.4 Communication—LIWC To investigate team communication in a rigorous and repeatable way, this chapter employs LIWC software—LIWC-22. This text analysis tool is widely used to analyse cognitive, emotional and language processes in various types of protocols such as speech and formal writing (Francis and Pennebaker 1992; Francis and Pennebaker 1993). In this chapter LIWC provides a means of capturing the features of design conversations by identifying psychologically meaningful language categories (Boyd et al. 2022). This software can identify multiple categories and more than 100 properties of text, as described in Table 2.2. Each category within LIWC-22 is constructed from a compilation of dictionary words chosen to encapsulate that specific feature. Using advanced linguistic analysis algorithms, LIWC-22 can automatically generate and classify quantitative data in terms of psychological and social aspects of communication (Boyd et al. 2022). The present application of LIWC focuses on the last two dimensions, ‘psychological process’ and ‘expended dictionary’, because they encompass cognitive and social significance.

2.3.5 Language Style Matching (LSM) Language Style Matching (LSM), which generates a metric that characterises an individual’s linguistic style, is employed as an indicator of social dynamics within a team (Gonzales et al. 2010). LSM is a factor of the extent to which each team member uses nine main function-word categories—auxiliary verbs, articles, common adverbs, personal pronouns, indefinite pronouns, prepositions, negations, conjunctions, and quantifiers (Gonzales et al. 2010; Ireland and Pennebaker 2010). For instance, the

26

J. H. Lee and M. J. Ostwald

LSM score for auxiliary verbs (av) between Persons 1 and 2 (LSM av ) is calculated as follows: L S Mav = 1 − (|av1 − av2|(av1 + av2))

(2.2)

A team’s LSM is determined as the average value of all team members’ LSM scores. Using LIWC, this chapter automatically calculates the LSM values of four team protocols that have transcriptions, team IDs and speaker IDs. Importantly, LSM can be used to predict a team’s cohesiveness and task performance (Gonzales et al. 2010). In this context, theoretically, cognitive complexity (H) is negatively related to LSM.

2.4 Results 2.4.1 Team C-S-T Process Table 2.3 reports the coding results of C-S-T activities (codes) for four team’s collaboration sessions (the percentage of the frequency weighted by time). In the four team meetings, the dominant system category produced was ‘task coordination’, while they developed the smallest volume of ‘visual communication’. The outcomes were unsurprising because they were focused on site analysis and task allocation during the conceptual design stage. On average, the ‘visual communication’ code accounts for 2.78%; the ‘design coordination’ accounts for 13.64% with three dominant codes (‘examining’: 7.40%, ‘setting’: 3.93%, ‘positioning’: 2.00%); the ‘verbal communication’ for 30.68% with three dominant codes (‘questioning’: 9.00%, ‘discussing’: 5.63%, ‘clarifying’: 5.16%); and the ‘task coordination’ for 55.62% with three dominant codes (‘negotiating’: 17.20%, ‘reminding’: 12.78%, ‘planning’: 8.92%). In remote design collaboration, designers tended to negotiate tasks, roles, or goals (‘negotiating’), remind or question tasks, roles, or goals (‘reminding’), and plan or create tasks, roles, or shared goals (‘planning’). Both mixed-cultural teams produced a relatively high number of ‘inspecting’ (3.57% and 1.49%, respectively) and ‘positioning’ (5.11% and 2.04%, respectively) activities. The mixed-cultural teams also produced more ‘visual communication’ and ‘verbal communication’ levels of design activities than the mono-cultural teams. In contrast, both mono-cultural teams produced a relatively higher number of ‘planning’ (14.22% and 12.54%, respectively) and ‘explaining’ (11.25% and 11.41%, respectively) activities. This result might reflect their collaboration strategy, where one or two designers in the mono-cultural often assumed leadership roles during meetings and were consequently focused on task and goal planning. In addition to these cognitive differences between mixed-cultural and monocultural teams, T1 and T3 produced larger amounts of ‘examining’ activities (10.56%

2 Online Design Processes and Communication Across Cultures …

27

Table 2.3 C-S-T coding results (the percentage of time duration)

Chat TCS Visual communication

TCS Design coordination

CSS Verbal communication

STS Task coordination

Mixed-cultural

Mono-cultural

T1

T2

T3

T4

3.19

2.81

1.32

2.79

Mean

SD

2.53

0.83

Creating

0.66

0.33

0.55

–

0.39

0.29

Modifying

0.71

–

–

–

0.18

0.36

Inspecting

3.57

1.49

0.88

0.78

1.68

1.30

Analysing

0.88

0.22

–

–

0.28

0.42

Evaluating

0.49

0.55

–

–

0.26

0.30

Setting

3.52

4.57

7.61

–

3.93

3.14

Positioning

5.11

2.04

0.83

–

2.00

2.24

Examining

10.56

7.1

10.36

1.57

7.40

4.20

Resolving

–

–

1.27

–

0.32

0.64

Proposing

3.02

3.19

–

–

1.55

1.79

Constraining

–

–

–

–

–

–

Questioning

8.96

8.87

6.36

9.00

2.22

Discussing

5.17

10.96

6.12

0.26

5.63

4.39

Clarifying

7.75

2.86

3.31

6.71

5.16

2.44

Reacting

4.18

2.81

4.52

2.96

3.62

0.86

Deciding

–

–

0.22

–

0.06

0.11

Moving

0.22

0.17

1.32

–

0.43

0.60

Planning

6.98

1.93

14.22

12.54

8.92

5.59

Explaining

4.12

8.04

11.25

11.41

8.71

3.43

Reminding

8.8

11.01

8.21

23.08

12.78

6.97

Assigning

11.8

6.82

1.54

7.72

3.66

4.94

2.86

Negotiating

12.64

27.53

7.55

21.08

17.20

8.86

Accepting

2.64

1.98

0.94

6.79

3.09

2.57

and 10.36%, respectively). That is, both teams frequently addressed conflicts or issues that arose among drawings or models within their designated team spaces. In contrast, T4 rarely engaged in such coordinating discussions or visual communication. Rather, the second mono-cultural team exhibited an excessive level of ‘task coordination’ activities, producing the highest percentages of ‘reminding’ (23.08%) and ‘examining’ (11.41%) and the second highest percentage of ‘negotiating’ (21.08%). In summary, the results indicate that the coding scheme developed in this chapter effectively enables the distinct recognition of four system categories in the C-S-T systems. Figure 2.3 illustrates the system categories of the four teams’ meetings over time. Like the coding results in Table 2.3, the primary differences are that in the monocultural team meetings, the ‘task coordination’ activities were dominant in volume

28

J. H. Lee and M. J. Ostwald

and occurred more consistently over time. Specifically, T4’s team meeting was predominantly focused on ‘task coordination’ activities. They talked about design coordination on their site only once throughout the meeting. Conversely, the ‘task coordination’ dialogues in the two mixed-cultural teams appeared to be reinforced by ‘verbal communication’ activities such as ‘proposing’, ‘discussing’, and ‘clarifying’. During their meetings, the ‘design coordination’ and ‘visual communication’ activities in the TCS also occurred between the CSS activities. Interestingly, T3’s team meeting displayed similar cognitive patterns observed in the mixed-cultural teams, although the mono-cultural team produced ‘design coordination’ activities more consistently over time. Compared to T4’s meeting, T3’s meeting in Fig. 2.3 showcases a distinct cognitive pattern that evolved over time. This disparity could be attributed to T3’s design activities being focused on the creation and testing of REVIT worksets for design coordination, a process with similarities to T2’s teamwork (see also Fig. 2.2).

Fig. 2.3 Four C-S-T categories of four teams’ meetings over time

2 Online Design Processes and Communication Across Cultures …

29

Table 2.4 Categorical LIWC results (the percentage of total words related to each category) Category Drives

Mixed-cultural

Mono-cultural

T1

T3

T2

Mean

SD

T4

4.59

4.53

5.08

6.17

5.09

0.76

18.48

18.63

17.95

20.75

18.95

1.23

Affect

2.05

2.27

1.26

1.42

1.75

0.49

Social

8.77

9.50

9.08

10.65

9.50

0.82

Perception

8.77

7.95

9.77

8.04

8.63

0.84

Conversation

5.80

3.61

4.00

3.85

4.32

1.00

Cognition

2.4.2 Team Communication Table 2.4 shows the LIWC results of the four team’s protocols. Excluding minor categories (those under 1%), the data describes the results of total words related to six categories, ‘drives’, ‘cognition’, ‘affect’, ‘social’, ‘perception’, and ‘conversation’. On average, the most prevalent category generated by the four teams was ‘cognition’ (19.95%), followed by ‘social’ (9.50%) and ‘perception’ (8.63%). As such, across the meetings, designers tended to produce words related to ‘cognitive process’ such as ‘insight’ (e.g., know, how, think), ‘discrepancy’ (e.g., would, can, want), ‘tentative’ (if, or, something), and ‘differentiation’ (e.g., but, not, or). Interestingly, the two mono-cultural teams used more words associated with ‘drives’ (5.08% and 6.17%, respectively) than the mixed-cultural teams. That is, they produced more terms related to ‘affiliation’ (e.g., we, our, help) and ‘achievement’ (e.g., work, better). In contrast, the two mixed-cultural teams used more words associated with ‘affect’ than the monocultural teams. That is, they produced more words related to ‘positive tone’ (e.g., good, well, love) and ‘positive emotion’ (e.g., good, happy, hope). These differences might be caused by their different team formations. Teams with a single cultural background might require a higher frequency of goal-driven language during their conversations, whereas mixed-cultural teams might need to display emotional and sentiment-based expressions in their communications. While the small sample size precludes a deeper analysis, this application confirms the potential benefit of using LIWC in the analysis of protocols.

2.4.3 The Relationship Between Process and Communication The results reported in this chapter not only highlight the differences across the different cultures present in the design teams, in terms of the C-S-T systems, but also suggest the relationship between C-S-T processes and communication in online teamwork. Figure 2.4a (upper figure) illustrates the coding results of four system categories across the four teams, while Fig. 2.4b (lower figure) illustrates the LIWC

30

J. H. Lee and M. J. Ostwald

results for ‘social’, ‘conversation’ and ‘perception’. While both figures depict distinct attributes (process and communication), their ranking orders collectively demonstrate a possible correlation between ‘task coordination’ activities and ‘social’ terms; between ‘verbal communication’ and ‘conversation’; between ‘design coordination’ and ‘perception’. This finding is both novel and conceptually acceptable. Table 2.5 describes the results of cognitive complexity (H) and LSM, which serve as an index of cognition and communication, respectively. T1 exhibited the highest level of cognitive complexity (1.7644), whereas T4 demonstrated the lowest (0.8322). This aligns with the fact that T4 displayed an elevated use of ‘task coordination’ activities. In contrast, designers in T2 and T4 communicated to each other in a notably similar style (0.94 and 0.93, respectively), while the four designers in T1 exhibited the least LSM (0.84). Collectively, there appears to be a negative relationship between cognitive complexity (H) and LSM, which also confirms the intuitive understanding

Fig. 2.4 Characteristics of C-S-T process and communication

2 Online Design Processes and Communication Across Cultures …

31

Table 2.5 Results of cognitive complexity (H) and LSM Mixed-cultural

Mono-cultural

Mean

SD

0.8322

1.4259

0.4082

0.93

0.90

0.05

T1

T2

T3

T4

H

1.7644

1.5406

1.5663

LSM

0.84

0.94

0.89

of the two mentioned previously. However, considering the results of T2 and T3, their relationship might be only marginal.

2.5 Discussion 2.5.1 C-S-T System Model and Its Applications The existing STS is comprised of two systems (social and technical), with cognition notably entrenched within a social system. However, as machines acquire the capability to emulate human decision-making and creation (Makarius et al. 2020; Seeber et al. 2020), technical systems can also directly link to cognitive process and communication. Thus, it is essential to consider a cognitive system as a pivotal component in the new era of AI. The C-S-T system model in Fig. 2.1, encompassing the interplay of three interdependent systems (cognitive, social, and technical), contributes to overcoming the current limitations of the STS paradigm and advancing organisational design and performance in the digital ecosystem. However, at the same time, the C-S-T system model adds another layer of complexity to the already intricate STS, requiring a multidisciplinary understanding of cognitive science, social science, and technical fields. Research on a cognitive system can also be time-consuming and resource-intensive. In this context, it is essential to customise the C-S-T model in diverse domains, while maintaining its core principles. As demonstrated in this chapter, the application of the C-S-T system model to design research shows promise. One notable application is its potential use as a knowledge framework for teamwork, wherein each system (‘C’, ‘S’, and ‘T’) is regarded as a key entity for modelling collaborative design processes in the digital era. That is, each system as an ontological category encompasses a distinct set of communicative, collaborative, and coordinative design activities. For example, the ‘C’ system encompasses a wide range of thinking processes and functions that contribute to problem-solving, decision-making, creativity, memory, reasoning, and knowledge in design collaboration. The ‘S’ system is focused on complex interpersonal interactions, communication dynamics, collaborative processes, and cultural considerations. Lastly, the ‘T’ system facilitates efficient design collaboration by managing tools, structured workflows, methodologies, and physical and digital

32

J. H. Lee and M. J. Ostwald

resources. In summary, the C-S-T approach can examine the interplay between cognitive processes, social interactions, and technical resources, providing a more holistic understanding of system behaviours in digital design collaboration. Past cognitive studies have investigated asynchronous (at any time) and dispersed (at any location) design collaboration in technology-rich, dynamic design environments, largely focusing on creativity and productivity in the digital ecosystem (Lee and Ostwald 2022; Lee et al. 2020). In contrast, this chapter has uncovered the presence of CSS and TCS through the use of protocol analysis in the domain of digital design collaboration, focusing on team ‘process’ and ‘communication’. The other interconnected factors such as ‘structure or culture’, ‘people’, ‘technology’, and ‘task’ in the C-S-T system model are not fully addressed in this chapter. In terms of traditional STS variables (Bostrom and Stephen Heinen 1977; Leavitt 1964), the four elements of ‘structure’, ‘people’, ‘technology’, and ‘task’ can not only exhibit interconnectedness in a complex system but also impact a team’s ‘process’ and ‘communication’. Furthermore, six interrelated factors—‘culture’, ‘goals’, ‘buildings and infrastructure’, ‘technology’, ‘processes and procedures’, and ‘people’— can jointly have an impact on the performance of an organisational system (Sackey et al. 2015). Specifically, ‘goals’ should be a crucial attribute influencing problemsolving behaviours in design teams (Stempfle and Badke-Schaub 2002). Likewise, in addition to ‘collaboration design’ and ‘institution design’, ‘machine artefact design’ would be an emerging design area for human–machine collaboration, where (i) appearance, (ii) sensing and awareness, (iii) learning and knowledge processing, (iv) conversation, (v) architecture, (vi) visibility and reliability are significant themes (Seeber et al. 2020).

2.5.2 Limitations and Future Directions The results of cognitive complexity (H) and LSM indicate that the multi-cultural teams, exemplified by T1 in this study, might exhibit higher levels of productivity and creativity compared to the mono-cultural teams. This advantage could be caused by the ability to draw upon diverse viewpoints and insights originating from various cultural backgrounds (Karlusch et al. 2018; Santandreu Calonge and Safiullin 2015). However, like many protocol studies, the findings of this research are constrained by the small sample size, limiting generalisability. The collected data do not represent the broader population of their cultural backgrounds. As such, this research is limited in the representation of diverse perspectives, as the sample does not represent all possible scenarios in online design processes and communications. Furthermore, the dynamics of remote design collaboration across cultures are not fully examined in this chapter. Collectively, this research is limited to the exploratory application of the C-S-T approach to collaborative design research. Future research is needed to elaborate and comprehensively test the C-S-T system model using various methods and scenarios across disciplines. For example, design communication is related to co-ideation (Boudhraa et al. 2021; Chowdhury and Schnabel 2020; Dorta 2008) or

2 Online Design Processes and Communication Across Cultures …

33

co-creation (Gül and Maher 2009), collectiveness (Kim et al. 2020; Wu and Duffy 2004) and communicative interactions (Eris et al. 2014; Kim et al. 2020). Spatial cognition (Kim and Maher 2008; Rahimian and Ibrahim 2011) and interactions with digital media (Gül and Maher 2009; Tang et al. 2011) are on-going research themes. A follow-up study can also employ other empirical research methods such as interview, observation, experiment, simulation, and reflection and theorising (Cross 2011). However, protocol analysis should still be one of the best research techniques to quantify cognitive processes in design teamwork, although encoding processes by researchers are time-consuming. LIWC may be regarded as an alternative, automatic technique as presented in this chapter. Qualitative content analysis (Lahti et al. 2004), dialogue analysis (Koutsabasis et al. 2012) and a video ethnographic approach (Christensen and Abildgaard 2018) can also be used for this purpose.

2.6 Conclusion This chapter has introduced a new C-S-T system model identifying two cognitiondriven systems (CSS and TCS). The results indicate that the cognition-driven systems can effectively capture design processes and communications during online teamwork. Building upon the traditional STS models, the addition of an interconnected cognitive system is valuable, offering effective decision-making and joint optimisation of complex collaboration systems. Furthermore, the C-S-T knowledge framework has potential implications for future design and multidisciplinary research. To provide a holistic understanding of system complexities, the C-S-T coding results are compared with those of LIWC. The C-S-T patterns can be naturally induced by using the team’s strategies and preferences, which are also consequences of cognitive and linguistic representations. Thus, this chapter seeks to gain a more comprehensive understanding of the relationship between cognition and language use in the context of remote teamwork. The combined method of cognitive and linguistic research techniques, which have rarely been applied before in a cohesive manner, offers an innovative way of examining the system behaviours within design teams operating online and across diverse cultures. Acknowledgements This research was supported by the ARC (DP230100605) and UNSW Scientia program.

References Badke-Schaub P, Neumann A, Lauche K, Mohammed S (2007) Mental models in design teams: a valid approach to performance in design collaboration? CoDesign 3(1):5–20. https://doi.org/10. 1080/15710880601170768

34

J. H. Lee and M. J. Ostwald

Banks AP, Millward LJ (2009) Distributed mental models: mental models in distributed cognitive systems. J Mind Behav 30(4):249–266 Baxter G, Sommerville I (2010) Socio-technical systems: from design methods to systems engineering. Interact Comput 23(1):4–17. https://doi.org/10.1016/j.intcom.2010.07.003 Bednar PM, Welch C (2020) Socio-technical perspectives on smart working: creating meaningful and sustainable systems. Inf Syst Front 22(2):281–298. https://doi.org/10.1007/s10796-019-099 21-1 Bennett KB, Bryant A, Sushereba C (2018) Ecological interface design for computer network defense. Human Factors 60(5):610–625. https://doi.org/10.1177/0018720818769233 Bierhals R, Schuster I, Kohler P, Badke-Schaub P (2007) Shared mental models—linking team cognition and performance. CoDesign 3(1):75–94. https://doi.org/10.1080/157108806011 70891 Bonvillain N (2010) Language, culture and communication: the meaning of messages. Pearson, New York Bostrom RP, Stephen Heinen J (1977) MIS problems and failures: a socio-technical perspective. Part I: the causes. MIS Q 1(3):17–32. https://doi.org/10.2307/248710 Boudhraa S, Dorta T, Milovanovic J, Pierini D (2021) Co-ideation critique unfolded: an exploratory study of a co-design studio ‘crit’ based on the students’ experience. CoDesign 17(2):119–138. https://doi.org/10.1080/15710882.2019.1572765 Boyd RL, Ashokkumar A, Seraj S, Pennebaker JW (2022) The development and psychometric properties of LIWC-22. University of Texas at Austin, Austin, TX, pp 1–47 Chen L, Nath R (2008) A socio-technical perspective of mobile work. Inf Knowl Syst Manag 7(1,2):41–60 Chowdhury S, Schnabel MA (2020) Virtual environments as medium for laypeople to communicate and collaborate in urban design. Architectural Sci Rev 63(5):451–464. https://doi.org/10.1080/ 00038628.2020.1806031 Christensen BT, Abildgaard SJJ (2018) The oscillation between individual and social designing in co-located student teams. CoDesign 1–21. https://doi.org/10.1080/15710882.2018.1557695 Clark RE, Feldon DF, Van Merrienboer JJG, Yates KA, Early S (2008) Cognitive task analysis. In: Handbook of research on educational communications and technology. Routledge, pp 577–593 Cooke NJ (1994) Varieties of knowledge elicitation techniques. Int J Hum Comput Stud 41(6):801– 849. https://doi.org/10.1006/ijhc.1994.1083 Crandall B, Klein GA, Hoffman RR (2006) Working minds: a practitioner’s guide to cognitive task analysis. Mit Press, Cambridge, MA Cross N (2011) Design thinking: understanding how designers think and work. Bloomsbury, Oxford, New York Cross N, Clayburn Cross A (1995) Observations of teamwork and social processes in design. Design Stud 16(2):143–170. https://doi.org/10.1016/0142-694X(94)00007-Z Cuel R, Cacciatore E, Ravarini A, Varriale L (2022) Do organizations need a head of remote work? In: Cuel R, Ponte D, Virili F (eds) Exploring digital resilience. Springer International Publishing, Cham Davis MC, Challenger R, Jayewardene DNW, Clegg CW (2014) Advancing socio-technical systems thinking: a call for bravery. Appl Ergon 45 (2, Part A):171–180. https://doi.org/10.1016/j.ape rgo.2013.02.009 DeChurch LA, Mesmer-Magnus JR (2010) The cognitive underpinnings of effective teamwork: a meta-analysis. J Appl Psychol 95(1):32–53. https://doi.org/10.1037/a0017328 Dong A (2005) The latent semantic approach to studying design team communication. Design Stud 26(5):445–461 Dong A (2009) The language of design: theory and computation. Springer, London Dorta T (2008) Design flow and ideation. Int J Architectural Comput 6(3):299–316. https://doi.org/ 10.1260/1478-0771.6.3.299 Dorta T, Kinayoglu G, Boudhraâ S (2016) A new representational ecosystem for design teaching in the studio. Des Stud 47:164–186. https://doi.org/10.1016/j.destud.2016.09.003

2 Online Design Processes and Communication Across Cultures …

35

Emery FE (1959) Characteristics of socio-technical systems. Doc. 527. Tavistock Institute of Human Relations, London Eris O, Martelaro N, Badke-Schaub P (2014) A comparative analysis of multimodal communication during design sketching in co-located and distributed environments. Des Stud 35(6):559–592. https://doi.org/10.1016/j.destud.2014.04.002 Francis ME, Pennebaker JW (1992) Putting stress into words: the impact of writing on physiological, absentee, and self-reported emotional well-being measures. Am J Health Promot 6(4):280–287. https://doi.org/10.4278/0890-1171-6.4.280 Francis ME, Pennebaker JW (1993) LIWC: linguistic inquiry and word count. Technical report. Southern Methodist University, Dallas, TX Gleitman L, Papafragou A (2005) Language and thought. Cambridge handbook of thinking and reasoning, pp 633–661 Goldschmidt G (1995) The designer as a team of one. Des Stud 16(2):189–209. https://doi.org/10. 1016/0142-694X(94)00009-3 Gonzales AL, Hancock JT, Pennebaker JW (2010) Language style matching as a predictor of social dynamics in small groups. Commun Res 37(1):3–19. https://doi.org/10.1177/009365020935 1468 Gu N, Kim MJ, Maher ML (2011) Technological advancements in synchronous collaboration: the effect of 3D virtual worlds and tangible user interfaces on architectural design. Autom Constr 20(3):270–278. https://doi.org/10.1016/j.autcon.2010.10.004 Gül LF, Maher ML (2009) Co-creating external design representations: comparing face-to-face sketching to designing in virtual environments. CoDesign 5(2):117–138. https://doi.org/10. 1080/15710880902921422 Herrmann T, Jahnke I, Nolte A (2022) A problem-based approach to the advancement of heuristics for socio-technical evaluation. Behav Inf Technol 41(14):3087–3109. https://doi.org/10.1080/ 0144929X.2021.1972157 Ireland ME, Pennebaker JW (2010) Language style matching in writing: Synchrony in essays, correspondence, and poetry. J Pers Soc Psychol 99(3):549–571. https://doi.org/10.1037/a00 20386 Karlusch A, Sachsenhofer W, Reinsberger K (2018) Educating for the development of sustainable business models: designing and delivering a course to foster creativity. J Cleaner Prod 179:169– 179. https://doi.org/10.1016/j.jclepro.2017.12.199 Kelly JE (1978) A reappraisal of sociotechnical systems theory. Hum Relat 31(12):1069–1099. https://doi.org/10.1177/001872677803101204 Kim MJ, Hwang YS, Hwang HS (2020) Utilising social networking services as a collective medium to support design communication in team collaboration. Archnet-IJAR: Int J Archit Res 14(3):409–421. https://doi.org/10.1108/ARCH-02-2020-0025 Kim MJ, Maher ML (2008) The impact of tangible user interfaces on spatial cognition during collaborative design. Des Stud 29(3):222–253. https://doi.org/10.1016/j.destud.2007.12.006 Koutsabasis P, Vosinakis S, Malisova K, Paparounas N (2012) On the value of virtual worlds for collaborative design. Des Stud 33(4):357–390. https://doi.org/10.1016/j.destud.2011.11.004 Krippendorff K (2018) Content analysis: an introduction to its methodology. Sage Publications, Thousand Oaks, CA Lahti H, Seitamaa-Hakkarainen P, Hakkarainen K (2004) Collaboration patterns in computer supported collaborative designing. Des Stud 25(4):351–371. https://doi.org/10.1016/j.destud. 2003.12.001 Leavitt HJ (1964) Applied organization change in industry: structural, technical and human approaches. In: Cooper WW, Leavitt HJ, Shelly MW (eds) New perspectives in organisational research. Wiley, New York, pp 55–71 Lee JH, MJ Ostwald (2022) The impacts of digital design platforms on design cognition during remote collaboration: a systematic review of protocol studies. Heliyon 8(11). https://doi.org/10. 1016/j.heliyon.2022.e11247

36

J. H. Lee and M. J. Ostwald

Lee JH, Gu N, Ostwald MJ (2015) Creativity and parametric design? Comparing designer’s cognitive approaches with assessed levels of creativity. Int J Des Creativity Innov 3(2):78–94. https://doi. org/10.1080/21650349.2014.931826 Lee JH, Gu N, Ostwald MJ (2019) Cognitive and linguistic differences in architectural design. Architectural Sci Rev 62(3):248–260. https://doi.org/10.1080/00038628.2019.1606777 Lee JH, Ostwald MJ, Arasteh S, Oldfield P (2023) BIM-enabled design collaboration processes in remote architectural practice and education in Australia. J Architectural Eng 29(1):05022012. https://doi.org/10.1061/JAEIED.AEENG-1505 Lee JH, Ostwald MJ, Gu N (2016) The language of design: spatial cognition and spatial language in parametric design. Int J Architectural Comput 14(3):277–288. https://doi.org/10.1177/147807 7116663350 Lee JH, Ostwald MJ, Gu N (2020) Design thinking: creativity, collaboration and culture. Springer International Publishing, Cham, Switzerland Lewis R (2012) Language, mind and nature. Cambridge University Press, Cambridge Lundberg J, Johansson BJE (2021) A framework for describing interaction between human operators and autonomous, automated, and manual control systems. Cogn Technol Work 23(3):381–401. https://doi.org/10.1007/s10111-020-00637-w Maher ML, Bilda Z, Gül LF (2006) Impact of collaborative virtual environments on design behaviour. Design computing and cognition ’06. Dordrecht: Springer, Netherlands, pp 305–321 Makarius EE, Mukherjee D, Fox JD, Fox AK (2020) Rising with the machines: a sociotechnical framework for bringing artificial intelligence into the organization. J Bus Res 120:262–273. https://doi.org/10.1016/j.jbusres.2020.07.045 Naikar N, Moylan A, Pearce B (2006) Analysing activity in complex systems with cognitive work analysis: concepts, guidelines and case study for control task analysis. Theor Issues Ergon Sci 7(4):371–394 Naikar N, Pearce B, Drumm D, Sanderson PM (2003) Designing teams for first-of-a-kind, complex systems using the initial phases of cognitive work analysis: case study. Human Fact 45(2):202– 217. https://doi.org/10.1518/hfes.45.2.202.27236 Oosthuizen R, Pretorius L (2016) Assessing the impact of new technology on complex sociotechnical systems. South Afr J Ind Eng 27(2):15–29. https://doi.org/10.7166/27-2-1144 Oxman R (2006) Theory and design in the first digital age. Des Stud 27(3):229–265. https://doi. org/10.1016/j.destud.2005.11.002 Paletz SBF, Sumer A, Miron-Spektor E (2018) Psychological factors surrounding disagreement in multicultural design team meetings. CoDesign 14(2):98–114. https://doi.org/10.1080/157 10882.2017.1378685 Pasmore WA, Francis C, Haldeman J, Shani A (1982) Sociotechnical systems: a North American reflection on empirical studies of the seventies. Hum Relat 35(12):1179–1204. https://doi.org/ 10.1177/001872678203501207 Pirolli P, Card S (2005) The sensemaking process and leverage points for analyst technology as identified through cognitive task analysis. In: Proceedings of international conference on intelligence analysis. McLean, VA, USA Rahimian FP, Ibrahim R (2011) Impacts of VR 3D sketching on novice designers’ spatial cognition in collaborative conceptual architectural design. Des Stud 32(3):255–291. https://doi.org/10. 1016/j.destud.2010.10.003 Sackey E, Tuuli M, Dainty A (2015) Sociotechnical systems approach to BIM implementation in a multidisciplinary construction context. J Manag Eng 31(1):A4014005. https://doi.org/10.1061/ (ASCE)ME.1943-5479.0000303 Salah AA, Chandrasegaran S, Lloyd P (2022) Analysing storytelling in design talk using LIWC (Linguistic Inquiry and Word Count). In: Goldschmidt G, Tarazi E (eds) 13th design thinking research symposium: expanding the frontiers of design: a blessing or a curse?, pp 71–86 Santandreu Calonge D, Safiullin AF (2015) Can culturally, disciplinarily and educationally diverse (D3) teams function and be creative? A case study in a Korean university. Educ Stud 41(4):369– 392.https://doi.org/10.1080/03055698.2015.1018871

2 Online Design Processes and Communication Across Cultures …

37

Schraagen JM, Chipman SF, Shalin VL (2000) Cognitive task analysis. Lawrence Erlbaum Associates Publishers, Mahwah, NJ, US Seamster TL, Redding RE, Cannon JR, Ryder JM, Purcell JA (1993) Cognitive task analysis of expertise in air traffic control. Int J Aviat Psychol 3(4):257–283 Seeber I, Bittner E, Briggs RO, de Vreede T, de Vreede G-J, Elkins A, Maier R et al (2020) Machines as teammates: a research agenda on AI in team collaboration. Inf Manag 57(2):103174. https:// doi.org/10.1016/j.im.2019.103174 Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27(3):379–423. https://doi.org/10.1002/j.1538-7305.1948.tb01338.x Stempfle J, Badke-Schaub P (2002) Thinking in design teams—an analysis of team communication. Des Stud 23(5):473–496. https://doi.org/10.1016/S0142-694X(02)00004-2 Tang HH, Lee YY, Gero JS (2011) Comparing collaborative co-located and distributed design processes in digital and traditional sketching environments: a protocol study using the function–behaviour–structure coding scheme. Des Stud 32(1):1–29. https://doi.org/10.1016/j.des tud.2010.06.004 Tausczik YR, Pennebaker JW (2010) The psychological meaning of words: LIWC and computerized text analysis methods. J Lang Soc Psychol 29(1):24–54. https://doi.org/10.1177/0261927x0935 1676 Trist EL (1981) The evolution of socio-technical systems. In: de Ven AHV, Joyce WF (eds) Perspectives on organization design and behavior. Wiley, New York, pp 19–75 Trist EL, Bamforth KW (1951) Some social and psychological consequences of the longwall method of coal-getting: an examination of the psychological situation and defences of a work group in relation to the social structure and technological content of the work system. Hum Relat 4(1):3–38. https://doi.org/10.1177/001872675100400101 Valkenburg R, Dorst K (1998) The reflective practice of design teams. Des Stud 19(3):249–271. https://doi.org/10.1016/S0142-694X(98)00011-8 Vicente KJ (1999) Cognitive work analysis: toward safe, productive, and healthy computer-based work. CRC Press, Boca Raton, London, New York Wu Z, Duffy AHB (2004) Modeling collective learning in design. Artif Intell Eng Des Anal Manuf 18(4):289–313. https://doi.org/10.1017/S0890060404040193

Chapter 3

Evaluating the Use of Digital Technologies to Support Design Collaboration Ning Gu , Rongrong Yu , Kerry London , Zelinna Pablo , and Maria Roberts

Abstract Collaboration is particularly important for complex design and construction projects that often involve large and diverse professional and end-user stakeholder groups. Digital technologies have been found to be able to improve the collaborative process in these projects. However, currently there is still a lack of critical understanding about the effectiveness of different digital technologies, especially the emerging ones, during design collaboration. This chapter adopts a cognitive approach to studying different digital technologies in supporting design collaboration via comparison of two different types of collaborative technologies—a 3D modelling environment and an immersive design environment, benchmarking them against a more traditional face-to-face collaborative design environment, as well as comparing between two types of collaboration team dynamics (namely, experts with non-experts, and experts with other experts). The results of this preliminary study show that differences in design collaboration behaviour were apparent when compared across the three design environments, and the collaborative design behaviours of the two types of teams showed differences in collaborative practices related to Expertise, Organising Mechanisms, Problem Solving, and Shared Goals. It is suggested that designers can maintain comparable effort during collaborating and designing in the immersive 3D design environment as they do in the typical 3D modelling design environment. Keywords Design collaboration · Digital technology · Immersive design environment

N. Gu · R. Yu (B) · M. Roberts UniSA Creative, IVE: Australian Research Centre for Interactive and Virtual Environments, University of South Australia, Adelaide, SA 5000, Australia e-mail: [email protected] K. London · Z. Pablo CHSD: Centre for Healthy Sustainable Development, Torrens University Australia, Sydney, NSW 2007, Australia © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 J. H. Lee et al. (eds.), Multimodality in Architecture, https://doi.org/10.1007/978-3-031-49511-3_3

39

40

N. Gu et al.

3.1 Introduction As building design and construction projects grow in terms of both scale and complexity, collaborative practices are becoming increasingly important in the sector (London and Pablo 2017). Design collaboration is particularly important for those complex projects that often involve large and diverse professional and end-user stakeholder groups (Merschbrock and Munkvold 2015). With global digital transformations occurring across most sectors, digital collaboration is being increasingly adopted as the preferred mode for collaboration, including in the building and construction industries (Construction 2014). Digital technologies, such as Building Information Modelling (BIM) platforms, have been found to be capable of improving the collaborative process, particularly in large-scale and complex design and construction projects (Liu et al. 2017; Merschbrock and Munkvold 2015; Moum 2010; Oh et al. 2015). For example, through a case study (Merschbrock and Munkvold 2015), it was found that a BIM-based approach to building design and construction was able to overcome many of the challenges typically associated with complex projects. However, in the context of rapid digital evolution there is still a lack of critical understanding about the effectiveness of different digital technologies, especially the emerging ones, during design collaboration. To address the above research gap, this chapter adopts a cognitive approach to studying different digital technologies in supporting design collaboration by directly observing selected participants when they are using these different technologies for design collaboration. This preliminary study will formally investigate two perspectives. Firstly, the study compares two different types of collaborative technologies, namely the 3D modelling environment, a typical BIM platform supported with Zoom Meetings for online communication, and the immersive design environment, an emerging advanced design environment where designers and collaborators are immersed within the computer-based visualisation of the design (the Hyve-3D system is used in this instance). These are benchmarked against the more traditional face-toface collaborative design environment that uses analogue hand sketching. Next, the study compares two different types of team dynamics during the collaboration, one being architects collaborating with end-users and the other being architects collaborating with urban planners/designers. The first type represents team collaboration involving both experts and non-experts, and the second type represents team collaboration involving experts only, but from different disciplines. It is expected that having professional experiences and different disciplinary knowledge may have an impact on the application and effect of different digital technologies. The chapter is structured as follows. In the following section, background on two topics—design collaboration, and collaborative technologies in design—that are closely related to this study is provided. This is followed by the introduction of the cognitive methodology adopted for the investigation. The experiment design and coding scheme development will be presented and discussed. Next, results of the study will be presented and interpreted, focusing on the two comparative analyses. The chapter concludes by highlighting the impacts of different digital technologies

3 Evaluating the Use of Digital Technologies to Support Design …

41

on design collaboration and providing recommendations for future developments of collaborative technologies and collaborative practices in design and construction projects.

3.2 Background 3.2.1 Studying Design Collaboration Design collaboration refers to team-based design activities in which participants work toward shared design goals, with effective collaboration during the initial stages of design generally leading to fewer problems during the latter, more complex, design and construction stages (Leon et al. 2015). Design collaboration with multiplicate problem-solving approaches can align different stakeholders’ opinions toward a common baseline that can more optimally result in valuable project insights (Feast 2012). Tan (2021) argues that design collaboration enhances reflection upon the actions of designers within a team. Effective communication and common knowledge/expertise shared between team members are vital for the success of collaborative design projects, as the similarity of language among components of knowledge use contributes to a constructed shared mental representation of the design product (Dong 2005). Kleinsmann et al. (2012) emphasised that design expertise and design collaboration skills have a noticeable impact on collaborative design performance. Citing the need for remote collaboration during pandemic periods, Kim et al. (2020) noted that social networking services can increase idea clarification and the sharing of information to support the achievement of active design collaboration, while Combrinck and Porter (2021) found that the initial stages of design benefit directly from collaboration between architects and end-users. Design collaboration occurring at the early design stage is significant for achieving design innovation and, ultimately, optimal design solutions. The possibility of determining design perception, design physiology and design neurocognition in a collaborative environment also facilitates the development of improved design patterns, creativity, and reasoning among multiple users (Shealy et al. 2020). The collaborative activity of designers has been studied extensively, mostly demonstrated via protocol studies investigating designers directly involved in design. For example, Goldschmidt (1995) conducted a protocol study to compare individual designers’ behaviours with those of designers working in teams and found no significant differences in terms of achieving design goals between the individual designers and the team-based designers. Valkenburg and Dorst (1998) developed a description tool based on Schön’s reflective practice theory (Schön 1984) to improve team design activities, which proved to be effective in describing team activities. Using protocol analysis, Stempfle and Badke-Schaub (2002) studied collaborative design behaviour in a team design setting and proposed a two-process theory of thinking during design collaboration. Different collaborative design behaviours and design

42

N. Gu et al.

thinking were identified in two design teams from different organisations, demonstrating the impact of working culture on team design behaviour (Milovanovic et al. 2021). Another protocol study conducted by Gomes and Tzortzopoulos (2018) also suggested that misunderstandings emerge through the independent actions and wrong assumptions of team members which can lead to difficulty in constructing a shared understanding during design collaboration.

3.2.2 Collaborative Technologies in Design Researchers have emphasised that digital modalities and collaboration will affect the quality, efficiency, and accuracy of design (Froese 2010; Garber 2014; Succar 2009). Virtual/remote collaborative environments have become increasingly relevant and crucial since 2020, enabling distributed design collaboration and facilitating greater efficiency and cost-effectiveness in design (Abdelhameed 2012), as well as increasing design accuracy (Azmi et al. 2018; Froese 2010). In addition, such virtual environments can provide feedback about the 3D characteristics of the design, increasing participants’ level of engagement with the design, empowering visual thinking, and facilitating a deeper spatial understanding about a place (Okeil 2010). Early studies have explored the application of shared digital environments in collaboration during the design phase (Gross et al. 1998; Kalay et al. 1998; McCall and Johnson 1997), including the effects of those environments on designers. For example, Gu et al. (2011) suggest that 3D virtual worlds support the production of considerable perceptual events during synchronous design collaboration. Hong et al. (2019) also found that novelty and appropriateness are more highly exhibited in multi-user virtual environments than in online sketching environments due to the importance of explicit communication cues for sharing collaborative procedures and spatial information. Kim and Maher (2008) explore the impact of Tangible User Interface (TUI) on designers’ collaborative behaviour and identify that TUI changed designers’ spatial cognition in a way that saw ‘problem-finding’ behaviours increase. Recent developments in immersive virtual environments have facilitated intuitive virtual interactions between designers and also between designers and their design environments (Pour Rahimian et al. 2020), leading to better spatial perception (Paes et al. 2017) which can be beneficial for the design process. Collaborative digital technologies have developed further in recent decades, creating shared digital spaces, either co-located or distributed, at various levels of immersion. These two conditions, the remote and the co-located formed the basis of a comparative study of collaborative design environments by Gül and Maher (2009). The terms remote or distributed generally indicate physical or spatial distance, often between collaborators who are designing in virtual environments and/or are spatially remote from one another. Co-location refers to designers or participants working sideby-side or interacting in physical proximity (Gül and Maher 2009). Earlier co-located collaborative digital tools include some of the TUIs mentioned above. Many collaborative digital tools could be either co-located or used remotely, such as BIM 360,

3 Evaluating the Use of Digital Technologies to Support Design …

43

Hyve-3D and various types of shared whiteboards and other tools. There are a variety of virtual remote collaborative tools currently in use within the Architecture, Engineering and Construction (AEC) industry, such as Unity Reflect Review, Resolve, Trezi, Fuzor and BIM 360, which largely focus on design review or construction scheduling. Additionally, a few virtual collaborative tools have recently emerged that are intended for use at the concept design stage, including Wild, Mindesk and Arkio. For design purposes, most of these tools support the importation of 3D models from commonly used architectural design software such as Revit and SketchUp. Another virtual collaborative tool, Hyve-3D, is primarily focused on providing 3D sketching in immersive design environments to support both face-to-face and remote collaboration and ideation (Hyve-3D requires a Macbook, iPad Pro and a 4 K projector, however a VR headset is not needed for an immersive experience). The limitations, in terms of design support, of such emerging collaborative digital technologies include insufficient sketching and overly simplified modelling functions; furthermore, they exhibit a lack of advanced features, such as analyses for building performance, land use, etc., which are essential features for making informed design decisions. Aside from the above-mentioned collaborative technologies that are specific to the AEC industry, more generic communication and collaboration tools are also commonly used in design practices, including Microsoft Teams, Zoom Meetings, Slack, etc. Some of these generic teamwork tools also provide certain visual collaboration functions. For example, Miro has a 2D digital whiteboard for brainstorming, Asana can assist project workflow planning with visualisation features for timelines and calendars, and other tools such as Jira can be linked to Navisworks and allow the display of 3D models. However, within most generic communication tools the design features are highly limited. Thus far, various collaboration modes such as co-located, remote or distributed and hybrid, have been included, occurring either synchronously or asynchronously (Ens et al. 2019). However, the full impact of how these different settings support design collaboration is presently not sufficiently clear.

3.3 Methods This preliminary study adopts a cognitive approach using protocol analysis to study different teams in applying different digital technologies to support their collaboration. Protocol analysis is one of the most promising and reliable methods for studying design (Ericsson and Simon 1980; Gero and McNeill 1998), and collaboration is one of the most prominent topics in protocol studies. The general procedure of protocol analysis is as follows: Recruited designers are required to “think aloud” (which means to verbalise their thinking during the experiment), while designing and collaborating. The verbalisations and actions of the participants are both video recorded as collected protocol data, and then the data is transcribed and segmented. A customised coding scheme is applied to categorise the segmentations to enable subsequent detailed analysis. Due to the large volume of data produced by the protocol analysis method and the relative complexity of the analysis procedure, protocol studies have limited

44

N. Gu et al.

themselves to a relatively small number of subjects (often below 10). In a systematic review of protocol studies on design cognition, Hays et al. (2017) found the most common sample size across 47 studies was between 1 and 16 participants (mean = 6, median = 7). Nevertheless, protocol analysis is considered the superior methodology compared to alternative research techniques such as interviews, questionnaires, observations, and case studies (Cross 2001) since it can reveal designers’ thinking and process in greater detail. This study involves protocol data from four teams of eight participants in total, which is common for typical protocol studies. To ensure participants produce sufficient “think aloud” data, a “think aloud” training session has been provided to participants prior to the experiment. Employing protocol analysis, both concurrent and retrospective protocol collection methods can be applied to design experiments (Dorst and Dijkhuis 1995; Ericsson and Simon 1993). In this study, using the concurrent method, participants have been asked to verbalise their thinking processes during the collaborative design experiment. Further, retrospective protocol collection has been applied in which a follow-up interview is conducted with participants to further clarify the processes. By adopting a combined protocol collection method, it is ensured that sufficient data is collected in this research.

3.3.1 Coding Scheme Development In protocol studies, the coding scheme is vitally important for encoding and analysis of the data collected. The earliest coding scheme in cognitive studies was proposed by Eastman (1970), who used design units, constraints and manipulations to encode protocols and explore the behaviour graph during the design process. Since then, researchers have continued to develop a variety of coding schemes to address specific research problems, such as those developed by Suwa and Tversky (1997), Kan and Gero (2009), and Yu and Gero (2015). This study focuses on understanding how different digital technologies support design collaboration with a customised coding scheme specifically developed based on the collaborative practice model (London and Pablo 2021; Pablo and London 2017). The collaborative practice model was developed to understand the collaboration processes within the construction industry and as a way of understanding the network-based relationships associated with construction projects. That model identifies nine key elements of collaboration: Leadership, shared goals and norms, expertise, change, problem solving, investment in relationship, shared space, organising mechanisms, and technical standards (Pablo and London 2017). The collaborative practice model was mobilised for this study as it was developed for general practices that involved collaboration between multiple stakeholders during various stages of building design and construction. Our study only focuses on the collaborative design stage. Therefore, we have adapted key elements of collaboration in that model relevant to the collaborative design stage. For example, leadership is demonstrated in multiple stakeholders, however it is less frequently demonstrated

3 Evaluating the Use of Digital Technologies to Support Design …

45

in our collaborative design experiments, and investment in relationship is also not relevant to the design experiments, which may require longer term to establish. Our final coding scheme has five categories (Table 3.1) by adapting key elements from the collaborative practice model focusing on design collaboration. These five categories have emerged and were tested via the analysis of the first baseline experiment as the pilot study (see details in Sect. 3.2.).

3.3.2 Experiment Design As shown in Table 3.2, the experiment design for the study enables us to compare two different types of digital technologies in supporting design collaboration, and benchmark against the baseline, a more traditional face-to-face collaborative design environment using analogue hand sketching. The first digital technology selected is a 3D modelling environment consisting of a typical BIM platform supported with Zoom Meetings for online communication. The second digital technology selected is an emerging one—Hyve-3D system—an advanced immersive design environment where designers and collaborators are immersed within the computer visualisation of the design. Use of the BIM platform in conjunction with Zoom Meetings has emerged as a relevant combination as a direct result of the recent global pandemic. The selection of Hyve-3D as the immersive design environment is due to immersive design environments including Virtual Reality (VR) and Augmented Reality (AR) being emerging technologies increasingly applied in the AEC industry, and the fact that they can immerse designers in the design environment to provide more superior design visualisation and simulation and potentially affect designers’ collaboration. However, they have been little studied to date. This experiment design also enables us to compare two different types of team dynamics during the collaboration. For both experiments and the baseline, we recruited four collaborative teams using an actual mixed-use high-rise housing project. Teams 1 and 2 each consist of an architect with professional experience collaborating with an end-user, representing collaboration between experts and non-experts. Teams 3 and 4 each consist of an architect collaborating with an urban planner/designer, both with professional experience, representing collaboration amongst experts only but from different disciplines. Each team was asked to collaboratively conduct three simplified design tasks (with similar complexity) related to different aspects of the housing project in the three different collaborative design environments. Each collaborative design session lasted around 30–40 min, followed by an interview session to capture their perceptions and to clarify any issues that emerged from the collaboration, which lasted around 10–20 min. These sessions were video recorded and subsequently transcribed then encoded using the coding scheme above, following the protocol analysis conventions.

46

N. Gu et al.

Table 3.1 The coding scheme developed for the study, adapted from the collaborative practice model (London and Pablo 2021; Pablo and London 2017) Coding category

Definition

Example

Expertise

Segments related to expert knowledge or personal experiences ‘We have (for end users) in the context of solving design problem/ options to providing design solution adjust materiality’ ‘It could be timber; it could be tile’ ‘It’s a nicer material but difficult to maintain’ (materiality is a type of design expertise being focused on above)

Organising mechanisms

Segments related to the organisation and procedures that aim to progress the collaboration forward

‘Back to our previous conversation’ ‘What I’d like to touch on before we get to the end’ ‘Could you elaborate?’

Problem solving

Segments containing actual proposed solutions to the design task and/or potential improvements or alternatives to the solution

‘I would prefer a flat floor’ (amongst proposed solutions) ‘We can look at doing that’ (referring to a proposed solution) ‘I think you’re right’ (referring to a proposed solution) (continued)

3 Evaluating the Use of Digital Technologies to Support Design …

47

Table 3.1 (continued) Coding category

Definition

Example

Shared goals

Segments related to working towards a shared goal, either achieving an agreement or progressing towards an agreement, in the context of solving a design problem/providing design solution

‘Is it better this way or that?’ ‘No, so not a swimming pool?’ ‘Let’s say it’s transparent’ (when confirming with the collaborator)

Shared space

Segments related to the shared workspace (either physical or digital), including but not limited to discussions about the use of software, orientation/location of the observer to/in the design representation, and general discussions about the design representation

‘I’ll see if I can move this around’ (referring to the shared model) ‘Can you see two buildings?’ (in the shared model) ‘What’s over here?’ (in the shared model)

Table 3.2 Experiment design Baseline: face-to-face collaborative design environment, analogue hand sketching Team 1

Participants: architect 1 and end user 1

Design task: 1a

Team 2

Participants: architect 2 and end user 2

Design task: 1a

Team 3

Participants: architect 3 and urban planner/designer 1

Design task: 1b

Team 4

Participants: architect 4 and urban planner/designer 2

Design task: 1b

Experiment I: 3D modelling environment, BIM (Revit) supported with zoom meetings Team 1

Participants: architect 1 and end user 1

Design task: 2a

Team 2

Participants: architect 2 and end user 2

Design task: 2a

Team 3

Participants: architect 3 and urban planner/designer 1

Design task: 2b

Team 4

Participants: architect 4 and urban planner/designer 2

Design task: 2b

Experiments II: immersive design environment, Hyve-3D system Team 1

Participants: architect 1 and end user 1

Design task: 3a

Team 2

Participants: architect 2 and end user 2

Design task: 3a

Team 3

Participants: architect 3 and urban planner/designer 1

Design task: 3b

Team 4

Participants: architect 4 and urban planner/designer 2

Design task: 3b

48

N. Gu et al.

3.4 Results 3.4.1 Protocol Analysis Results Table 3.3 presents coding distributions for the four experiments in the three design environments (face-to-face, 3D modelling, and immersive). Since segment number varies for each design experiment, the frequency distributions of the coding were normalised by converting the occurrence frequencies of each code into percentages of the total number of codes, as shown in Table 3.4. Table 3.4 shows that, on average, Shared Goals has the highest coding percentage of 38.44%; this is followed with Expertise (25.19%), Problem Solving (15.72%) and Shared Space (15.09%), while the lowest coding percentage is Organising Mechanism (5.57%). This means participants expended a large amount of effort in applying their professional knowledge or personal experience during design collaboration. They also tend to give equal amounts of consideration towards solving design problems and navigating in their shared space (physical or virtual) during their design collaboration. In order to generalise these results, the mean value of all of the experiments is used, and SD value is presented to show the data spread among samples. The SD value in Table 3.4 is relatively large (all SD/Mean > 50%, except for shared goals being 34%). This means the data are widely spread, and the collaborative behaviour of designers varies in different experiments and design environments. Table 3.3 Coding distributions of collaborative design sessions by the four teams in the three design environments Design environments

Teams

Expertise

Organising mechanism

Problem solving

Shared goals

Shared space

Baseline: face-to-face collaborative design environment

1

203

31

20

46

2

2

123

34

57

247

48

3

80

7

71

111

93

4

56

4

74

123

17

3D modelling environment

1

67

23

33

77

35

2

52

19

89

80

69

3

86

11

60

169

82

4

36

9

6

117

51

1

63

20

43

68

35

2

45

17

115

109

103

3

45

10

32

141

11

4

64

15

5

120

28

Mean

76.67

16.67

50.42

117.33

47.83

SD

46.06

9.32

33.62

52.86

32.81

Immersive design environment

3 Evaluating the Use of Digital Technologies to Support Design …

49

Table 3.4 Normalised coding distributions of collaborative design sessions by the four teams in the three design environments Design environments

Teams

Expertise (%)

Organising mechanism (%)

Baseline: Face-to-face collaborative design environment

1

67.22

10.26

6.62

15.23

2

24.17

6.68

11.20

48.53

9.43

3

22.10

1.93

19.61

30.66

25.69

4

20.44

1.46

27.01

44.89

6.20

3D modelling environment

1

28.51

9.79

14.04

32.77

14.89

2

16.83

6.15

28.80

25.89

22.33

3

21.08

2.70

14.71

41.42

20.10

4

16.44

4.11

2.74

53.42

23.29

1

27.51

8.73

18.78

29.69

15.28

2

11.57

4.37

29.56

28.02

26.48

3

18.83

4.18

13.39

59.00

4.60

4

27.59

6.47

2.16

51.72

12.07

Mean

25.19

5.57

15.72

38.44

15.09

SD

14.18

2.96

9.44

13.31

8.66

Immersive design environment

Problem solving (%)

Shared goals (%)

Shared space (%) 0.66

3.4.2 Comparison I: Results Across the Three Collaborative Design Environments This section presents the comparative analysis between the three design environments: Analogue sketching (baseline), 3D modelling in BIM enhanced with Zoom meetings, and immersive design in Hyve-3D. Figure 3.1 shows the comparison of coding distribution in the three design environments. The coding distribution is presented as the average occurrence of the normalised coding percentage of the four experiments in the three design environments. From the figure, we can see that the coding distributions in Organising Mechanism, Problem Solving and Shared Goals are similar across the three design environments. The coding percentage of Expertise in the face-to-face collaborative environment (33.48%) is higher than in the 3D modelling environment (20.71%) and in the immersive design environment (21.37%). This may be due to the nature of the face-to-face sketching environment, which is a relatively natural environment where more conversation regarding either domain knowledge or personal experiences can occur. The coding percentage of Shared Space exhibits the highest value in the 3D modelling environment (20.15%), followed by the immersive design environment (14.61%) and face-to-face collaborative environment (10.50%). This may be due to the fact that the 3D modelling environment such as Revit is relatively complex, and thus shared protocols such as those regarding navigation and orientation between participants, may need to be established, which results in the high percentage of this coding category. The fact the

50

N. Gu et al.

two participants were physically remote from one another and not co-located may also have contributed to the high percentage of Shared Space in the 3D modelling environment. Meanwhile, in the face-to-face collaborative environment using sketching, which is the most natural environment to most participants, there may not be much additional effort required relating to Shared Space. Figure 3.2 presents a comparison of the coding distributions of the three design environments for the architect and end-user teams. In the figure, we can see that for the architect and end-user teams, some of their design collaboration behaviour in the face-to-face collaborative design environment differs substantially from the other two digital design environments. The Expertise coding percentage in the faceto-face collaborative design environment (45.69%) is higher than the two digital design environments (22.67% and 19.54%), which may be because the more natural face-to-face setting can trigger more conversations regarding an end-user’s personal experiences. The Problem Solving coding percentage in the face-to-face collaborative design environment is lower (8.91%) than in the two digital design environments (24.42 and 24.17%), which may be due to the limitation of sketching as a 2D design environment. It can be more difficult for the participants especially end-users to push for design solutions that are three-dimensional. Similarly, the Shared Space coding percentage is noticeably lower in the face-to-face collaborative design environment (5.05%) than in the other two digital design environments (18.61 and 20.88%), which may reflect end-users needing professional architects to assist them more frequently with navigation and operation in these digital technologies (that are more complex compared to the face-to-face collaborative design environment) during collaboration. Figure 3.3 presents a comparison of the coding distributions of the three design environments for the architect and planner teams. The figure shows that the design collaboration between architects and planners exhibits a number of differences for the three design environments. Firstly (and quite opposite from what was seen in the architect and end-user teams) the Problem Solving coding percentage is higher in the face-to-face collaborative design environment (23.31%) than in the other two digital design environments (8.72% and 7.77%), which may be because experts tend to be more comfortable and accustomed to seeking design solutions using sketching tools, especially during conceptual design. Secondly, the expert teams had a slightly higher

Fig. 3.1 Comparison of coding distributions in the three design environments

3 Evaluating the Use of Digital Technologies to Support Design …

51

Fig. 3.2 Comparison of coding distributions in the three design environments for the architect and end-user teams

Shared Goals coding percentage in the immersive design environment (55.36%), compared to the 3D modelling environment (47.42%) and the face-to-face collaborative design environment (37.78%). Furthermore, the Shared Space coding percentage was higher in the 3D modelling environment (21.69%) than in the face-to-face collaborative design environment (15.95%) and immersive design environment (8.34%), which is also the opposite compared to the architect and end-user teams. This may be due to the strong three-dimensional spatial cognition of experts potentially making it easier for them to understand 3D spaces within the immersive design environment, while remote collaboration in the 3D modelling environment may possibly lead to more exploration in terms of Shared Space. For the purposes of deepening the investigation, we have used “overlapping coding” to describe the frequent situation where one segment has necessarily been allocated to multiple codes. Table 3.5 shows the overlapping coding percentage of the

Fig. 3.3 Comparison of coding distributions in the three design environments for the architect and planner teams

52

N. Gu et al.

four experiments in the three design environments. Thresholds of 10% (high overlapping) and 3% (medium overlapping) were set to indicate the significant overlapping percentage. Results in Table 3.5 reveal three high overlapping codes: Expertise and Shared Goals in the face-to-face collaborative design environment (10.71%), 3D modelling environment (11.70%) and immersive design environment (14.67%). When establishing Shared Goals during design collaboration, it is common for participants to use their expert knowledge or their personal experience. Another relatively high overlapping code is Expertise and Problem Solving across all three design environments, which indicates that problem solving needs to apply professional knowledge or personal experience. Furthermore, Problem Solving overlapped relatively frequently with Shared Goals in the immersive design environment (3%), compared with those in the 3D modelling environment (1.24%) and face-to-face collaborative design environment (1.21%). This suggests the immersive design environment may be beneficial for supporting problem-solving activities while setting up shared goals during participants’ collaboration. Additionally, Shared Goals and Shared Space overlapped relatively frequently in the 3D modelling environment (4.32%), which suggests that setting up shared goals occurred relatively frequently during designers’ establishment of shared space in the digital collaborative environment. Table 3.5 Average of overlapping coding percentages of the four experiments in the three design environments Design environments Face-to-face collaborative design environment

3D modelling environment

Expertise (%)

** >

10%,

* >3%

Problem solving (%)

Shared goals (%)

Organising mechanism

0.37

–

–

–

Problem solving

4.68*

0.18

–

–

Shared goals 10.71**

0.94

1.11

–

Shared space

2.67

0.15

1.12

1.86

Organising mechanism

0.53

–

–

–

Problem solving

3.06*

0.08

–

–

Shared goals 11.70** Immersive design environment

Organising mechanism (%)

0.84

1.24

–

Shared space

4.71*

0.52

0.66

4.32*

Organising mechanism

0.06

–

–

–

Problem solving

3.11*

0.35

–

–

Shared goals 14.67**

0.91

3.00*

–

Shared space

0.13

0.32

1.33

1.06

3 Evaluating the Use of Digital Technologies to Support Design …

53

3.4.3 Comparison II: Results Across the Two Different Collaborative Team Dynamics This section presents the comparative analysis between the two different team dynamics in design collaboration: Architects collaborating with end-users, and architects collaborating with urban planners/designers. Figure 3.4 shows the comparison of the coding distributions of the two different team dynamics in all three design environments. In the figure, we can see that most of the coding exhibits differences, except for Shared Space. The Expertise coding percentage is higher in the architect and end-user teams (29.30%), compared to the architect and urban planner/designer teams (21.08%), which may be due to architects needing to frequently familiarise end-users with expert knowledge. The Organising Mechanism coding percentage is higher in the architect and end-user teams (7.66%), compared to the architect and urban planner/designer team (3.47%), which suggests architects needing to take the lead in the expert and novice design collaboration. And the Problem Solving coding percentage is slightly higher in the architect and end-user teams (18.17%) compared to the other teams (13.27%). The Shared Goals coding percentage is higher in the architect and urban planner/designer teams (45.85%), compared to the architect and end-user teams (30.02%), which may be because, for expert designers, it is more likely that they can effectively establish shared goals during collaboration, based on their expert knowledge and professional experience. Figure 3.5 presents the coding distributions of the two collaborative team dynamics in the baseline: Face-to-face collaborative design environment. From the figure, we can see that significant differences are exhibited in comparing the two team dynamics: The coding distribution of Expertise is much higher in the architect and end-user teams (45.69%) than in the architect and urban planner/designer teams (21.27%). This may be because architects will often need to explain expert knowledge to end-users to frequently familiarise them during collaborative design. Similarly,

Fig. 3.4 Comparison of coding distributions of the two different collaborative team dynamics in the three design environments

54

N. Gu et al.

Fig. 3.5 Comparison of coding distributions of the two collaborative team dynamics in the faceto-face collaborative design environment

the Organising Mechanism coding percentage in the architect and end-user teams is higher (8.47%) than in the other teams (1.70%). That may be because architects often take the lead during the design collaboration with end-users. On the other hand, the Problem Solving coding percentage is higher in the architect and urban planner/ designer teams (23.31%) than in the architect and end-user teams (8.91%), suggesting that experts are more effective in problem-solving during the baseline design collaboration. The Shared Space coding percentage is also higher in the architect and urban planner/designer teams (15.95%) than in the other teams (5.05%), which indicates that expert teams may pay more attention to establishing shared spaces during the baseline design collaboration. Shared Goals coding percentages are similar between the two. Figure 3.6 compares the coding distributions of the two team dynamics for the first digital technology being explored: the 3D modelling environment. Results show that three coding percentages illustrate significant differences, including Organising Mechanism, Problem Solving, and Shared Goals. Just as in the face-to-face collaborative design environment, the Organising Mechanism coding percentage is higher in the architect and end-user teams (7.97%) than in the architect and urban planner/ designer teams (3.40%). The Problem Solving coding percentage is also higher in the architect and end-user teams (21.42%) than in the other teams (8.72%)—that may be because in the 3D modelling environment, architects and end-users tended to push to achieve design solutions, while architects and urban planners/designers tended to review existing solutions and discuss the design from their own professional perspectives. On the other hand, the Shared Goals coding percentage is higher in the architect and urban planner/designer teams (47.42%) than in the architect and end-user teams (29.33%), which suggests that in the 3D modelling environment, the expert only teams established more shared goals than the expert-novice teams. Expertise and Shared Space coding distributions are similar between the two teams. Figure 3.7 presents results of the coding distributions of the two team dynamics in the second digital technology being explored: the immersive design environment. Results suggest that, just as in the 3D modelling environment, the Problem Solving coding percentage is higher in the architect and end-user teams (24.17%) than in the

3 Evaluating the Use of Digital Technologies to Support Design …

55

Fig. 3.6 Comparison of coding distributions of the two collaborative team dynamics in the 3D modelling environment

architect and urban planner/designer teams (7.77%), and the Shared Goals coding percentage on the contrary is higher in the architect and urban planner/designer teams (55.36%) than in the other teams (28.86%). Exhibiting differently from the 3D modelling environment, the Shared Space coding percentage is higher in the architect and end-user teams (20.88%) than in the architect and urban planner/designer teams (8.34%), which may be due to a lack of the end-users’ 3D spatial cognition ability, whereby the architects may be required to orient and familiarise the end-users more often during collaboration in the immersive design environment. Table 3.6 shows the average of the overlapping coding percentages of the design experiments in the two teams. From the table we can see that Shared Goals and Expertise has the highest overlapping coding percentage in the architect and planner group (20.29%), which indicates that often professionals set up their shared goals based on their professional knowledge or experiences. This is followed with the overlapping coding of Problem Solving and Expertise in the same group (5.21%), which means that experts tended to use expert knowledge to solve design problems. Another relatively higher overlapping coding is Shared Goals and Expertise in the architect and end-user group (4.43%); however, it is still much lower than in the experts’

Fig. 3.7 Comparison of coding distributions of the two collaborative team dynamics in the immersive design environment

56

N. Gu et al.

Table 3.6 Average of the overlapping coding percentages of the design experiments between the two team dynamics Teams

Architect and end-user

Architect and urban planner/ designer

** >

10%,

*>

Expertise (%)

Organising mechanism (%)

Problem solving (%)

Shared goals (%)

Organising mechanism

0.26

–

–

–

Problem solving

2.02

0.41

–

–

Shared goals 4.43*

1.57

2.63

–

Shared space 0.58

0.45

0.53

1.26

Organising mechanism

0.39

–

–

–

Problem solving

5.21*

0.00

–

–

Shared goals 20.29**

0.22

0.94

–

Shared space 5.04*

0.08

0.87

3.75

3%

group, which suggests that setting up shared goals using professional knowledge is common but collaborating professionals do it more frequently.

3.5 Discussion We have explored the effect of two different digital technologies (3D modelling and immersive) for supporting design collaboration in two different modes (co-located and remote), with two different types of team dynamics (experts with non-experts and experts with other experts), benchmarked against a baseline (face-to-face). The following discussion further elaborates on these results.

3.5.1 Effect of Different Digital Technologies in Supporting Design Collaboration Across the three design environments, the collaboration process exhibits similarities in some areas, such as Organising Mechanism, Problem Solving and Shared Goals; however, significant differences were found in other areas, such as Expertise and Shared Space. In the face-to-face collaborative design environment, participants tended to apply their expert knowledge or personal experience more frequently than

3 Evaluating the Use of Digital Technologies to Support Design …

57

in the two digital environments. We can infer that participants may find face-toface settings more familiar and relaxing and may more easily commit to the task by applying their knowledge and experience. Some participants were also more comfortable when utilising sketching. For example, one architect commented: “I think it’s nice when I can hand a pen over and say this is… And it’s nice to feel like we’re both coming at the problem, and we’re both contributing in a way where it’s documented.” On the other hand, some participants commented about the limitation that a sketching environment can present such as the lack of design details: “Because it was just paper-based, some details we can’t see. The whole details are just two dimensions in the photo. We can’t analyse all of the details”. The 3D modelling environment requires participants especially non-experts to expend more effort in being accustomed to the shared digital environment, since it is less intuitive than the baseline or the immersive environment and it also needs greater digital skills in order to use the 3D modelling tools. One participant commented: “So my, call it, natural ability to navigate was probably a bit subpar… But I think it has overall limitations because of its disconnectedness… I’m going to say less fluid and less conversational. It really had to be quite specific and targeted, which then takes the personality out of it and it takes the conversation out of it.” Also, some participants believed that the 3D modelling environment is the least effective in explaning their ideas. However, some other participants pointed out the strong capability of the 3D modelling environment for facilitating visual representation and detail as one commented: “There’s an immediacy to seeing quite a strong visual… if we want to go into details, maybe the second [3D modelling environment]… had a lot of information.” For the immersive environment, participants believed that it is suitable for the conceptual design stage and that its immersive experience is beneficial for collaboration. One participant commented: “It’s physically at a scale where you can feel like you have multiple people engaging, not just two people over a pen and paper. So, …, you can have remote people in, you could all digitally create, and you could do it in different parts of that building at the same time. Whereas that’s very challenging to do over the floor plan.” Also, “Because of the feeling you are standing right there, a feeling of life experience, you could relate with the things. … I think that the technology helps because it was pretty interactive and quickly responds when we are trying to express some ideas there.” On the other hand, some participants and end-users in particilar also admitted that they needed some guidance to understand the immersive environment. Particpants also suggested that although well suited for conceptual design; Hyve-3D is less useful for analytical aspects of the design. For example, urban planners often obtain significant data using GIS software to assist with their decision making, which is not currently connected to Hyve-3D. Most participants agreed that they can see a lot of potential for the immersive technologies to be applied in design and collaboration, as Okeil (2010) suggested, emerging virtual environments are beneficial for increasing participants’ level of engagement, empowering visual thinking, and facilitating a deeper spatial understanding.

58

N. Gu et al.

3.5.2 Modes of Collaboration Compared with the co-located mode (in the face-to-face collaborative design environments and immersive design environments) of collaboration, the remote design collaboration mode (in the 3D modelling environment) exhibits higher coding percentage in Shared Space. This may be because with remote collaboration there is more effort needed in establishing the shared digital space between the participants, while co-located collaboration is more natural and familiar for most participants, where they can also express and communicate more freely using aids such as verbal cues, gestures or expressions. Kniel and Comi (2021) argued that remote team members “struggle to communicate information that is nonverbal, to prevent misinterpretations or information loss, and to transfer the right amount of information over communication channels.” Remote collaboration has a strong reliance on effective communication technologies (Hinds and Weisband 2003), and the collaborative method used remote team communication is crucial to the success of the remote team (Jordan and Adams 2016). Kleinsmann et al. (2012) states that knowledge is an important cause of successful collaboration, and in all three design environments, whether remote or co-located, participants tended to be able to apply their professional knowledge or personal experience to set up their shared goals and sometimes also solve design problems during collaboration. Co-located collaborators more frequently established shared goals while solving design problems in the immersive design environment, and remote collaborators tended to more frequently set up shared goals while exploring shared digtial spaces in the 3D modelling environment.

3.5.3 Two Different Team Dynamics in Design Collaboration Across the Three Design Environments Collaboration between the two types of design team dynamics was found to be different in many areas across the three design environments. The coding percentages of Expertise, Organising Mechanism and Problem Solving were all higher in the architect and end-user teams compared to the architect and urban planner/designer teams. The cause may be because the architects often needed to explain expert knowledge to the end-users (who are non-experts) and take the lead more frequently during collaboration. Also, the architect and end-user teams in our study tended to push harder to solve the design problem (perhaps due to the strong personal interest and relevance of the end users), while in the expert-only teams, they tended to focus more on reviewing and explaining the design and alternatives to each other from their disciplinary perspectives rather than focusing on detailing the problems. On the other hand, the expert-only teams established shared goals more frequently than the other teams, which means it may be more effective for experts to work with other

3 Evaluating the Use of Digital Technologies to Support Design …

59

experts (compared to with non-experts) to set up shared goals and therefore progress and deepen the design. Across the three design environments, collaboration between the two different team dynamics was very different in terms of Expertise in the face-to-face collaborative design environment compared to the other two digital environments. In the face-to-face collaborative design environment, the architect and end-user teams have a much higher Expertise coding percentage than the expert-only teams, while in the other two digital environments the Expertise coding percentage was similar during design collaboration. The particular reason of this may be due to the end-users finding the face-to-face collaborative design environment relatively more natural and familiar, which made them relaxed and more inclined to discuss their personal experience. As one end-user said: “Having information that is provided and having some knowledge of the environment made it very easy for me to provide comments.” Problem Solving exhibited a higher coding percentage in the architect and end-user teams than in the expert-only teams in both digital design environments, while the result was the exact opposite in the face-to-face collaborative design environment. The likely cause may be that design professionals are more accustomed to using traditional sketching (featured in the face-to-face collaborative design environment) for problem solving especially during conceptual design. As one architect stated: “It was easy to achieve the task that we required. Using a pen and paper is quite common for me. So I think the tools and the access to the pen and paper, it’s a basic part of what we do as an architect.” … “Quite an easy process to, again, draw over a piece of paper and then explain how you might move through the space or where things are located.” As for Shared Space, in the face-to-face collaborative environment, the expert-only teams had a higher coding percentage than the other teams, while in the immersive design environment the architect and end-user teams had a higher coding percentage. The former is indicative of experts paying more attention to establishing shared spaces during the baseline experiment and once again may be due to their familiarity and preference for traditional sketching in design collaboration and exploration. The latter potentially suggests a lack of end-users’ 3D spatial cognition ability, whereby their expert collaborators may be required to frequently orient and familiarise them during collaboration in the immersive design environment. The architect and urban planner/designer teams tended to more frequently establish shared goals by applying their expert knowledge than the other teams. The same pattern was also observed when the expert-only teams solved design problems and expended effort in establising and exploring shared spaces in the design environment, compared to the other teams. These results suggest that expert knowledge is essential in supporting effective collaboration in both the physical and digital design environments. This finding is in accordance with Dong (2005) argument that expert knowledge could contribute to shared mental representations of design, which can be interpreted as shared goals. For example, one urban planner in our study commented: “In some cases architects don’t have enough knowledge about the urban regulations and construction regulations… because we have different expertise, I mean I’m an urban planner and she’s an architect, and there’s a little bit of lack of knowledge for me about architecture and for her about urban planning.” As a result, and it is through the

60

N. Gu et al.

process of exchanging and complimenting each other’s expert knowledge, together they establish shared goals and solve design problems through collaboration.

3.5.4 Contribution of the Study In relation to understanding digital technologies for supporting design collaboration, the immersive design environment shows similarities with the 3D modelling environment. Thus, we can infer that the additional immersive qualities of this advanced design environment (relative to the conventional non-immersive 3D modelling environment) do not appear to occupy substantially more cognitive load for the design teams during collaboration. This finding is significant, because it shows that designers can potentially take full advantage of the advanced features offered by this new digital technology in design collaboration, without a substantial increase in their cognitive load, which is often a main barrier that can hinder the adoption of new technologies. Studies have established the benefits of immersive virtual environments for design that fully immerse users inside the virtual world which they become a part of Mostafavi (2021); and suggested that immersive virtual environments can facilitate more intuitive interactions between designers and also between designers and the design environment (Pour Rahimian et al. 2020) leading to better spatial cognition (Paes et al. 2017), which may be beneficial for more effective design collaboration. These were echoed by our participants who considered the immersive environment is more interactive and engaging and has significant potential for further supporting design. Immersive design environments replicate physical built environments more realistically compared to typical 3D modelling environments. Therefore, it is also relatively easy for both expert and non-expert designers to understand the space and establish/explore the shared working environment. As shown in the results, a higher coding percentage of Shared Space was observed in the remote collaboration mode (3D modelling environment) than in the co-located mode (face-to-face collaborative design environment and immersive design environment), which suggests the remote collaboration mode and/or the more complex design/modelling features of the 3D modelling environment may have caused the additional effort related to Shared Space. Further, end-users are often more attracted to novelty and learning/using new technologies, which make immersive design environments very ideal for end-user engagement. The advancement of immersive technologies is leading the digital infrastructure upgrade in the AEC industry, and future research is needed concerning multiple key stakeholders including design professionals and organisations, as well as software/ hardware developers. Immersive design environments are still evolving, and research is needed to critically understand how such environments may support or hinder design collaboration, and the level of immersion may impact designers. In practice, the wider adoption of immersive technologies in the AEC industry may also require cultural shift from various parties, especially from organisations or the entire sector.

3 Evaluating the Use of Digital Technologies to Support Design …

61

For example, immersive design visualisation is beneficial for engaging with endusers to support co-design and to boost customer satisfaction; however, at a practical level the additional investment/workload may become a potential barrier and would need to be recognised and addressed through budgetary or other contractual means. Another main contribution of the study is that it further extends the existing collaborative practice model in the following two ways. Firstly, it enriches and adapts the general definitions of the nine key elements of the model into five categories specific for design collaboration, as defined in the coding scheme of this study (see Table 3.1). Secondly, the study defines ‘overlapping coding’ which explored overlapping conditions through of the key elements in design collaboration and shows that they are not separate discrete elements, in which demonstrates and affirms such potential of the collaborative practice model (Pablo and London 2020). In specific empirical settings as described in this study, pairs of elements have been shown to be closely linked. This finding will have practical implications, for example, in building collaborative capacity of teams, such overlapping conditions suggest that skill development should be made more inclusive, and can consider/prioritise compensatory skills. For example, since Shared Goals and Expertise have substantial overlaps, when resources are constrained or deadlines are tight, then it may become possible (though not ideal) to drive one instead of having to build capacity in both areas, for example, collaborative capacity building might mean taking pathways that rely more heavily on expertise, instead of having teams go through prolonged processes of negotiating for shared goals.

3.6 Conclusion This chapter presents a preliminary study with some early results from four design teams collaborating across three different design environments: A traditional faceto-face collaborative design environment as the baseline, and two different digital technologies, namely a typical 3D modelling environment, and an advanced immersive design environment. The study compares two types of team dynamics: architects collaborating with end-users, and architects collaborating with urban planners/ designers. The results suggest that differences in design collaboration were apparent when comparing the three design environments, in terms of how participants utilise expert knowledge and/or personal experience (Expertise), and how participants establish/explore the shared physical or digital design environment (Shared Space). Collaborative design behaviours of the two types of team dynamics also exhibit differences in most areas, including Expertise, Organising Mechanism, Problem Solving, and Shared Goals. As a preliminary investigation, we are aware that the sample size of this experiment was limited, thus may not be sufficient to generate statistically significant results. In addition, the varying levels of individual participants in terms of both expert knowledge and digital skillset may potentially cause bias to some extent, and individual team dynamics may also affect the results. Our immediate future studies will consider

62

N. Gu et al.

enlarging the sample size to obtain more generalisable findings. Another future direction will examine additional aspects of collaboration using the date collected, potentially including an interpretive layer by incorporating gesture or body language, as suggested by Shaw (2019), to augment the current verbal communication protocol analysis, to further enrich these findings. Acknowledgements This research was funded by the Australian Research Council’s Discovery Projects scheme (Project ID: DP180101178). The authors would like to acknowledge the contributions of all participants of the design experiments.

References Abdelhameed W (2012) Micro-simulation function to display textual data in virtual reality. Int J Archit Comput 10:205–218. https://doi.org/10.1260/1478-0771.10.2.205 Azmi NF, Chang SC, Li WC (2018) Building information modeling (BIM) in architecture, engineering and construction (AEC) industry: a case study in Malaysia Combrinck C, Porter CJ (2021) Co-design in the architectural process. Archnet-IJAR 15(3):738– 751. https://doi.org/10.1108/ARCH-06-2020-0105 Cross N (2001) Design cognition: results from protocol and other empirical studies of design activity. In: Charles E, Michael M, Wendy N (eds) Design knowing and learning: cognition in design education. Elsevier Science, Oxford, pp 79–103 Dong A (2005) The latent semantic approach to studying design team communication. Des Stud 26(5):445–461 Dorst K, Dijkhuis J (1995) Comparing paradigms for describing design activity. Des Stud 16(2):261– 274. https://doi.org/10.1016/0142-694X(94)00012-3 Eastman CM (1970) On the analysis of intuitive design process. Emerging methods in environmental design and planning Cambridge. The MIT Press, Mass Ens B, Lanir J, Tang A, Bateman S, Lee G, Piumsomboon T, Billinghurst M (2019) Revisiting collaboration through mixed reality: the evolution of groupware. Int J Hum Comput Stud 131:81– 98. https://doi.org/10.1016/j.ijhcs.2019.05.011 Ericsson KA, Simon HA (1980) Verbal reports as data. Psychol Rev 87(3):215–251. https://doi. org/10.1037/0033-295x.87.3.215 Ericsson KA, Simon HA (1993) Protocol analysis: verbal reports as data. MIT Press, Mass Feast L (2012) Professional perspectives on collaborative design work. CoDesign 8(4):215–230. https://doi.org/10.1080/15710882.2012.734828 Froese TM (2010) The impact of emerging information technology on project management for construction. Autom Constr 19(5):531–538. https://doi.org/10.1016/j.autcon.2009.11.004 Garber R (2014) BIM design: realising the creative potential of building information modelling Gero JS, McNeill T (1998) An approach to the analysis of design protocols. Des Stud 19(1):21–61. https://doi.org/10.1016/s0142-694x(97)00015-x Goldschmidt G (1995) The designer as a team of one. Des Stud 16(2):189–209. https://doi.org/10. 1016/0142-694X(94)00009-3 Gomes D, Tzortzopoulos P (2018) Building shared understanding during early design. In: IGLC 2018—proceedings of the 26th annual conference of the international group for lean construction: evolving lean construction towards mature production management across cultures and frontiers Gross MD, Do E-L, McCall RJ, Citrin WV, Hamill P, Warmack A, Kuczun KS (1998) Collaboration and coordination in architectural design: approaches to computer mediated team work. Autom Constr 7(6):465–473. https://doi.org/10.1016/S0926-5805(98)00055-7

3 Evaluating the Use of Digital Technologies to Support Design …

63

Gu N, Kim MJ, Maher ML (2011) Technological advancements in synchronous collaboration: the effect of 3D virtual worlds and tangible user interfaces on architectural design. Autom Constr 20(3):270–278. https://doi.org/10.1016/j.autcon.2010.10.004 Gül LF, Maher ML (2009) Co-creating external design representations: comparing face-to-face sketching to designing in virtual environments. CoDesign 5(2):117–138 Hay L, Duffy A, McTeague C, Pidgeon L, Vuletic T, Grealy M (2017) A systematic review of protocol studies on conceptual design cognition: design as search and exploration. Des Sci 3:E10. https://doi.org/10.1017/dsj.2017.11 Hinds PJ, Weisband SP (2003) Knowledge sharing and shared understanding in virtual teams. In: Cohen SG, Gibson CB (eds) Virtual teams that work: creating conditions for virtual team effectiveness. Jossey-Bass, pp 21–37 Hong SW, El Antably A, Kalay YE (2019) Architectural design creativity in multi-user virtual environment: a comparative analysis between remote collaboration media. Environ Plann b: Urban Anal City Sci 46(5):826–844. https://doi.org/10.1177/2399808317733267 Jordan S, Adams R (2016) Perceptions of success in virtual cross-disciplinary design teams in large multinational corporations. CoDesign 12(3):185–203. https://doi.org/10.1080/15710882.2016. 1146303 Kalay YE, Khemlani L, Choi JW (1998) An integrated model to support distributed collaborative design of buildings. Autom Constr 7(2):177–188. https://doi.org/10.1016/S0926-5805(97)000 54-X Kan JWT, Gero JS (2009) Using the FBS ontology to capture semantic design information in design protocol studies. In: McDonnell J, Lloyd P (eds) About: designing. Analysing design meetings. Taylor & Francis, New York, pp 213–229 Kim MJ, Maher ML (2008) The impact of tangible user interfaces on spatial cognition during collaborative design. Des Stud 29(3):222–253. https://doi.org/10.1016/j.destud.2007.12.006 Kim MJ, Hwang YS, Hwang HS (2020) Utilising social networking services as a collective medium to support design communication in team collaboration. Archnet-IJAR: Int J Architectural Res 14(3):409–421. https://doi.org/10.1108/ARCH-02-2020-0025 Kleinsmann M, Deken F, Dong A, Lauche K (2012) Development of design collaboration skills. J Eng Des 23(7):485–506. https://doi.org/10.1080/09544828.2011.619499 Kniel J, Comi A (2021) Riding the same wavelength: designers’ perceptions of shared understanding in remote teams. SAGE Open 11 Leon M, Laing R, Malins J, Salman H (2015) Making collaboration work: application of a conceptual design stages protocol for pre-BIM stages. WIT Trans Built Environ 149:205–216 Liu X, Wang X, Wright G, Cheng JCP, Li X, Liu R (2017) A state-of-the-art review on the integration of building information modeling (BIM) and geographic information system (GIS) 6(2):53 London K, Pablo Z (2017) An actor–network theory approach to developing an expanded conceptualization of collaboration in industrialized building housing construction. Constr Manag Econ 35(8–9):553–577. https://doi.org/10.1080/01446193.2017.1339361 London KA, Pablo Z (2021) Simulation-based collaboration training: strengthening the industry’s capacity to collaborate. In: Underwood J, Shelbourn M (eds) Handbook of research on driving transformational change in the digital built environment. IGI Global Publishing, pp 404–429 McCall R, Johnson E (1997) Using argumentative agents to catalyze and support collaboration in design. Autom Constr 6(4):299–309. https://doi.org/10.1016/S0926-5805(97)00041-1 McGrawHill Construction (2014) Smart market report the business value of BIM for construction in major global markets. McGraw Hill Construction, New York Merschbrock C, Munkvold BE (2015) Effective digital collaboration in the construction industry— a case study of BIM deployment in a hospital construction project. Comput Ind 73:1–7. https:// doi.org/10.1016/j.compind.2015.07.003 Milovanovic J, Gero J, Becker K (2021) Does it matter where design teams come from in design studies? In: ASME 2021 international design engineering technical conferences and computers and information in engineering conference

64

N. Gu et al.

Mostafavi A (2021) Architecture, biometrics, and virtual environments triangulation: a research review. Architectural Sci Rev 1–18. https://doi.org/10.1080/00038628.2021.2008300 Moum A (2010) Design team stories: exploring interdisciplinary use of 3D object models in practice. Autom Constr 19(5):554–569. https://doi.org/10.1016/j.autcon.2009.11.007 Oh M, Lee J, Hong SW, Jeong Y (2015) Integrated system for BIM-based collaborative design. Autom Constr 58:196–206. https://doi.org/10.1016/j.autcon.2015.07.015 Okeil A (2010) Hybrid design environments: immersive and non-immersive architectural design. Itcon 15:202–216 Pablo Z, London K (2017) Developing collaborative capacities in industrialised building: roadmap for knowledge transfer. In: Proceedings of the joint CIB W099 and TG59 international safety, health, and people in construction conference: towards better safety, health, wellbeing, and life in construction Pablo Z, London KA (2020) Stable relationality and dynamic innovation: two models of collaboration in SME-driven offsite manufacturing supply chains in housing construction. Eng Constr Archit Manage 27(7):1553–1577 Paes D, Arantes E, Irizarry J (2017) Immersive environment for improving the understanding of architectural 3D models: comparing user spatial perception between immersive and traditional virtual reality systems. Autom Constr 84:292–303. https://doi.org/10.1016/j.autcon.2017. 09.016 Rahimian P, Farzad SS, Oliver S, Rodriguez S, Dawood N (2020) On-demand monitoring of construction projects through a game-like hybrid application of BIM and machine learning. Autom Constr 110:103012. https://doi.org/10.1016/j.autcon.2019.103012 Schön DA (1984) Problems, frames and perspectives on designing. Des Stud 5(3):132–136. https:// doi.org/10.1016/0142-694X(84)90002-4 Shaw E (2019) Gesture in Multiparty Interaction. Gallaudet University Press Shealy T, Gero J, Mo H, Milovanovic J (2020) Concept generation techniques change patterns of brain activation during engineering design. Design Sci 6:e31. https://doi.org/10.1017/dsj.202 0.30 Stempfle J, Badke-Schaub P (2002) Thinking in design teams—an analysis of team communication. Des Stud 23(5):473–496. https://doi.org/10.1016/S0142-694X(02)00004-2 Succar B (2009) Building information modelling framework: a research and delivery foundation for industry stakeholders. Autom Constr 18:357–375 Suwa M, Tversky B (1997) What do architects and students perceive in their design sketches? A protocol analysis. Design Stud 18(4):385–403. https://doi.org/10.1016/S0142-694X(97)000 08-2 Tan L (2021) Collaborative cultures of architecture teams: team learning and reflective practice. Des J 24(3):489–498. https://doi.org/10.1080/14606925.2020.1864942 Valkenburg R, Dorst K (1998) The reflective practice of design teams. Des Stud 19(3):249–271. https://doi.org/10.1016/S0142-694X(98)00011-8 Yu R, Gero J (2015) An empirical foundation for design patterns in parametric design. In: Proceedings of the 20th international conference on computer-aided architectural design research in Asia (CAADRIA 2015). Daegu, South Korea

Chapter 4

Multimodality in Virtual Co-urban Design Shuva Chowdhury

and Marc Aurel Schnabel

Abstract A multimodal co-design in virtual urban design refers to a human– computer interaction system composed of the comprehensive usage of various input and output channels. The chapter introduces a tailored design thinking framework for the Multimodal Virtual Communication System for Co-Urban Design (MVCSD), demonstrated via practical multimodal scenarios. A distributed modality design unit has been developed to test the scope of active virtual collaboration with the generation of instantaneous urban artefacts. As input, the system generates the artefacts with VR hand controllers, which produce visual cues of new urban scenarios on screens and immersive environments so that the co-designers can actively participate in the design decision-making process. The enhanced perceptual understanding of the virtual environments, the generated visual cues of the 3D artefacts, the affordance of the virtual tools and their instant visual feedback, and verbal exchange lead to the decision-making process attained at the negotiated level. The research explores the embodiment of collective intelligence within these contexts, underlining the fusion of diverse insights and perspectives that propel innovative urban design outcomes. Keywords Virtual environment · Co-urban design · Collective intelligence

4.1 Introduction Technology’s rapid advancement has profoundly influenced how we perceive and interact with the world in recent years. As the boundaries between physical and virtual realms blur, innovative tools and systems have emerged to enhance communication and collaboration in various domains. Among these transformative technologies,

S. Chowdhury (B) North Carolina Agricultural and Technical State University, Greensboro, NC 27411, USA e-mail: [email protected] M. A. Schnabel CIC FORUM8 Lab, Minato, Tokyo, Japan © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 J. H. Lee et al. (eds.), Multimodality in Architecture, https://doi.org/10.1007/978-3-031-49511-3_4

65

66

S. Chowdhury and M. A. Schnabel

a multimodal virtual communication system stands at the forefront, redefining the landscape of co-urban spatial design and the design thinking process behind it. The traditional approaches to urban design have often been limited by their reliance on physical models and static representations. Such methodologies, while essential, cannot often convey complex urban concepts effectively and elicit meaningful feedback from stakeholders. However, our illustrated exploration found that with the advent of multimodal virtual immersive communication systems, urban designers and planners can possess a powerful tool to transcend these limitations and create immersive experiences that engage and empower various stakeholders in decision-making. This chapter explores the MVCSD (Multimodal Virtual Communication System for Co-Urban Design) process through tested case studies to engage non-expert designers in a virtual environment to collaborate actively as a design unit to develop the negotiated state of their collective decisions. The chapter at the beginning discusses the virtual co-design process, communication, perceptual awareness, and collective intelligence. Then, sections illustrate an example of engaging the community in the virtual enhanced communicative medium to make collaborative urban design decisions. Finally, the presence of collective intelligence that emerges from collaboration in the remote decision-making process is discussed.

4.2 Background This section looks at (i) the concepts of co-design and the role of artefacts in the design process, (ii) the impact of perceptual awareness in collaborative virtual environments, (iii) the concept of coding in communication research and finally, and (iv) the meaning of collective intelligence and its relationship to virtual co-design.

4.2.1 Concept of Co-design and Artefacts Co-design or collaborative design is a process where different parties like designers, architects, engineers and sometimes clients work together to achieve a shared design goal (Gül 2020). They work together on a design artefact or parts of it. Co-design is generally an individual’s mental process that establishes shared goals and develops a shared understanding of the design brief, searching through design precedents for inspiration, defining design constraints, framing and examining design problems, and materialising a design solution. In a “co-located” situation, architects or designers are in the same room, allowing natural verbal and visual communication between parties. Visual communication’s effectiveness depends on “shared representations” and a shared workspace. The term shared representation means all types of external representations that architects or designers rely on for communicating design ideas

4 Multimodality in Virtual Co-urban Design

67

through artefacts like sketches, physical or digital models, diagrams, graphs or notations. In a co-design approach, external representation plays a significant role in interacting with the artefacts. When design thoughts are externalised through artefacts, each artefact contains properties for future interpretations that designers can negotiate during further design development. Artefacts that can be pointed to, talked about or sketched on, play an important role in the conversation with oneself and others (Perry and Sanderson 1998). Also, these external representations become the ground for conflicts and collaboration. Arias et al. (2000) argue that externalisations are important for two reasons: (1) to create a record of mental efforts, one that is “outside us” rather than a vague memory, and (2) to represent artefacts that can talk back to us. Studies have shown in co-design situations that the process of collaborative negotiation and evaluation depends on the expertise of the designers (Gül and Maher 2009; Kvan et al. 1998). During co-design, designers want to become aware of each other’s activities. Being unaware of others’ activities would break the flow of codesigning. Gutwin and Greenberg (1998) argue that in co-design, workspace awareness is important for two reasons: the amount of power it provides to the user and its degree of visibility to the group in collaborative work. Workspace awareness can be achieved through “consequential communication” and feedback. Consequential communication allows the characteristic movements of an action to communicate its character and content to others (Gül 2020). Feedback is produced when artefacts are manipulated and provide clues of that manipulation to others (Dix et al. 2003).

4.2.2 Perceptual Awareness in Collaborative Virtual Environment Perceptual awareness is important in virtual design collaboration, as evidenced in ethnographic studies (Maher 2011). This is because independent participants in the collaborative design process need to be able to coordinate and inform their activities through background or peripheral awareness of one another’s activities. So, Collaborative Virtual Environments (CVE) provide new ways to meet communication needs when negotiation is important and frequent. An important aspect of collaborative design is that the meeting focuses on the design ideas and models rather than only a discussion between designers. Developing a shared understanding of the design problem and potential solutions is necessary. However, communication among the participants in the environment allows individuals to pursue their tasks and focus on a shared task. Also, studies report that designers move fluidly from working individually to working together when engaged in virtual collaborative design. The affordance of peripheral awareness for collaborative design is explored and documented in research by Gül and Maher (2009) in the article on co-creating face-to-face sketching and designing in VEs. The research develops a coding scheme to analyse different modes of interactions through external representations to

68

S. Chowdhury and M. A. Schnabel

assist design collaboration. The article refers to external representations of drawing sketches and digital models. The encoded protocol represents the context of collaborative designing, how designers collaborate and communicate, and their interactions with the design representation. The study aims to characterise the collaborative design process when designers use traditional materials—pen, paper, scale, etc.—and digital systems for designing and communication. The comparison study is done in three collaborative design sessions: face-to-face with remote sketching, face-to-face with a 3D virtual world, and face-to-face with 3D virtual world sketching in a fixed period on similar design tasks. The participation of the designers is video recorded with the conversation. One of the results of this study showed that in a remotely located virtual world, the designers can move from meeting mode to individual work mode while coordinating with their collaborators. It also reports that the designers conduct continuous actions while designing in the 3D modelling environment with more detail in the co-created representation. The study ends by stating that the developments in collaboration and design technology encourage designers to consider new media for communication and design. The cognitive impact of CVE on designing must, therefore, be addressed. It also suggests that analysing the collaborative design protocols provides a basis for a better understanding of the interactions with different representation techniques. The authors speculate that the acquired knowledge has implications for both developments in future CVEs and choosing an appropriate medium for designing. However, the study does not explore the possibility of design collaboration when immersed in the VE, where the cognitive load is significantly less to perceive. According to Marshall McLuhan, “media are the extensions of mankind,” so the perceptual affordance of digital media has made a significant contribution to virtual design collaboration. Koutsabasis et al. (2012) investigate the value of VWs affordances and tools that can contribute to collaborative design projects that involve designers’ cooperation and client feedback. The affordance of VWs is exploited to foster collaborative activities in various stages of design: communication, embodiment, presence and co-presence, 3D visualisation and interaction, and increased user engagement, as well as all of the above. Despite the increasing interest in exploring the affordances of VWs as a platform or “tool” for mediating collaborative design activities, design studies in VWs are still scarce. This is because the design community still shows interest in the pragmatic uses of technologies in existing practices. However, the value of VWs for collaborative design activities can contribute to the phases and activities of authentic collaborative design projects that involve designers’ cooperation and client feedback. The design community has not accepted Collaborative Virtual Environments (CVEs) for their everyday work, possibly because of the high cost of immersive hardware and the limited availability of generic software platforms for immersive VR applications combined with the time and money needed to develop customised solutions.

4 Multimodality in Virtual Co-urban Design

69

4.2.3 Concept of Coding in Communication and Protocol Analysis Donnellon et al. (1986) draw an interesting point: that the minimal degree of shared understanding need not be conscious or verbalisable; it can be a repertoire of behavioural options that members of a given society can recognise, respond to, and use to interact with one another. This repertoire of communication mechanisms is the means through which people collectively develop interpretations of their experiences. Despite apparent differences in the interpretations of those behaviours, shared forms of communication forge some agreements. To identify the shared forms of communication, Donnellon et al. (1986) propose semantic coding of the transcription of interaction that could be found after a recording. This also helps to identify shifts of interpretation of the events and members’ inclinations towards specified actions. He also proposes linguistic analysis to examine the sequential and multi-level communication behaviours associated with the actions of the members. He concludes the article with a proposition that in the communication process, meaning and action are related in a complex iterative process in which meanings are continually constructed and destroyed as more sense-making communication occurs and new actions are taken. Protocol analysis with a coding scheme is used to identify different design activities and reveal different mental models and the knowledge structures of designers. In general, protocol data is based on samples of observations that are mainly qualitative (Kan and Gero 2017). It refers to a set of methods for obtaining reliable information about what people think while participating in a task. The importance of protocol analysis is to understand the design process, as it helps to reveal the traits of design thinking between action and perception (Goldschmidt 2014). It is an empirical research method to investigate the cognitive behaviours and thinking processes generally adopted by problem solvers (Akin 1986). Ericsson and Simon (1993) developed the foundation of using verbal protocols and concurrent reporting as quantitative data to examine the thought process. In a collaborative design process, it is impossible for individual members to think out loud; however, they support Cross et al. (1996, p. 3) statement that “the verbal exchanges of members of a team engaged in a joint task seem to provide data indicative of the cognitive activities that the team member is undertaking.” As Kan et al. (2011) state, many researchers use protocol analysis techniques to study design collaboration. They mainly focused on verbal communication as a form of talking aloud and considered the raw protocol data. The concept of thinking loud’ during problem-solving means that the subject keeps talking, speaking out loud while performing the task (Van Someren et al. 1994). This method does not lead to much disturbance in the thought process. The subject solves a problem while the talking is executed. The subject does not give an interpretation of their thoughts. The protocol is not necessarily complete because a subject may verbalise only a part of their thoughts. It is also accepted that protocol analysis is limited in capturing the non-verbal thought processes during the design process; therefore, important non-verbal communication is often neglected.

70

S. Chowdhury and M. A. Schnabel

4.2.4 Collective Intelligence (CI) and Virtual Co-design The relationship between Collective Intelligence (CI) and virtual co-design is symbiotic and complementary. CI leverages the collaborative efforts of individuals with diverse backgrounds and expertise to harness their collective wisdom, problemsolving abilities, and creativity (Leimeister 2010; Murty et al. 2010). Virtual codesign, on the other hand, facilitates this collaboration by providing digital tools and communication platforms that allow participants to work together remotely, breaking down geographical barriers (Lee et al. 2023; Wang et al. 2021). Besides, CI is crucial in virtual design thinking by leveraging participants’ diverse perspectives and knowledge. When individuals from different backgrounds, experiences, and expertise collaborate virtually, they contribute unique insights and ideas that enrich the design thinking process. This collective intelligence fosters a more comprehensive understanding of the problem at hand and encourages the generation of innovative solutions. Through virtual co-design, participants can contribute their ideas, insights, and expertise in real-time, irrespective of their physical location. This collective input enhances the overall pool of knowledge and perspectives, leading to more informed and innovative solutions. Combining CI and virtual co-design allows groups to tackle complex problems and co-create more inclusive, efficient, and practical solutions. In essence, virtual co-design provides the infrastructure and means for collective intelligence to thrive. In contrast, CI enriches the co-design process by tapping into participants’ collaborative capabilities and diversity. They enable organisations and communities to make better decisions, generate groundbreaking ideas, and create solutions that address various challenges. CI plays a significant role in the design domain, primarily transforming how designers and design processes operate. It refers to the synergy and collaboration between individuals, enabling them to collectively contribute their knowledge, skills, and ideas to achieve a common design goal. CI generally depends on three requirements: (i) Communication (C), (ii) Representation (R) and (iii) Motivation (M) (Murty et al. 2010). Effective communication co-located or geographically dispersed plays a key role in developing concepts and providing design commentary on the representation contents. Communication requirements depend on the mode of communication (synchronous like voice chat or asynchronous like email), on the type (direct line between two or more people or indirect through shared representation), on the content (what is being communicated like design idea, comment on the process) and the structure (properties of the communications network connections and distribution). According to Murty et al., the CI is represented in Formula 4.1: CI = {C + R + M}

(4.1)

CI relies on the shared representation of the contents, which may imply media like voice, text, sketches, 2D or 3D models, immersive virtual environments, and representation functions, including visualisation. The final one is motivation, which allows

4 Multimodality in Virtual Co-urban Design

71

the CI applications to be objectified through attraction, intrigue, challenges, and rewards to participants. The motivations can also recognise the social opportunities associated with their material rewards.

4.3 Methodology The methodology is built upon insights from previous research that focused on virtual urban co-design. By re-evaluating the outcomes of these experiments through the lens of CI requisites in collaborative design, the research here explores a novel perspective on these prior results gained through protocol analysis. To accomplish this, the initial protocol schemes introduced by Chowdhury and Schnabel (2020) are first restructured to align with the imperatives of communication, representation, and motivations. Then, contextualisation and reanalysis of the experiment’s findings were scrutinised against these newly established criteria. A multimodal design unit had been developed primarily based on an interaction between human–human and human–computer—the first participant designs by being immersed in the environment via a Head-Mounted Display (HMD) using the VR controllers to create the 3D artefacts in the virtual environment. The second participant visualises the design in real-time on the 80-in. display screen (Fig. 4.1). Both discussed and decided together on the interactions with the model. Participants were recruited through social media and poster invitations. Participants were based in New Zealand, of mixed ages, and were familiar with the site’s context. There were three sessions, each of one and a half hours long. Fig. 4.1 Arrangement of design team forming a design unit

72

S. Chowdhury and M. A. Schnabel

Fig. 4.2 The system architecture used in MVCSD

The design participation happened in groups. A design task was introduced, supporting the contextual requirements. For the design proposal in a local suburb of Wellington, New Zealand, participants were asked to design building blocks on the empty corner plot of the neighbourhood. The session began by introducing the participants to the virtual instruments and familiarising them with the site on a Google map. The participants could extend their design ideas beyond the assigned plot if they wished. The design sessions were held at the local community centre right next to the site. Design conversations were recorded for protocol analysis. The 3D models produced were saved for experts to evaluate the outcomes. Due to its flexibility to create iterative 3D models through hand gestures, an immersive instrument “Sketch Space” (Innes et al. 2017) has been developed using in-game-engine software “Unity3D”. Sketch Space was adapted and extended to match the surrounding urban context. One person at a time is immersed in the virtual environment, whilst the other person visualises real-time design output on a 2D display screen and provides verbal feedback to the first participant. The research designed and developed an immersive virtual multimodal human– computer interaction system, MVCSD (Multimodal Virtual Communication System for Co-Urban Design) and implemented it as a client architecture system, as shown in Fig. 4.2.

4.4 Findings The transcribed conversation was coded to organise the data. The details of the schemes and results are reported in (Chowdhury 2020; Chowdhury and Schnabel 2020). The coding scheme was applied to investigate, analyse, and understand how laypeople as designers communicate with the design instrument and control design ideas in VE. The four major categories of the scheme are (i) Communication Control (CC), (ii) Design Communication (DC), (iii) Social Communication (SC), and (iv) Communication Technology (CT). The coding scheme is explained in Table 4.1. These four major categories can be categorised under the requirements of CI’s Communication (C), Representation (R) and Motivation (M).

4 Multimodality in Virtual Co-urban Design

73

Table 4.1 The coding scheme for VR collaboration used in Chowdhury and Schnabel (2019) Communication control

Code

Description

Interruption by design

INT

When a design member interrupts another member

Interruption by instrument

INTS

When a design member interrupts by instrument functioning. E.g., wrong button/unexpected VR movement/instrument shut down

Handing-over the conversation

HAN

Handing over the conversation from a design member to another member. Possibly through questions or by specifically naming the next speaker

Pause

PAU

Pausing during the communication

Design communication

Code

Design concept

Description What is communicated

Introduction of idea

IDE

When a design member directly or indirectly introduces an idea

Acceptance of idea

ACC

When a design member accepts an idea of another member

Rejection of idea

REJ

When a design member does not accept an idea of another member

Clarification of idea

CLA

When a design member explains why the idea is appropriate

Seek clarification of idea

CLAS

When a design member seeks clarification of another member’s decision

Development of idea

DEV

When a design member further develops an idea

Evaluation of idea

EVA

When a design member spends time evaluating an idea

Design detail

How the concept is created

Discussion of size

VSZ

When design members discuss the size of the 3D object/ building

Discussion of shape

VSP

When design members discuss the shape of a 3D object/ building

Discussion of movement

VSM

When design members move in the VR environment

Discussion of type

VST

When design members discuss building types

Discussion of space

VSS

When design members discuss spatial attributes. E.g., site entry, openness or closeness, orientation, etc.

Discussion of colour/ texture

VCL/ VTXT

When design members discuss the colour and texture on a 3D building or parts of it

Design task

How the design is implemented

Task questioning

TKQ

When design members ask questions about their design task

Agenda referring

AAR

When design members refer to the agenda

Instructing

INS

When a design member instructs another member

Working status

VWS

When design members state what they are currently doing or what they have done. E.g., “I just finished the walls.” (continued)

74

S. Chowdhury and M. A. Schnabel

Table 4.1 (continued) Social communication

Code

Description

Non-task-related social communication

NRT

When design members talk about non-task-related things

Joking

JOK

When a design member laughs or makes a joke

Communication technology

Code

Description

VR instrument

VTL

When design members discuss the use of tools for design in the VR environment

Examining

EXA

When a design member examines what has been done by using the instrument

The subcategories of Communication Control (CC) can be placed under the CI’s Communication (C) category. It has four subcategories: “Interruption by Design” members (INT); “Interruption by Instrument” (INTS); “Handing-over the Conversation” (HAN); and “Pause” during the communication (PAU). Pause (PAU) is used if there is a temporary cessation of conversation during design collaboration in a virtual environment. We added the section of INTS due to computer and software running interruptions. The subcategories of the Design Communication (DC) scheme also can be placed under CI’s Communication (C) category. DC had been sub-categorised by Design Concept, Design Details, and Design Tasks, where sub-category Design Details and Design Tasks can be moved under CI’s Representation (R). Design Concept includes how design ideas are handled during the design process such as “Introduction of Idea” (IDE), “Acceptance of Idea” (ACC), “Rejection of Idea” (REJ), “Clarification of Idea” (CLA), “Seek Clarification of Idea” (CLAS), “Development of Idea,” and “Evaluation of Idea” (EVA). Design Detail (DD) sub-categories can be placed can be placed under CI’s Representation (R) category. It comprises the sub-categories “Discussion of Size” (VSZ), “Discussion of Shape” (VSP), “Discussion of Movement” (VSM), “Discussion of Type” (VST), “Discussion of Space” (VSS) and “Discussion of Colour/Texture” (VCL/VTXT). The coding scheme of Design Task (DT) category can be placed under CI’s Motivation (M) includes “Task Questioning” (TKQ), “Agenda Referring” (AAR), “Instructing” (INS), and “Working Status” (VWS). Social communication (SC) also can go under CI’s Communication (C), which comprises “Non-task-related Social Communication” (NRT) and “Joking” (JOK) in between conversations. This coding scheme documents the moments of conversation that are not related to design tasks. The coding scheme of Communication Technology (CT) consists of “VR Instrument” (VTL) and “Examining” (EXA). The VTL scheme is used when design participants discuss the use of the instrument. The EXA scheme documents when design participants discuss their use of the instrument.

4 Multimodality in Virtual Co-urban Design

75

In summary, the coding recategorisation, the cluster sets for CI’s requirements are: C = {CC + DC + SC + CT} = {{INT, INTS, HAN, PAU} + {IDE, ACC, REJ, CLA, CLAS, EVA} +{NRT, JOK} + {VTL, EXA}}

(4.2)

R = {VSZ, VSP, VSM, VST, VSS, VCL/VTXT}

(4.3)

M = {TKQ, AAR, INS, VWS}

(4.4)

After recategorising the results of the protocol analysis, the new results offer a novel understanding of the impact of collective intelligence happened through which aspects. After re-clustering the coding results in percentages under communication, representation and motivation, the results show that the users are usually occupied with the system communication (Fig. 4.3). The frequencies for all codes have been counted in percentages for three sessions. The percentage of communication intelligence for all three sessions was higher than 50%. In two sessions, the percentage for representation comes second highest, and the average value is below 20%. The remaining average value for motivation is close to 15%.

Fig. 4.3 MVCSD collective intelligence

76

S. Chowdhury and M. A. Schnabel

4.5 Discussion Our previous studies on the protocol analysis found that the conversations between the participants were relevant to the design intervention, were context related and arose naturally from the synchronised setup of design engagements and task distribution among the participants (Chowdhury and Schnabel 2019, 2020). The designer acted in the immersive world via HMDs, and fellow participants provided verbal feedback via the 80-in. display screen. In between snippets of conversation, the computer produced 3D urban forms, which provided visual feedback to facilitate further discourse. The design session extended beyond the task-related conversation. It indicated that the continuity of the conversation was naturally free flowing. Participants talked about the impact of a new design on the environment, social cohesion, and the inclusivity of the neighbourhood. The dialogue exchange facilitated the transfer of subject-specific knowledge among the participants. However, it is to be noted that a given percentage of the conversations were disrupted due to technological interruption. Compared to that study, this reinterpretation of the CI’s requirements re-validates the performance of the MVCSD as a part of collective actions. This analysis ensures that the participants spent time developing the design concept, discussing design detail, and referring to the design task. The collective design discussions advanced when every action of the designer produced visual information, which initiated the next level of design action. This can only be done if the design communication media is able to provide such continuous visual feedback to the designer, thus maintaining a successful design interaction. From the CI’s point of view, communication happens through the exchange of information, ideas, knowledge, and options among individuals to achieve a shared goal or solve a problem. In that sense, the findings showed that the MVCSD system leverages the participants to engage with the system and co-designers. Communication happens in multiple modes: communication through the immersive cues of the urban environment, the VR controllers, the generation of 3D artefacts, non-immersive designers’ communication through the 2D display screen, and verbal communication. From the analysis, it seems that the participants occupied significant time in communication. The representation contents of the 3D artefacts were associated with the design tasks, which eventually brings meaning to the continuous communication. The affiliation of the participants with the neighbourhood design tasks and the instant feedback on the design action through verbal and virtual artefacts support the CI’s requirements for motivation. Besides, the perceptual awareness of the virtual environments and the physical awareness of the instrumental modes made communication effective. Such awareness in the MVCSD system also validates the impact of collective intelligence during the co-design sessions. Though, the coding results include some percentage of disruption from technological interruptions. Despite the technical interruption, the technological affordance through the interfaces and hand controllers let the participants engage with the system and eventually with the co-designers.

4 Multimodality in Virtual Co-urban Design

77

As mentioned earlier, the collective motivation happened by including the design tasks, which established the connections between the participants and the MVCSD system. The given tasks and the system’s capability to act on the tasks instantly allowed the participants to take part actively in the design sessions. In the system, predominantly, the human–computer interaction happened via VR hand controller with the immersive designers. The evidence of both analysis, protocol and its further validation through CI’s requirements show that the interactions were instantaneous in producing the visual artefacts. Despite technological disruptions, the system, to some extent, generated visual feedback for the co-designers to perceive meaningfully. However, further investigation is required to measure the level of cognitive load for the immersive users during the phasing of the collective design generation. Besides, the used framework can also be explored in a completely remote environment where the co-designers interact actively with the external mode of communication techniques.

4.6 Conclusion The chapter showcases real-world case studies involving the application of multimodal virtual immersive communication systems in urban design projects, illustrating their successful utilisation to harness collective intelligence within the codesign framework. Venturing into the realm of multimodal virtual immersive communication systems for spatial co-urban design, it becomes apparent that this technology has the potential to reshape our approach to city envisioning, planning, and development. Through a thorough exploration, the chapter enquires about the manifestation of collective intelligence, drawing from protocol analysis findings. By redefining coding schemes in accordance with collective intelligence prerequisites, the study substantiates the profound influence of CI during virtual co-urban design collaboration sessions. The underlying system infrastructure empowers participants to take part actively as co-designers, instigating a significant contribution. This research sets forth a foundation for further evidence-based investigations into the system, aimed at comprehending the intricacies of cognitive behaviour when engaged with the MVCSD system.

References Akin Ö (1986) Psychology of architectural design. Pion Limited Arias E, Eden H, Fischer G, Gorman A, Scharff E (2000) Transcending the individual human mind—creating shared understanding through collaborative design. ACM Trans Comput Human Interact (TOCHI) 7(1):84–113 Chowdhury S (2020) Virtual environments as medium for laypeople’s communication and collaboration in urban design

78

S. Chowdhury and M. A. Schnabel

Chowdhury S, Schnabel MA (2019) Laypeople’s collaborative immersive virtual reality design discourse in neighborhood design. Front Robot AI 6:97. https://doi.org/10.3389/frobt.2019. 00097 Chowdhury S, Schnabel MA (2020) Virtual environments as medium for laypeople to communicate and collaborate in urban design. Archit Sci Rev 63(5):451–464. https://doi.org/10.1080/000 38628.2020.1806031 Cross N, Christiaans H, Dorst K (1996) Introduction: the Delft protocols workshop. Analysing design activity, pp 1–14 Dix A, Dix AJ, Finlay J, Abowd GD, Beale R (2003) Human-computer interaction. Pearson Education Donnellon A, Gray B, Bougon MG (1986) Communication, meaning, and organised action. Adm Sci Q 43–55 Ericsson KA, Simon HA (1993) Protocol analysis. MIT press Cambridge, MA Goldschmidt G (2014) Linkography: unfolding the design process. Mit Press Gül LF (2020) Co-design. https://www.materiart.org/glossaryofmateriart-co-design. Accessed 6 May 2020 Gül LF, Maher ML (2009) Co-creating external design representations: comparing face-to-face sketching to designing in virtual environments. CoDesign 5(2):117–138 Gutwin C, Greenberg S (1998) Design for individuals, design for groups: tradeoffs between power and workspace awareness. In: Proceedings of the 1998 ACM conference on computer supported cooperative work Innes D, Moleta TJ, Schnabel MA (2017) Virtual inhabitation and creation: a comparative study of interactive 1: 1 modelling as a design method. In: Conference: DADA 2017 international conference on digital architecture: “digital culture”. Nanjing, China Kan JWT, Gero JS (2017) Quantitative methods for studying design protocols. Springer Kan JWT, Tsai JJ-H, Wang X (2011) “Scales” affecting design communication in collaborative virtual environments. Collaborative design in virtual environments. Springer Koutsabasis P, Vosinakis S, Malisova K, Paparounas N (2012) On the value of virtual worlds for collaborative design. Des Stud 33(4):357–390 Kvan T, West R, Vera AH (1998) Tools and channels of communication: dealing with the effects of computer mediation on design communication. Int J Virtual Reality 3(3):21–33 Lee JH, Ostwald MJ, Arasteh S, Oldfield P (2023) BIM-enabled design collaboration processes in remote architectural practice and education in Australia. J Archit Eng 29(1):05022012. https:// doi.org/10.1061/JAEIED.AEENG-1505 Leimeister JM (2010) Collective intelligence. Bus Inf Syst Eng 2:245–248. https://doi.org/10.1007/ s12599-010-0114-8 Maher ML (2011) Designers and collaborative virtual environments. In: Collaborative design in virtual environments. Springer, pp 3–15 Murty P, Paulini M, Maher ML (2010) Collective intelligence and design thinking. In: DTRS’10: design thinking research symposium. Indiana, USA Perry M, Sanderson D (1998) Coordinating joint design work: the role of communication and artefacts. Des Stud 19(3):273–288 Van Someren MW, Barnard YF, Sandberg JAC (1994) The think aloud method: a practical approach to modelling cognitive. Citeseer Virtual Environments as Medium for Laypeople’s Communication and Collaboration in Urban Design (2020) Victoria University of Wellington. http://researcharchive.vuw.ac.nz/handle/ 10063/9068. Accessed 14 Aug Wang P, Bai X, Billinghurst M, Zhang S, Zhang X, Wang S, He W, Yan Y, Ji H (2021) AR/MR remote collaboration on physical tasks: a review. Robot Comput Integr Manuf 72:102071. https://doi. org/10.1016/j.rcim.2020.102071

Chapter 5

Hybrid Practice: Exploring the Complexities of Cross-Cultural Collaboration Through the Dialogue of Two International Practices Dijana Alic , Mladen Jadric, and Geun Ju Yoon

Abstract This chapter explores the design and communication processes of a multilingual, multicultural design team using different technical modes. The discussion focuses on the competition-winning project for the ‘Seoul Photographic Art Museum’, undertaken by the two architectural practices ‘Jadric Architektur’ in Vienna, Austria and ‘1990uao’ in Seoul, Korea. Based on the discussions with the design directors, the chapter explores the multi-modal process of online and offline communication between the two practices. Employing the qualitative research method of narrative inquiry, the study delves into the social, cultural, and interpersonal factors that frame the architects’ recollection of the design process. The chapter demonstrates the complexities involved in the development and execution of design development, offering to contextualise the use of various online, offline, digital, and analogue modes of operating necessary in undertaking a project of this size. This chapter reveals that this cross-cultural project highlights the diversity of contextual awareness that underpins architectural collaboration, from specific construction requirements to cultural practices, holidays, and work disruptions. The conversation between these two practices offers valuable insights into the dynamic and demanding processes that the contemporary architectural field encounters while operating within a multicultural and global context, all within the framework of complex technology. Keywords Design collaboration · Cross-cultural collaboration · International practice · Communication

D. Alic (B) School of Built Environment, Faculty of Arts, Design and Architecture, The University of New South Wales, Sydney, NSW 2052, Australia e-mail: [email protected] M. Jadric Institut for Architecture and Design, TU Wien, Vienna, Austria G. J. Yoon 1990 Urban Architecture Office, Jung-Gu, Seoul, South Korea © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 J. H. Lee et al. (eds.), Multimodality in Architecture, https://doi.org/10.1007/978-3-031-49511-3_5

79

80

D. Alic et al.

5.1 Introduction The competition for the Seoul Photographic Art Museum marked the beginning of a challenging and extensive project collaboration. The winning project was the product of a professional partnership between Jadric Architektur based in Vienna and 1990 urban architecture office (1990uao) in Seoul. Referring to the outcomes of the competition in December 2019, the Principal Architect of 1990uao, Geun Ju Yoon expressed that winning was just the first step with the real challenges lying ahead in the realisation of the project. The onset of the COVID-19 pandemic introduced further complexities imposing travel restrictions and impacting the teamwork, design development, and communication between the two offices. The discussion of the collaboration between Jadric Architektur and 1990uao for the Seoul Photographic Art Museum project serves as an exemplary demonstration of the intricate nature of the design process. From the initial conceptual design, which was presented for the design competition, to the final construction and execution of the design (current), a well-established sequential process was followed. However, beneath this apparent linearity lay a complex web of non-linear and iterative processes where the architects not only revisited previous stages and refined proposed solutions, but also negotiated the specifics of the project context, and the policies and processes unique to the project requirements. This approach allowed for a dynamic exploration of ideas and facilitated the integration of new insights and perspectives throughout the design journey. This chapter aims to qualitatively examine the collaborative processes of design, development and communication between the two architectural offices. It argues that understanding the collaborative processes utilised in this project not only contextualises the use of various analogue and digital communication tools but also provides a structured framework that can guide designers through the creative problem-solving journey. Additionally, this chapter expands our understanding of the ‘reflective practitioner’ model to include multi-practice teamwork and multi-lingual communication, offering insights into what constitutes a contemporary hybrid and reflective practice.

5.2 Methodology The phenomenon of cross-cultural design and communication within international teams is inherently dynamic and complex. To comprehensively explore this realm, this chapter employs the qualitative research methodology of narrative inquiry. This method offers a framework for examining the intricate interplay of personal experiences, cultural dynamics, and communication strategies within the context of crosscultural design processes. Drawing upon the foundational works of Riessman (1993), Clandinin (2007), and Clandinin and Michael Connelly (2000), narrative inquiry delves into the significance of personal accounts in constructing meaning. This study employs a dual layer of interpretation within the narrative analysis. The first layer

5 Hybrid Practice: Exploring the Complexities of Cross-Cultural …

81

involves architects Jadric and Yoon’s professional, yet personal accounts of the design process, examined within the framework of “reflective practitioner”. The second layer encompasses the authors’ subsequent collaborative narrative construction supported by a series of discussions involving Assoc. Prof. Dijana Alic, Assoc. Prof. Mladen Jadric, and Prin. Arch. Geun Ju Yoon. This chapter delves into the intriguing intersection of personal accounts and experiences with professional processes and the utilisation of technology. This unique approach establishes a vital connection between the research methodologies employed in the social sciences, and the ongoing discussions surrounding the utilisation of technology within professional contexts. As one navigates through these narratives, intricate layers of meaning deeply woven into the architects’ personal accounts are uncovered. This interpretive process plays a crucial role in facilitating our understanding of the architects’ distinct insights into their experiential journeys. The application of narrative inquiry, a qualitative research method, proves to be well-suited for exploring the multifaceted aspects of the design process, which inherently encompasses both creative and rational dimensions. The notion of the “reflective practitioner” originates from Donald Schön’s seminal work, “The Reflective Practitioner” (1983). Schön’s proposition places a spotlight on the intrinsic value of reflection within professional domains, including architecture and design, as a navigational tool for negotiating intricate and uncertain scenarios. Schön introduces two pivotal components within this framework: “reflection-inaction” and “reflection-on-action.” The former encapsulates the real-time process of thinking and adapting during professional tasks, while the latter encompasses retrospection on past experiences to critically assess actions and gather fresh insights for subsequent practice. This reflective process is underpinned by Schön’s concept of “knowing in action,” denoting tacit knowledge that informs decision-making in the practitioner’s field. The reflective practice concept is commonly understood as a systematic and iterative approach used by designers to solve problems and create innovative solutions. The method builds upon an understanding of the design process, which typically involves several stages or steps that are followed sequentially or cyclically. As a method of work, the design process involves continuous cycles of prototyping, testing, and refining. The linear and cumulative nature of this process involves stages such as conceptual design, design development, detailed design, and construction, and acknowledges the need for the practitioners’ “reflection-in-action” and “reflection-on-action.” Jadric and Yoon’s perspectives on reflective practice were explored in discussion. Both Jadric and Yoon subscribed to the model of reflective practitioners. They recognised and emphasised the crucial importance of clear and genuine communication throughout the design process. “By prioritising understanding and mutual respect, we were able to foster an environment of effective collaboration and bridge the gaps that arose due to our diverse cultural backgrounds,” Jadric concluded. Jadric and Yoon specifically highlighted active listening and mutual respect as key elements that contributed to their success. Ultimately, the architects’ commitment to clear and genuine communication, coupled with their commitment to advancing their architectural agenda, fostered a positive and collaborative working environment.

82

D. Alic et al.

5.3 Findings 5.3.1 The Project Brief and Design Response The Photographic Art Museum competition was initiated by the Seoul Metropolitan Government. The competition, which took place in late 2019, attracted 128 design entries from around the world, and 10 practices were selected for the second round. The project was driven by the objective to address the lack of cultural amenities in the densely populated area of Dobong-gu. The project aimed to create a dynamic ‘cultural mile’ art precinct, fostering an environment where artists and creative professionals could thrive (Design Competition Guidelines 2019). The competition brief identified an ideal location for the Museum, situated between Seoul Arena Theatre to the northeast and Chang-dong Station to the southwest. This strategic positioning aligned the Museum with the broader Chang-dong Sanggye City Regeneration Plan, maximizing its impact. The designated site covered an area of 2500 m2 and proposed a land usage rate of 60% along with a floor area ratio of 400%. The competition guidelines emphasised the transformative nature of the new museum, envisioning its role in shaping the urban fabric of the area. The careful selection of the Museum site accounted for the significant cultural and commercial developments in the surrounding area. On the southwest side of the site, an allocated area was reserved for the construction of the Multi Transfer Centre, ensuring seamless connectivity with the Changdong Private Complex Station. Adjacent to this, the growing Chang-dong Sanggye City Entrepreneurial and Cultural Industrial Complex served as a central hub for local businesses and employment opportunities, further enhancing the overall vibrancy and vitality of the area. The development plans in the vicinity of the Museum site also included the emerging Chang-dong Sanggye City Generation Convergence Startup Centre and the 50 Plus Campus, catering to the needs of different age groups within the community. Efforts were dedicated to the development of the Culture and Art Theme Street, designed to offer rest areas, and showcase public art installations, including sculptures, signs, and lighting. As part of this vision, a proposal was made for a sizable apartment complex on the northern side of the site. Additionally, a Robot and Science Museum, spanning 6305 m2 of floor space, was planned for the eastern end. It was envisioned that the construction of the Robot and Science Museum would proceed concurrently with the development of the Seoul Photographic Art Museum, complementing each other in the overall project design. By situating the Photographic Art Museum within this cultural area, the competition aimed to foster the growth of a vibrant arts community and address the existing gap in cultural amenities in the area. The broader urban intention was to create a dynamic and inclusive environment that supports artistic expression, facilitates collaboration, and engages the local community in Seoul’s cultural landscape (Design Competition Guidelines 2019). The winning proposal for the Photographic Art Museum in Seoul presented a sculptural design, characterised by a monolithic shape. The notable feature of the

5 Hybrid Practice: Exploring the Complexities of Cross-Cultural …

83

Fig. 5.1 Conceptual design

conceptual design in Fig. 5.1 was a dynamic twist in the structure that connected the building’s form to the public realm, enriching both the surrounding public space and the overall architectural identity of the district. On the street level, the proposal connected to the surrounding space by drawing the visitors into an open guest room that offered ample space for exploration and interaction. The seamless connection between the building and its surroundings enhanced the welcoming and inclusive nature of the museum, inviting visitors to engage with the art and the architectural experience. The competition for Seoul’s inaugural dedicated photographic museum sparked discussions surrounding the building program and exhibition content, focusing on Korean photography, its origins, and its current state. Given the relatively young nature of photography as a discipline in Korea, there were no established architectural or exhibition program precedents to guide the process. This absence of guidelines led to questions regarding the inclusion of different types of photographs or documentation in the exhibition. Since there was no specific collection designated for the new museum, the architects faced the task of addressing and responding to these questions. They deliberated on whether to incorporate videos alongside photographs and whether the photographs should be presented in analogue or digital format, or both. These considerations shaped the final proposal for the building exterior (Fig. 5.2) where a photographic façade at once preserves the rich heritage and showcases the evolving landscape of Korean photography.

84

D. Alic et al.

Fig. 5.2 Design proposal—perspective

5.3.2 The Design Process and Communication: Creating the Teams After winning the competition, architecture firms Jadric Architektur and 1990uao found themselves located in different countries across two continents. Although this project marked their first joint success in an international competition, the principal architects of both firms, Mladen Jadric and Geun Ju Yoon, had previously collaborated. Both Jadric and Yoon are not only architectural practitioners in their respective countries but also academics who have worked at various institutions across Europe and Asia. Jadric, with his extensive experience collaborating with Korean and Chinese institutions, described his practice as “more than a typical Austrian office embarking on its first foreign venture.” With a relevant background and established international connections, Jadric brought a wealth of experience and trusted partnerships to the table. According to Jadric, the collaboration with the Korean team was built upon previous experiences and established connections and the extensive work with international practitioners and students, particularly in Korea, that spanned over a decade. This shared history underscored their familiarity with the local landscape, culture, and regulations. In the architectural practice 1990uao, located in Seoul, and led by Geun Ju Yoon, the Viennese team found partners with relevant professional experience and deep

5 Hybrid Practice: Exploring the Complexities of Cross-Cultural …

85

knowledge of local regulations. Their expertise complemented Jadric’s cross-cultural understanding and established connections, fostering a collaborative environment built on the alignment of professional approach and shared expertise. To address the issue of working from different countries, the Vienna office took a strategic step at the project’s outset and appointed a Korean-Austrian architect to assist their office. As Jadric explained, this architect not only served as a translator of language but also bridged cultural and technological differences. Jadric described this appointment as “indispensabl”, as having someone deeply immersed in both worlds facilitated seamless collaboration between the two offices, with “no issues encountered throughout the process”. By leveraging their collective experience and established networks, the Jadric Architektur and 1990uao teams could draw upon a foundation of trust and a shared understanding of cross-cultural dynamics. This enabled them to navigate the intricacies of the project with confidence and effectively merge their respective knowledge and perspectives.

5.3.3 Communication and Engagement The first post-competition stage was marked by the Seoul Metropolitan Government’s revision of the project guidelines. The most significant was the request for an increase in the capacity of the building by almost 30%. In addition, the Metropolitan Government office was required to ascertain community involvement and feedback on the projects run by the city. This placed a strong emphasis on user-centeredness, ensuring that the design catered to the needs and preferences of museum visitors. The requirement to respond to the feedback on the competition-winning proposal provided by the community and diverse other stakeholders was embedded in the competition design process. Yoon highlighted the significance of extensive community involvement. The approach adopted for the design of this museum departed from traditional practices that rely solely on the professional judgment of an “elite group”. He also stated, the design embraced a “participatory design” method that actively sought input from the client and users via an advisory committee, considering and incorporating their opinions from multiple perspectives. The process involved an extensive series of official meetings, consultations, and deliberations, exceeding 200 sessions, as presented in Table 5.1. In the pursuit of inclusive decision-making, the project team sought and reviewed over a thousand opinions. Their iterative process integrated the use of diverse digital communication software and strategies into a dynamic and collaborative design process. This approach ensured that a wide range of viewpoints were reflected and incorporated into the design. This user-centric approach ensured that the museum would deliver a fulfilling and captivating experience, aligning the design with the expectations and desires of its intended audience.

86

D. Alic et al.

Table 5.1 Analogue and digital forms of communication and design development Forms of communication

Software

When was it used/stage

Frequency

What documentation was produced

Digital

SketchUp

Preliminary design

Not very frequently

3D design development and construction details

Digital

AutoCAD

All stages

Almost Entire daily—for documentation—from 4 years—ongoing preliminary design to detailed drawings

Digital

Photoshop

All stages

Very frequently

Entire documentation—from preliminary design to detailed drawings

Digital

Rhino 3D

All stages

Very frequently

Entire documentation—from preliminary design to detailed drawings

Digital

Grasshopper Plug-Inn

All stages

Not very frequently

Specific details of construction and visualisations

Communication

Zoom

Meetings, two Very frequently times a week for duration between two years, later 1 and 2 h once a week—ongoing

Travel M. Jadric

Vienna Seoul 2023 every three Limited during months Corona, after 2022 frequently

Consultations and research

Travel Geung Ju

Seoul Vienna Two times in 2019 and 2022

Not very frequently

Consultations and research

NAS

Servers in Seoul and Vienna

Very frequently

Storage and data sharing

Frequently

–

All stages

Bilingual A part-time All stages architect Korean/ employee for German two years, now once a week and occasionally

–

Co-design and co-creation are collaborative approaches that foster the active involvement of end-users or stakeholders from diverse cultures in the design process. By incorporating a range of perspectives, local knowledge, and cultural insights, these approaches facilitate the development of design solutions that are contextually relevant and culturally appropriate.

5 Hybrid Practice: Exploring the Complexities of Cross-Cultural …

87

However, Yoon also recognised the challenges presented by the extensive and occasionally bureaucratic processes involved. Referring to the potential improvement in the process, Yoon suggested that balancing the integration of results derived from citizen participation with the specific requirements of architectural design was a challenge. To optimise the design process and minimise unnecessary deliberations and committee meetings, he suggested that in the future a more streamlined approach might be more effective. Simplifying the processes would allow the focus to remain on achieving the best possible design outcome while reducing potential bottlenecks caused by excessive bureaucracy. By implementing more efficient and streamlined procedures, the project could progress smoothly, enabling the architects to concentrate on creating an exceptional design that meets the needs of both the community and the architectural vision. Driven by their problem-solving mindset, the teams focused on developing protocols that could enable them to effectively address and overcome challenges encountered during the design development phase.

5.3.4 Cross-Cultural Collaborations Cross-cultural design collaborations are renowned for their complexity, encompassing various fields and requiring the adept use of design methods and tools. Effective communication became even more crucial in this collaboration, given the language differences and the challenges in understanding, interpreting, and translating ideas and concepts. By actively engaging stakeholders and incorporating their feedback, the project team aimed to create a museum that resonated with its audience and fostered a sense of ownership in the community. The challenge of managing this communication was addressed in several ways. Communication and language play crucial roles in cross-cultural design collaboration. Effective communication strategies, such as visual communication, storytelling, and visual representations, can help bridge language barriers and facilitate mutual understanding. Jadric emphasised the importance of respectful communication: We recognised that this open and respectful communication was paramount, particularly when navigating cultural differences …Throughout the project, we were committed to thoroughly addressing any areas of misunderstanding or difficulty without glossing over them…prioritising our common architectural agenda while being mindful of the need to be respectful and recognise everyone’s contribution, allowed us to foster an environment of effective collaboration and bridge the gaps that arose due to our diverse cultural backgrounds.

“In essence”, suggested Yoon, “the language of architecture can be considered a common language in itself.” This is because architectural practices share universal similarities, including sketching, modelling, adherence to architectural laws, utilisation of technology, and budget management. These shared aspects serve as a foundation for mutual understanding among architects from diverse backgrounds. Yoon further stated, “Our ability to deliver a satisfactory outcome within the designated timeframe was largely attributed to our commitment to actively listen to each other’s opinions throughout the process while maintaining mutual respect”.

88

D. Alic et al.

Referring to their frequent online discussions via Zoom meetings, Jadric explained, We often found ourselves engaged in discussions that extended for 3-4 hours. We encountered situations where approaches that seemed feasible in the Austrian context were not readily applicable in Korea. However, we persevered and actively sought answers by asking questions and engaging in thorough explanations.

The architects explained that in some instances, they invited the participation of the Korea Cooperative Rescue Office to their Zoom meetings to collectively address topics such as structural systems. “We recognised the value of their expertise and collaboration in finding suitable solutions for the project.”, Jadric concluded. The same collaborative approach was taken when discussing the details such as the concrete louvres that formed the outer shell of the building. The Vienna team constructed 1:1 physical models and tested specific details with real materials to anticipate the final outcome. Such a meticulous approach and attention to detail allowed the teams to ensure accuracy and make informed decisions throughout the project.

5.3.5 Zoom and Video Communication Virtual meeting platforms, such as Zoom, offered invaluable opportunities for online design collaboration. Through video calls, screen sharing, and real-time discussions, the platform supported essential activities of the teams, such as brainstorming sessions, design reviews, and presentations. Virtual meetings bridged the geographical barriers between Austria and Korea, allowing design teams to collaborate effectively regardless of their physical locations. Zoom conferences became a primary mode of communication, with meetings starting early in the morning to accommodate the time difference between Vienna and Seoul. As the project progressed, the length of meetings increased with sessions lasting two to three hours. For the first two years, the meetings were conducted twice a week, and later when the project moved to the construction stage the meeting frequency was reduced to once a week. This regular communication served as a constant connection that both Jadric and Yoon agreed, grew stronger over time. In early 2020, the onset of the COVID-19 pandemic started to disrupt travel and prompted the restructuring of communication processes. Additional online meetings and frequent progress reports became essential to maintain coordination and ensure the smooth progression of the project. Though the increased engagements significantly contributed to the smooth running and progression of the project they also increased the demands for thorough and complete documentation and by extension the labour costs. The teams’ reliance on the intensive conferences that encompassed comprehensive discussions, addressing all relevant issues, and even involving live sketching during the meetings increased. “Despite the challenges we faced, both teams persisted

5 Hybrid Practice: Exploring the Complexities of Cross-Cultural …

89

in asking questions and actively listening to each other’s responses, all while maintaining a deep respect for one another”, affirmed Jadric. Jadric Architektur and 1990uao recognised the crucial role of effective communication in their continuing collaborative efforts. They understood that establishing a robust communication protocol was essential, but they went a step further by structuring their teams to align with the project requirements. To overcome language barriers and promote a productive and cohesive working relationship, they not only employed a bilingual architect to support the core team in Vienna but also assigned additional members exclusively dedicated to this project within the Seoul team. Reflecting on the careful and considered bi-lingual team building Jadric stated that both teams were positively surprised, “that Zoom was working so very well and allowed us to include our colleague, who worked from our Vienna office to translate.” With the translator on board, it was possible to engage in more sophisticated discussions and focus on the specific regulatory context and details. Furthermore, the productive online discussions reduced the need for the staff from the Seoul office to travel to Vienna. As part of fostering robust communication between Vienna and Seoul, an assigned team from Vienna was sent to assist the home office in Seoul throughout the project’s duration. With the travel restrictions imposed Jadric visited Seoul in April 2020. During the height of the COVID-19 pandemic, travel to Seoul was restricted, making frequent visits impossible. Travel resumed in November 2021 and slowly increased in frequency in 2022. Starting in 2023, Jadric resumed regular three-monthly trips to Seoul. Though there was less need for staff from the Seoul team to visit Vienna, Yoon spent ten days in Vienna and Europe in 2022 extending their collaboration.

5.3.6 Online Design Collaboration Tools The online competition required the participants to submit two sets of two panels. The required drawings included a bird’s-eye view and free-expression master plan and architectural overview; floor plan, elevation, and section plan and two panels of free-expression drawings. The winning design concept proposed by Jadric Architektur and 1990uao is evocative of free flow and movement. The monolithic form of the building appears to emerge from the surroundings, serving as a prominent landmark along the planned culture mile. The building seamlessly integrates with the square, creating an inviting and accessible entry into the building that ultimately blends with the public space. The proposed design submission emphasised the symbiotic relationship between photography and architecture, exploring the interplay of light and form. The architects described the dynamic movement of the building as a metaphorical snapshot, capturing the shared characteristics of architecture and photography (as seen in Fig. 5.2). Online design tools have revolutionised the way designers collaborate, offering real-time collaboration capabilities. These tools provide shared workspaces where

90

D. Alic et al.

multiple team members could simultaneously engage on the same task, along with communication features such as chat or comments which allowed effective discussion and feedback. SketchUp, a 3D modelling software, was primarily used in the initial stages by the Jadric Architektur office for creating the final competition submission drawings. SketchUp provided the necessary tools and features to visualise and present exhibition layouts effectively. While AutoCAD was the main software utilised by the Jadric Architektur office for overall design and documentation, SketchUp specifically catered to competition-related drawings and design development. It allowed for the creation of detailed and accurate representations of the interior exhibition spaces, ensuring that the vision and requirements were effectively communicated. Photoshop played a crucial role in aligning the proposed designs with the prescribed standards. “Every single presentation without exception,” stated Jadric, was created using Photoshop. The versatility and capabilities of Photoshop made it the go-to software for crafting and refining the proposed designs. It allowed the architects to manipulate images, adjust colours, create intricate graphical elements, and ensure that every detail complied with the required standards. Photoshop became an essential tool in presenting and communicating design ideas to diverse audiences, including consultants and government bodies. “Whether it was producing visual representations of architectural plans, showcasing 3D renderings, or creating realistic mock-ups, Photoshop provided the necessary features and flexibility to bring our ideas to life,” stated Jadric. Its extensive array of tools and functionalities allowed the teams to accurately depict construction elements, demonstrate material choices, and showcase the desired aesthetics. Even in the later stages, Jadric stated, “Photoshop remained an indispensable tool for architectural and construction documentation, enabling us to create visually appealing and accurate presentations that adhered to the prescribed standards”. Its continued usage underscored its importance in the industry and its ability to facilitate effective communication and understanding of design concepts. Photoshop was utilised for digital production, further enhancing the design documentation. Although SketchUp was employed during the initial phase, it was not the primary software used for the design development. Instead, AutoCAD served as the primary software for 2D drawings. AutoCAD played a crucial role in all stages of the project. The post-competition phase required the architects to significantly change their drawings to accommodate the increased space requirements and respond to the community feedback as well as make other design adjustments. SketchUp was extensively used for the project revisions, used to showcase the volumes and spatial arrangements while Rhino 3D with the Grasshopper Plug-In further extended the capacity of the 3D drawings. These software tools continued to be used throughout the design and construction process. Currently (2023) as the building is under construction, additional drawings from the architects are constantly required. Throughout the entire documentation process, from preliminary to construction and final site drawings, AutoCAD was the dominant software. However, adjustments were necessary to meet Korean standards. To comply with local regulations 1990uao office commonly redrew and created new and supplementary drawings to align the

5 Hybrid Practice: Exploring the Complexities of Cross-Cultural …

91

project with the rigorous Korean standards and regulations. Discussions between the two offices continued in Zoom as meetings were dedicated to determining the best approach for dealing with revised project requirements. Some elements were used as they were originally designed, while others required further modification or were completely dismissed and redesigned. Throughout the design process, various tools were utilised, with Rhino 3D playing a significant role in supporting the production of 3D designs. Unlike traditional 2D CAD software, Rhino 3D with Grasshopper Plug-In treats all information as vector values, providing the advantage of easy separation and conversion into 2D drawings. Yoon highlighted the benefits of using this software in documenting the complex structural system incorporated in the design. He stated that Grasshopper’s capacity to transform and separate 3D information into 2D vectors allowed the architects not only to check the accuracy of the drawings but also to calculate quantities and costs, thereby facilitating budget management. Further, the utilisation of Rhino 3D facilitated the creation of numerous shell modules, and its integration with the Grasshopper Plug-In provided comprehensive support throughout the design process, from initial concepts to precise construction details. This combination of software tools enabled the architects to visualise and analyse the design in 3D, while also ensuring accuracy, budget control, and efficient documentation. The architects’ reliance on Rhino 3D and Grasshopper Plug-In showcased the importance of advanced software tools in enhancing the design process, enabling efficient management of complex structures, accurate documentation, and informed decision-making. Adapting the construction drawings and detailing to meet Korean standards was also necessary. Korea has comprehensive requirements for the design of structural elements, and all architectural details were required to adhere to these standards. Complying with these regulations ensured a smoother construction process as if a structural or construction detail failed to meet the standards, it had to be additionally reviewed and approved by the relevant authorities, resulting in significant construction delays. For instance, numerous layers of walls and roof plates needed to be drawn in a specific manner according to the prescribed list of construction elements. Consequently, the process involved a degree of difficulty, where initial drawings had to be adjusted based on Korean standards and regulations. The project required a significant number of 3D drawings due to the complexity of the building’s design. It was essential to provide detailed explanations and visualisations in three-dimensional form. Merely relying on 2D drawings of the facade and sections was insufficient to convey the intricate nature of the structure. Even section-by-section 2D extrusion drawings were inadequate, necessitating the use of 3D. To illustrate this significance of 3D documentation Jadric suggested that the concrete firm involved in the construction exclusively focused on the 3D drawings of the building’s upper levels and first-floor plan. A section of the facade alone consisted of over 303,800 distinct elements that required individual drawings. These elements were then extruded in 3D to accurately depict the concrete cuts needed for construction purposes.

92

D. Alic et al.

Fig. 5.3 Final competition presentation developed using multiple software tools

The final competition presentation was developed using multiple software tools, to create a vivid and accurate depiction of the design with all its intricacy and in its realworld context. In Fig. 5.3, the seamless integration of these design software is readily apparent, showcasing the meticulous precision with which they were combined to produce a comprehensive and detailed design presentation. The digital production was supported by numerous physical models of the proposal as a whole and its various parts. Physical models provided a tangible representation of the design, allowing the architects, engineers, and construction professionals to examine and correlate the drawings. These models also assisted in communicating structural and construction details. Despite the advanced use of digital technologies, physical models were considered indispensable tools in the production and the successful realisation of the proposed architectural vision. In addition to the previously mentioned software tools, the architects heavily relied on the KakaoTalk app for their communication needs. According to Jadric, there were situations where it was necessary to quickly ask a question, share a picture, or provide an example, and KakaoTalk proved to be a highly suitable and easily accessible option. Developed by a South Korean internet company, KakaoTalk is a popular mobile messaging application widely used in South Korea and other countries. Throughout the project, the communication flow remained consistent. In cases where more in-depth discussions were required, the team would switch to Zoom. However, for matters that could be addressed through regular check-ins, the teams maintained a weekly or bi-weekly meeting frequency. This approach ensured

5 Hybrid Practice: Exploring the Complexities of Cross-Cultural …

93

effective communication while utilising the appropriate tools based on the nature and urgency of the communication needs. By leveraging online tools and digital production techniques, the two offices were empowered to create collaboratively, addressing cultural nuances, and improving design outcomes.

5.3.7 Data Management Leveraging NAS Servers and Extensive Documentation Efficient data management and reliable storage services are crucial for the success of design collaborations. The design teams of Jadric Architektur and 1990uao understood this and implemented strategies to ensure reliable data storage, seamless collaboration, and a comprehensive overview of project progress. The utilisation of Network-attached storage (NAS) servers located in Seoul and Vienna played a pivotal role in the data management process. These NAS servers provided ample storage space and hosted a vast collection of drawings and image-based documentation, enabling easy access and retrieval of project data. By leveraging NAS servers and maintaining meticulous documentation, the design team ensured streamlined data management, facilitating collaborative work and efficient data sharing between the teams regardless of the different time zones. The comprehensive project statistics exemplify the team’s commitment to documenting project progress in a thorough and detailed manner. This dedication to documentation promoted transparency and facilitated informed decision-making throughout the entire project lifecycle.

5.4 Discussion 5.4.1 Seamlessly Integrating Analogue and Digital The successful design process and collaboration between Jadric Architektur and 1990uao serve as a testament to the importance of efficient communication and collaboration in the development of design proposals. To optimise and diversify their workflow processes, the teams utilised a range of software tools and virtual meeting platforms, leading to enhanced team productivity, and efficient coordination. By employing widely used software programs such as AutoCAD, SketchUp, Photoshop, and Rhino 3D, the design teams were able to seamlessly communicate and exchange information. The compatibility of these commonly used software tools ensured smooth coordination and effective collaboration throughout the design process. Although Building Information Modelling (BIM) was not utilised in this particular project, the team recognised the value of their chosen software stack in facilitating

94

D. Alic et al.

ongoing communication. Jadric mentioned that in his opinion using BIM would have required more extensive collaboration, and possibly additional team members or an expanded documentation team. While the lack of familiarity with BIM software was the main reason for not using the software in this project, Jadric acknowledged the potential embedded in the platform.

5.4.2 Navigating Multicultural Challenges in Contemporary Architecture: Insights from Collaborative Dialogue The collaborative process used by Jadric Architektur and 1990uao for the Seoul Photographic Art Museum project exemplifies a design process that harnessed the diversity of communication and collaboration tools. The architects embraced a fluid and dynamic approach, where iterative refinement, creative problem-solving, and user-centric design principles converged to deliver a successful architectural endeavour. The field of contemporary architecture operates within a multicultural and global context, requiring broad contextual awareness. From specific construction requirements to cultural practices, holidays, and work disruptions, the dialogue between two design practices highlighted the exciting yet challenging processes faced in this dynamic landscape. The design teams from Vienna and Korea demonstrated mutual respect for each other’s inputs, recognising the complexity of making such a project successful. Geographical distance posed a minimal obstacle for the teams, as the architects drew from their extensive professional and academic experience and their prior collaborations. These experiences and exposures fostered an understanding and adaptability that transcended physical boundaries, facilitating smooth collaboration in the project. In terms of project statistics, the total time spent on the design development spanned 542 days, excluding the design period before the contract. A total of 222 formal meetings, reports, deliberations, and certifications took place, excluding informal meetings. The team generated approximately 20,000 pages of delivery documents specifically for the detailed design. It is worth noting that this figure excluded basic, interim delivery, meeting and report materials, as well as documentation from the six associated consultancies other than architecture. When stacked as A4 paper, the 20,000 pages would reach a height of 2.20 m—the fact that emphasises the comprehensive nature of the documentation needed for the successful completion of this project. This extensive experience of both architectural practices provided a distinct advantage, that extended well beyond technological proficiency. Making the process a “reflective practice” the teams were able to negotiate the cultural differences, problem-solve, and integrate communication strategies and legal aspects into the

5 Hybrid Practice: Exploring the Complexities of Cross-Cultural …

95

design process, ensuring a holistic approach to design. By embracing multiculturalism and global perspectives, the design teams navigated the challenges and opportunities presented by diverse contexts. Their collaborative dialogue sheds light on the importance of respectful engagement, financial diligence, and the ability to transcend geographical limitations in delivering successful architectural projects. The two practices have continued their collaboration beyond the Seoul Photographic Art Museum and are currently working on a project in Deulan Village within the cultural ground around Suseong Pond. The project involves the conversion of one of the apartment buildings surrounding Deulan Children’s Park into an “Artist in Residence” centre for local artists. Daegu is a major high-tech industrial city in Southeast Korea with a population of 2 million people. This project runs parallel to the museum project and through this project, the teams continue to strengthen their collaboration and improve the communication between the two teams. Highlighting the reciprocal nature of the design process Jadric states, We have to accept, of course, different opinions and limits of such a kind of communication and cooperation because you have to acknowledge and accept differences. You have to accept that not everything is going to be possible to be implemented from one country to another.

5.5 Conclusion The collaboration between Jadric Architektur and 1990uao for the Seoul Photographic Art Museum project exemplifies the intricacies of modern architectural design processes. The chapter highlights the usefulness of narrative inquiry in exploring cross-cultural design processes and communication within an international team. Through the analysis of the in-depth interviews and subsequent narrative construction, the chapter captures the unique perspectives of the architects involved and their practices. This exploration of the project’s progression uncovers the interdependencies between the iterative design development process and the underlying computer procedures that support it. By acknowledging the influences of social, cultural, historical, and interpersonal factors, the discussion provides a holistic understanding of the intricate interplay between personal experiences and broader design processes. The iterative nature of narrative inquiry further enhances the depth and quality of insights gained, contributing to a refined understanding of cross-cultural design dynamics. Ultimately, narrative inquiry not only contributes to theoretical development but also aids in understanding complex human experiences in fields where diverse perspectives and dynamic processes are crucial. The examination presented in this chapter enriches the discourse around the “reflective practitioner” model by incorporating elements of hybrid, multi-practice teamwork, and multi-lingual communication. Through the lens of narrative analysis methodology, the architects’ personal and collaborative journeys yield insights into their distinct viewpoints, underscoring the significance of reflection-in-action and reflection-on-action. This chapter examines the use of technology from the designers’ point of view, shedding light on the intricate interdependencies between these two

96

D. Alic et al.

entities. It underscores the crucial point that successful cross-cultural design development hinges upon a combination of technological proficiency and the adoption of an adaptable and receptive framework known as “reflective practice.” This collaborative effort and its in-depth analysis serve to illuminate the ever-evolving landscape of architectural teamwork, communication strategies, and the indispensable role that technology plays in shaping contemporary design processes. Acknowledgements This chapter builds upon the presentations by Assoc. Prof. Mladen Jadric and Prin. Arch. Geun Ju Yoon, facilitated by Assoc. Prof. Dijana Alic at the ‘Multiculturism and Multimodality in Architecture’, Design and Culture session (Chair: Dijana Alic and Mladen Jadric) at 1st AKAN Symposium, 25 August 2022, Hanyang Univ., Seoul, Korea and UNSW, Sydney, Australia. Image credit and copyright: Jadric Architektur and 1990uao. 3D Visualisations: Claudio Anderwald Computergraphics.

References Abdel H (2023) Daegu Anchor Facility/1990uao + Jadric Architektur. ArchDaily. https://www.arc hdaily.com/999442/pocheon-house-1990uao?ad_medium=office_landing&ad_name=article. Accessed 1 June 2023 Benkö L (2023) Symbiosis between photography and architecture, UBM magazine, Visualizations: Claudio Anderwald Computergraphics. https://www.ubm-development.com/magazin/ seoul-photographic-art-museum/. Accessed 4 April 2023 Clandinin DJ, Michael Connelly F (2000) Narrative inquiry: experience and story in qualitative research, 1st edn. Jossey-Bass Clandinin DJ (2007) Handbook of narrative inquiry: mapping a methodology. SAGE Publications, Inc. Harrouk C (2019) Jadric Architektur + 1990uao selected to create the Seoul photographic art museum. https://www.archdaily.com/930480/jadric-architektur-plus-1990uao-selected-tocreate-the-seoul-photographic-art-museum. Accessed 4 April 2023 jadric architektur + 1990 uao win competition to design Seoul photographic art museum. designboom. https://www.designboom.com/architecture/jadric-architektur-1990-uaoseoul-photographic-art-museum-11-26-2019/. Accessed 1 June 2023 Riessman CK (1993) Narrative analysis. Sage Publications Schön D (1983) The reflective practitioner: how professionals think in action. Basic Books Seoul Photographic Art Museum, Design Competition Guidelines. August 2019. Seoul Metropolitan Government (Urban Space Improvement Bureau). I Seoul U

Part II

Technology

Chapter 6

Multimodality and Architectural Technology Michael J. Ostwald

Abstract Architectural technology enables the creation, generation and sharing of innovative design solutions that have the potential to transform society. While most descriptions of architectural technology focus on its capacity for documentation, simulation, and transmission of design information, it would be equally valid to view it as providing support for diverse modes of communication, interaction, and engagement. As such, technology and multimodality have a natural connection in architecture. For this reason, Part II of this book introduces multimodality in architecture, focusing on technical applications and innovations. This chapter commences by defining key terms and concepts in Part II before presenting an overview of the history of architectural technology, drawing out its role as either an enabler or system supporting multimodal communication. Four contributions to research in multimodal architectural technology are then introduced, from advanced communication and cognition systems to applications of artificial intelligence (AI) and collective intelligence (CI). Finally, a modality–efficacy model is introduced to illustrate connections between the contributions in Part II. This chapter contributes to the exploration of technology, communication, and engagement in architecture. Keywords Architectural technology · Multimodal technology · Immersive virtual environments · Artificial intelligence · Collective intelligence

6.1 Introduction This chapter is about multimodal communication and interaction in architecture, focusing on technological developments and applications. It provides a background to critical concepts and introduces the primary themes that connect the four main research contributions in Part II of this book. M. J. Ostwald (B) School of Built Environment, Faculty of Arts, Design and Architecture, The University of New South Wales, Sydney, NSW 2052, Australia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 J. H. Lee et al. (eds.), Multimodality in Architecture, https://doi.org/10.1007/978-3-031-49511-3_6

99

100

M. J. Ostwald

The term “multimodal” refers to the simultaneous presence of more than one type of active engagement in a process or situation. Its use is often linked to communication or interaction, where different modes serve specific needs and collectively provide an enhanced user experience or increased levels of understanding. Some examples of professional modes used in architectural practice are visual (drawings or other depictions), spatial (physical models or immersive simulations), linguistic (spoken word) and gestural (movement of hands during a presentation or discussion). As a basic example of the value of multimodality in architecture, a technical drawing could be considered a single mode of visual communication, which will only be accurately read and understood by a specially trained person. However, when this drawing is explained in a meeting and illustrated with a simple physical model, more people will be able to understand its content. This is because the barriers to communication implicit in the use of a single mode of communication (the drawing) can be at least partially overcome by adding three further modes: verbal and gestural in the meeting and spatial in the model. Thus, multimodality is core to transmitting architectural information, and supporting collaboration and innovation. “Technology” refers to items (devices, tools, hardware, software) that support the application of knowledge or processes. In architectural practice, technology is used for a wide range of purposes, many associated with modes of communication and interaction. For example, architects use technology to document a design in a digital model, which can be dynamically shared with different team members, allowing them to contribute their expertise. The precise platforms used for this purpose will often be variations of CAD (Computer Aided Design) or BIM (Building Information Modelling) software operating on a combination of local hardware and cloud-distributed systems. Architects also use technology like this to assist virtual team meetings, share visualisations, and transmit data. As such, multimodality in architectural technology encompasses digital functions that support design communication, documentation, and simulation. Indeed, most modes of engagement and communication used in contemporary architectural practice are technologically enabled. In this context, where multimodality and technology are closely linked, this chapter presents a background to the two, commencing with an overview of their development in architecture as either a system or enabler supporting practice. Thereafter, the four main research contributions in Part II are introduced. The first is concerned with increasing the effectiveness of design teams working in immersive virtual environments (IVEs). It examines barriers to technologically-enabled modes of communication and how they can be overcome. This first contribution is centred on a case study of an IVE for operatic design, where a diverse team use rapid visualisation and environmental scripting to overcome barriers to effective collaboration. The second contribution examines a novel use of technology to create replicas of valuable artefacts, with the intention of using these to expand the communicative potential of the originals. This concept is developed through a research project exploring the use of replicas in museums to provide new modes of communication for people with levels of visual impairment. The third research contribution proposes ways to improve the capacity of Artificial Intelligence (AI) image-generating software (e.g.,

6 Multimodality and Architectural Technology

101

Midjourney, DALL-E 2, and Stable Diffusion) to create useful reference images for design communication. Reference images are an essential mode of design communication between architects and non-experts (often clients or the public). The final contribution introduced in this chapter, and presented in Part II of this book, proposes a different type of collaboration between users and technology. Using genetic algorithms (GA), this research demonstrates that it is possible to treat a team of people and technical systems as a type of collective intelligence (CI). The chapter commences with an overview of the use of technology for the representation, documentation, and translation of designs into reality. It introduces the four research contributions, emphasising their connections to multimodality and technology in architecture. The relationships between the four are then discussed using a modality–efficacy model for comparative purposes.

6.2 Multimodality in Architectural Technology Some of the earliest examples of the use of technology in architectural design can be traced to Ancient Egypt and Greece, where styli, “straight-edges,” and wax tablets were used (Kostof 1977). These devices enabled the inscribing of information (with the stylus) in an accurate way (using the straight edge) on a surface that both retained the information and was transportable (the wax tablet). Collectively, these tools supported the recording and representation of information about a design. However, additional modes of communication were required to translate such representations into reality, starting with systems of proportions and measurement. These systems explained how the information in the drawing was intended to be scaled or converted into a building of the correct size. Initially, proportional systems and measures were extrapolated from the human body (cubit, foot, or dactyl), and later they were converted into reproducible systems. For example, in ancient Rome, standard measurements were inscribed on ebony rods so that they could be applied consistently across the empire (Kostof 1977). The use of enablers and systems of this type to support the design process has been largely unchanged, conceptually at least, throughout history. The stylus and tablet were replaced by pencil and paper, respectively, and later, technical pens and tracing film became the dominant tools, but their use and purpose have endured (Ostwald 2012). Furthermore, when the first CAD systems were introduced in the last few decades of the twentieth century, their functions replicated historic analogue technology, replacing, for example, the graphite line drawn by a pencil on paper with a digital line constructed using coordinates on screen (Yu et al. 2021). The transition from analogue to digital technology in architecture was generally predicated on increased efficiency, either in terms of time-saved or costs-reduced. When BIM gradually replaced CAD in the early twenty-first century, the intention of reducing cost and time was similarly cited, albeit often couched in terms of rapid visualisation, version control, improved coordination of services, waste reduction, and so on (Eastman 2008). However, more recent technology has begun changing

102

M. J. Ostwald

the architectural design and production process, blurring the distinction between enablers and systems. For example, BIM’s support for project coordination across teams and the use of parametric scripting and GA to evolve design solutions are just a few examples where the historical division between systems and enablers has been eroded (Terzidis 2006). Where once architects employed technology to support design communication, it has recently become embedded in almost every aspect of their work (Ostwald 2012; Yu et al. 2021). As Holzer (2023, p. 3) explains, today, the primary use of technology is to allow “architects to expand their design options, evaluate different scenarios on the fly, and push for novel solutions that would otherwise be very difficult (if not impossible) to achieve.” Design technology is fully integrated into all parts of contemporary architectural practice, potentially supporting heightened efficiency while achieving novel outcomes and innovative solutions (Lee et al. 2020). Furthermore, it is common for multiple modes of design communication to be used simultaneously. Some of the earliest modes of design communication were drawings and specifications (written supplements to drawings), both of which are types of professional and technical languages that require training to comprehend. Contemporary BIM models use three-dimensional (3D) data to describe the form and dimensions of a building, but components in the models may also be linked to data derived from material libraries and national construction codes (Yu et al. 2021). Semantic web technologies provide computer-to-computer communication in these models, enabling, for example, automated decision-making about energy optimisation, planning compliance or building cost. BIM can provide the basis for a wide range of simulations— from maintenance scheduling to pedestrian movement—and Digital Twins (often an enhanced BIM using sensor data and the “Internet of Things”) provide the capacity to refine design operations using big data (Lee et al. 2021). Most significantly, design is a team process using synchronous and asynchronous modes of communication, whether copresent (in the same physical space) or not. Indeed, technology’s capacity to support design collaboration on a global scale has been the catalyst for rapid growth in research about multimodal technology in architecture. The following section introduces four new contributions to multimodality in architectural technology. These use advanced computational methods—AI, Machine learning (ML), and GA—along with 3D scanning and printing hardware and fully immersive virtual environments for collaborative design.

6.2.1 Multimodal Collaborative Design in an Immersive Virtual Environment Effective design team interactions in IVEs are the focus of Chap. 7. In “Multimodal Collaborative Design in an Immersive Virtual Environment: Opera Staging

6 Multimodality and Architectural Technology

103

and Performance Design using iModel”, Ostwald, Thurow, Del Favero and ScottMitchell address a two-part aim: to (i) “identify and examine barriers to effective collaborative design in IVEs” and (ii) “to demonstrate how these barriers may be overcome”. For the first aim, Ostwald et al. use a structured literature review to examine barriers identified in past research to effective interaction in IVEs. Through this review, they develop a classification system with two primary categories and eight sub-categories. The first category is made up of five technical barriers: (i) lack of immersion, especially visual immersion, (ii) limited capacity to represent realworld objects or spaces, (iii) lack of instantaneous feedback and limited access time, (iv) the cost of hardware, and for wearable devices, its scale or weight, and finally (v) problems arising from software interoperability. The second category comprises three social barriers: (vi) the complexity of representational modes used for visual communication, (vii) the complexity of language and terminology used for verbal communication, and (viii) levels of technical literacy. In a sense, these eight are all technical barriers. The first five are associated with the capacity of hardware and software to support modes of design interaction, the next two are about the technical languages experts use to communicate, and the final one concerns each person’s innate confidence in using technology. Ostwald et al. note that the negative impacts of the five hardware and software barriers have gradually reduced over time, which is why the conceptual focus of Chap. 7 is on the first two social barriers, visual and verbal communication. These two are especially important when an expert team from diverse professional and technical backgrounds work on a project together. The second aim of Ostwald et al.’s. research is to demonstrate how several of these barriers to effective design interaction in IVEs may be overcome. To address this knowledge gap, Ostwald et al. present a descriptive case study of a technologically enabled, multimodal immersive design platform for a non-cognate team of expert users to engage in operatic design. Called iModel, this IVE supports performance and staging design, incorporating the location and movement of stage machinery, sets, props, lighting, and people during a performance. Ostwald et al. describe the technical implementation of iModel in an AVIE (360-degree interactive cinematic visualisation system) and its application in the design of two major operatic productions (Thurow et al. 2021). At a time when multinational design practices increasingly rely on technology to support their creative processes and production schedules, Ostwald et al. demonstrate two completed projects developed in an IVE. This is a crucial difference from most past literature on IVEs, which has tended to describe the results of experiments or isolated events (for example, a client meeting or public consultation workshop) to develop data about the effectiveness of IVEs. The two projects described in the iModel case study are both complex, realised performances which many thousands of people have seen.

104

M. J. Ostwald

6.2.2 Multimodality and Touch Access for Museums Multimodal design encompasses multiple, parallel or complementary methods of representation or communication. Through the careful deployment of additional modes, various barriers to effective communication may be overcome. This idea is core to Chap. 8, which examines technology’s capacity to expand the range of interaction opportunities available for people with reduced visual acuity levels. In “The Museum of Touch: Tangible Models for Blind and Low Vision Audiences in Museums”, Reinhardt, Holloway, Thogersen, Guerry, Corvalan Diaz, Havellas, and Poronnik examine the ways 3D printed models might provide an alternative mode of communication for museum visitors. Museums are locations for displaying, securing, storing, and restoring valuable artefacts. These artefacts, which range from historic objects to artworks, are typically presented in a way that blocks physical contact. Thus, a visitor may be positioned behind a barrier, or an object may be presented in a glass case so that the primary interaction is visual. Signage and recorded commentary often provide secondary communication modes in museums. While this widespread practice appears to balance security with accessibility, for people with levels of visual impairment, it is largely ineffective for communicating a deep understanding of the artefacts (Candlin 2003). In this context, Reinhardt et al. examine the use and production of “handling surrogates”, objects with some of the physical characteristics of the original, which can be touched and felt to provide an additional mode of communication. Reinhardt et al. explore three primary themes in their research. First is 3D scanning and printing to produce a simulacra of historic, fragile, or valuable objects. The scanning stage captures data describing an object’s form (shape), and the printing stage creates a replica to serve as a surrogate. There are, however, limitations to the usefulness of this process. For example, Reinhardt et al. note that while heightened levels of realism are desirable for an effective surrogate, form and texture are potentially effective without replicating object weight or temperature. Furthermore, “complete” or “full” replicas are potentially more effective surrogates than partial or cut-away sections from objects. The second theme in Reinhardt et al.’s research is about the use of additional modalities, such as sound and Braille labelling, to increase communication and heighten engagement and understanding for visitors with diverse perceptual needs. Reinhardt et al. examine the capacities of these two modes for improving communication, emphasising that audio “is one of the simplest formats to implement and it has multiple uses”, including explaining how the handling surrogate differs from the original. The last theme in Chap. 8 is about the potential for sharing touch and sound models through open platforms so that the additional levels of experience and communication afforded by these models can be made more accessible. While there is a technical side to this theme associated with data formats, interoperability and the limits of printing hardware and software, the more critical challenge is developing appropriate guidelines. While research about handling surrogates has been published for several

6 Multimodality and Architectural Technology

105

decades, it has taken time for practical advice to be identified which describes “best practice” for a range of museum surrogates to serve diverse visitor needs (Eardly et al. 2016).

6.2.3 Architectural Visualisation Using Image Gen AI Historians may look back on 2023 as the year when AI became widely accessible to the public for the first time. From large language models (LLMs) like ChatGPT to data analytics platforms like Tableau and image generation tools like Midjourney and Stable Diffusion, in 2023, AI-enhanced systems became available for a growing range of applications (Enjellina et al. 2023). The use of AI-supported image generation in architecture is the focus of Chap. 9. In “How to Enhance Architectural Visualisation Using Image Gen AI”, Lee, Jeong, Kim, Choi, Jo, Chae and Yoo explore the optimisation of AI generative techniques for images of buildings and spaces. Architects commonly use such pictures as “reference images” or on “mood boards” in the early stages of the design process, as they are ideal for communicating with clients and the public. These images are examples of “non-technical communication” because they do not rely on special knowledge embedded in professional visual and verbal languages. Instead, reference images present typical environmental characteristics (colour, lighting, texture, furnishing) in an indicative or imprecise way. They communicate general feelings or ideas rather than specific, measurable properties (Phare et al. 2018). Despite their use for this purpose, designers face significant challenges in creating useful images using AI generative tools. This is the catalyst for Chap. 9. In Chap. 9, Lee et al. argue that “generating a single image involves a complex process and requires significant time, economic, and human resources.” In response, they propose a framework for supporting the production of appropriate and valuable images. Their research has two stages. First, they undertake a detailed testing process for a default generative AI model, and second, they refine this model. For the first stage, Lee et al. use a “text-to-image” (rather than “image-to-image”) approach, wherein descriptive prompts generate target images, which can then be refined to guide more appropriate image choices. In the second stage, three steps are used to refine the model. The first is developing and pre-processing data to provide a practical search function. The second requires optimising parameters (such as “learning rate” and “batch size”) to improve the results. The final step, shared with a range of ML applications, is training, in this case using a Low-rank Adaptation (LoRA) method, to refine the outcome. In their research, Lee et al. demonstrate the use of their framework to generate four types of architectural images: (i) interiors with defined styles, (ii) exterior facades in particular regions, (iii) images of bathrooms, and (iv) images of architect-designed houses. They present the results of their work in a series of generative image tables, commencing with style descriptions (for example, “zen”, “retro”, and “industrial”) and then a serial refinement of the interior views produced in response to this text.

106

M. J. Ostwald

Then, examples of text-to-image AI generation of bathroom and façade design are presented. Finally, Lee et al. demonstrate the results of addition (a + b), subtraction (a − b) and multiplication as generative constraints, producing, for example, design images merging the styles of Antoni Gaudi and Frank Lloyd Wright.

6.2.4 Collective Intelligence for Building Energy Efficiency Just as Ostwald et al. in Chap. 7 identify the need to resolve technical and social issues to support effective design in IVEs, in Chap. 10, Xiao, Ding and Prasad identify the need to combine social and technological factors to resolve building operational challenges. In “Development of Collective Intelligence for Building Energy Efficiency”, Xiao et al. present a model where building energy systems and occupants are treated as intelligent agents in an extensive self-organising system. Previous attempts to optimise building energy systems have focused on either technical systems or changing social behaviours. The innovation in Xiao et al.’s research is to treat the two, people and systems, as a type of CI. Using a collaborative CI model supported by GA, Xiao et al. have two research aims. First, to explore how energy systems and occupant behaviours can achieve an optimal energy outcome. Second, to determine how CI models of this type can enable appropriate collaboration (between users and systems) to improve building energy use. Xiao et al. demonstrate their model using a case study of a 25-storey apartment building in Sydney, Australia. One significant aspect of Xiao et al.’s CI model is its use of GA. Their model codes critical occupant behaviours and system capacities as genotypes. The five social genotypes considered are (i) space usage, (ii) temperature setting, (ii) clothes washing, (iii) dishwashing, (iv) showering, and (v) window operation. The parallel technical genotypes are “smart” (vi) air-conditioning and (vii) lighting. Using these eight genotypes as variables and an agent unified modelling language (UML) class system, optimal corporative conditions from each successive generation (through 50 generations with an initial population size of 10) are examined. The model shows how aggregate decision-making and responses from occupants and technical systems can produce an effective outcome.

6.3 Discussion Past research typically suggests that the inclusion of additional modes will increase the efficacy of any interaction, leading to a higher level of understanding and engagement. This could be expressed as a general model (Fig. 6.1), where each additional mode (1, + 2, + 3, + 4, …) increases the effectiveness of the outcome (a → b → c → d …). Thus, whereas one mode (1) will result in an efficacy of a, two (1 + 2) will improve this to b, and so on. Such a modality–efficacy relationship is not, however,

6 Multimodality and Architectural Technology

107

infinitely scalable. Using more modes to communicate will not necessarily increase understanding. In practice, information overload would limit the relationship between modality and efficacy. Furthermore, there are technical (T ) and social (S) barriers to the effectiveness of each additional mode, which may grow as more modes are included. As an example of how additional modes of interaction might shape the efficacy of architectural communication, consider the case of an architectural drawing (Fig. 6.2). Architectural drawings (1) are a type of technical communication and documentation that can communicate a level of information (a) if the user has appropriate training or experience. Because such drawings have their limits, even for professionals, they are typically paired with a written specification (2), collectively improving their efficacy (b). However, for a person with less experience with these modes, communication might be enhanced by the addition of an opportunity to discuss the content of the drawing in person (3) and the viewing of a scaled prototype of the design (4), collectively increasing the efficacy of the interaction (d). Undermining this idea, the discussion with the architect could be ineffective because of the use of a complex professional language, a type of social barrier (S), and the model may be presented in virtual reality, which the user is unable to access, a type of technical barrier (T ). Thus, additional modes don’t necessarily increase efficacy. The modality–efficacy model provides a means of comparing the cases presented in Part II of this book. Taking the first case as an example, Ostwald et al.’s research about the barriers to effective design collaboration in IVEs acknowledges that the benchmark for effective communication in design teams is copresent and synchronous interaction (same place, same time). They also note that the problem of professional languages means that even this cannot be regarded as entirely effective. They further observe that there are multiple social and technical barriers to raising the efficacy of communication in an IVE. Thus, their research could be conceptualised

Fig. 6.1 The modality–efficacy model

108

M. J. Ostwald

Fig. 6.2 Example of the modality–efficacy model

as noting the limitations of drawings (1) and models (2) and testing assumptions about the effectiveness of design communication and interaction in an IVE. Their research identifies barriers to improving quality (from a to d) and that these may be overcome in more immersive and responsive collaborative environments (Fig. 6.3). In their research about multimodality and touch access for museums, Reinhardt et al. present one of the clearest examples of how the assumptions underpinning the modality–efficacy model work (Fig. 6.4). Their research commences with the observation that the traditional modes of communication used in museums prioritise vision (1), with sound (2) and text (3) as occasional supplements, leading to a lowquality outcome (a → b) for visitors with vision impairment. However, through the

Fig. 6.3 Conceptualising the barriers to effective collaboration in an IVE using the modality– efficacy model

6 Multimodality and Architectural Technology

109

Fig. 6.4 Using multiple modes to improve communication

addition of touch models (4) and Braille text (5), they show how communication and understanding will be improved (c). While this is a clear example of multimodality and technology aligned to a common goal, it is also an instance where the limits are apparent. As Reinhardt et al. note, some additional modes of interaction provided for visitors will enhance engagement and appreciation, but others may confuse them. This is why there are limits to the effectiveness of multimodality and architectural technology, encapsulated in the social and technical barriers identified previously. Lee et al.’s research about architectural visualisation using AI-generated images is founded on the realisation that the text-to-image (1) mode of input produces a relatively low-quality (Image1 ) output with limited capacity to be an effective reference image (a). To address this problem, they introduce a series of technical process modes, starting with pre-processing and optimisation (3), to create an improved outcome (b, Image2 ). They further enhance this outcome through the addition of ML (4), training the process to generate multiple, better quality (c) outputs (Images3 ), which enhance communication (Fig. 6.5). Xiao et al.’s research about CI and building optimisation starts with the assumption that neither technical systems (1) nor human users (2) will, in isolation, produce an optimal result (>b) (Fig. 6.6). However, if the two are treated as a CI (3), it is possible to identify a better outcome (c), and when successive generations are evolved from the optimal genotypes (4), a process of evolution, the CI will produce an even higher quality outcome (d). These technical modes (1, 2, 3, 4) overcome a series of barriers (T, S) to reach the highest quality outcome.

110

M. J. Ostwald

Fig. 6.5 Improving the quality of AI-generated images as communication tools for non-expert participants

Fig. 6.6 Conceptualising the impact of additional technical modes in a CI model for building optimisation

6.4 Conclusion This chapter defines two concepts which are core to Part II of this book—“multimodal” and “technology”—and introduces their role in architecture. It provides a historical overview of the transformation of multimodal technology, conceptualised as enablers and systems, over time, until today, when they are merged into large digital platforms that serve the needs of the architectural profession. The chapter introduces a new modality–efficacy model to understand both the potential and limits

6 Multimodality and Architectural Technology

111

of multimodal thinking in architecture. Using this model, four new research contributions are compared, demonstrating their capacity for increasing understanding and engagement and the barriers that must be overcome to achieve this. The four research contributions which make up the remainder of Part II employ a wide range of technologies. From IVEs to AI, ML, GA and CI, the research explores physical and cognitive advances, typically to support collaboration or communication. The research also addresses the challenges of diverse teams, diverse sensory experiences, and multiple types of human–computer interactions. Acknowledgements This research was supported by the ARC (DP230100605).

References Candlin F (2003) Blindness, art and exclusion in museums and galleries. Int J Art Design Educ 22(1):100–110 Eardley AF, Mineiro C, Neves J, Ride P (2016) Redefining access: embracing multimodality, memorability and shared experience in Museums. Curator Museum J 59(1):263–286 Eastman C (2008) BIM handbook. Wiley, Hoboken, NJ Enjellina EV, Beyan P, Rossy AGC (2023) A review of AI image generator: Influences, challenges, and future prospects for architectural field. JARINA 2(1):53–65 Holzer D (2023) Design technology in contemporary architectural practice. Routledge, London Kostof S (1977) The practice of architecture in the ancient world: Egypt and Greece. In: Kostof S (ed) The architect: chapters in the history of the profession. Oxford University Press, New York, pp 3–27 Lee JH, Ostwald MJ, Ning G (2020) Design thinking: creativity, collaboration and culture. Springer, Cham Lee JH, Ostwald MJ, Kim MJ (2021) Characterizing smart environments as interactive and collective platforms: a review of the key behaviours of responsive architecture. Sensors 21:3417 Ostwald MJ (2012) Systems and enablers: modeling the impact of contemporary computational methods and technologies on the design process. In: Gu N, Wang X (eds) Computational design methods and technologies. IGI Global, Pennsylvania, pp 1–17 Phare D, Ning G, Ostwald M (2018) Representation in design communication: meaning-making in a collective context. Front Built Environ 4(36):1–9 Terzidis K (2006) Algorithmic architecture. Elsevier, Oxford Thurow S, Del Favero D, Wallen L (2021) Inhabitable models—immersive intelligent aesthetics for scenographic design. Theatre Performance Des 7(1–2):82–95 Yu R, Ning G, Ostwald MJ (2021) Computational design: technology, cognition and environments. Taylor and Francis, CRC Press, London

Chapter 7

Multimodal Collaborative Design in an Immersive Virtual Environment: Opera Staging and Performance Design Using iModel Michael J. Ostwald , Susanne Thurow , Dennis Del Favero , and Michael Scott-Mitchell

Abstract The focus of this chapter is multimodal and collaborative design in immersive virtual environments (IVEs). The research reported in this chapter aims to identify and examine barriers to effective collaborative design in IVEs and to demonstrate how these may be overcome. The chapter uses a two-stage method to address the aim, the first is a literature review, and the second is a descriptive case study. For the first stage, the chapter presents a background to recent research about multimodality and participatory or collaborative design in IVEs. It analyses the types of interaction identified in past research, along with technical and social barriers to the effectiveness of IVEs. In the second stage, the chapter introduces a specific case that addresses many of these barriers. That case is iModel, a technologically enabled, multimodal design system that engages diverse expert users in creating an operatic design. Its collaborative IVE supports staging and performance design, including placement and dynamic coordination of stage machinery and sets within 3D modelled spaces, alongside choreography of lighting, props, and actor movement timed to the musical score. Through its implementation in major productions and use of immersive simulations of existing performance venues, the case study demonstrates that an IVE can assist expert teams in an operatic design process. This chapter contributes to the development of new knowledge about barriers to effective design team collaboration in IVEs, and potential ways of overcoming them. Keywords Multimodal design · Participatory design · Immersive virtual environment · Design visualisation M. J. Ostwald (B) School of Built Environment, Faculty of Arts, Design and Architecture, The University of New South Wales, Sydney, NSW 2052, Australia e-mail: [email protected] S. Thurow · D. Del Favero · M. Scott-Mitchell iCinema Research Centre, Faculty of Arts, Design and Architecture, The University of New South Wales, Sydney, NSW 2052, Australia © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 J. H. Lee et al. (eds.), Multimodality in Architecture, https://doi.org/10.1007/978-3-031-49511-3_7

113

114

M. J. Ostwald et al.

7.1 Introduction This chapter is about the challenge of overcoming barriers to effective multimodal, collaborative design teams working in immersive virtual environments (IVEs). This is a significant topic for contemporary research because the design ecosystem has become increasingly global in recent decades. Today, design is often a team activity involving multinational, multicultural, and multilingual collaborations to achieve complex and creative outcomes (Lee et al. 2020). Because of the global nature of this design ecosystem, there is a need to develop effective methods of virtual collaboration that acknowledge the challenges of accommodating team members with diverse skills and experience. As such, the aim of the research in this chapter is to: 1. identify and examine barriers to effective collaborative design in IVEs and 2. to demonstrate how these barriers may be overcome. This two-part aim responds to a defined knowledge gap in the literature. In a systematic review of research about design in IVEs, Yu et al. (2022) identify the need for examples of collaboration utilising rich cognitive modes of communication with multi-user capacity. The research in this chapter begins to address this lacuna, which occurs at the intersection between participatory or collaborative design, multimodal design, and IVEs. The gap is associated with two related factors: the types of people interacting in the design process and how immersive modes of interaction can overcome various barriers to facilitate the process. For the first factor, interaction, past research about design in IVEs has tended to emphasise either collaboration in cognate teams (for example, architects working with architects or engineers with engineers) or, conversely, teams combining expert and non-expert members (for example, architects and building users or stage designers and the general public) (Manzini and Rizzo 2011; Sleeswijk Visser et al. 2007). In the case of the second factor, barriers to effective multimodal design in IVEs, despite growth in research about technologically mediated collaboration (Lee et al. 2020; Yu et al. 2021), face-to-face interaction is still considered the benchmark (Benetti et al. 2023). This is because co-present (within the same space) design interactions have traditionally been viewed as superior to technologically mediated ones, such as those occurring online or in a virtual reality simulation. The rationale for this view is that co-present interaction allows for more simultaneous and intuitive modes of interaction—vocal, linguistic, acoustic, visual, gestural, and spatial—to be used (Sleeswijk Visser et al. 2007). Therefore, the knowledge gap addressed in the present chapter is associated with the lack of descriptions of the experience of using an IVE to support a particular type of design team and process. This design team is one wherein non-cognate experts (experts in different fields) work on the same project. Such teams are important in design, but they also present a particular challenge for researchers. For example, the design of a major building typically involves a large, non-cognate expert team (architects, engineers, surveyors, town planners, construction managers, etc.) working for multiple years together. However, the scale of this work tends to limit both the amount of time spent in IVEs and the capacity to implement and observe a complete design process.

7 Multimodal Collaborative Design in an Immersive Virtual Environment …

115

This chapter uses a two-stage method to address the research aims and the knowledge gap. For the first aim—“to identify and examine barriers to effective collaborative design in IVEs”—a review of recent research about design in IVEs is undertaken (Sect. 7.2). Through this literature review, two primary barrier types are identified, the technical and the social, along with eight separate sub-categories of barriers (Sect. 7.3). For the second aim—“to demonstrate how these barriers may be overcome”—a descriptive case study method is used (Sect. 7.4). However, rather than using an architectural non-cognate expert team as an example, the case described in this chapter comes from the field of “operatic design”. Operatic design is the process of determining the staging (sets, lighting, sound) and performance (movement, choreography, live music) for a production. Effective design for the performing arts requires a collaborative team process wherein people from diverse professional cultures and backgrounds—set designers, costume designers, lighting designers, sound designers, choreographers, and scenographers— must communicate effectively with one another using diverse modes (models, drawings, text, gestures, movement) (Di Benedetto 2013). Depending on the production scale and whether it must be relocated between performances, the team may also include specialist engineers, architects, and logistics managers. As such, operatic design could be considered an example of a rapid (occurring over several months) micro-design process (compared with the construction industry’s multi-year, macroprocess) with a high level of social diversity. This diversity arises partially from professional experience and enculturation, as the team will have varied technical and creative skill sets, qualifications, and training, leading to communication barriers arising from using “professional languages” and “cultural values”. The case described in this chapter is an IVE for operatic design called iModel. Developed and tested in partnership with Opera Australia (OA), the Australian Research Council (ARC), and the iCinema Research Centre at the University of New South Wales (UNSW) Sydney, iModel supports collaborative and creative visualisation and simulation in operatic design. To respond to the second aim of this research, the chapter describes the implementation of iModel and illustrates its use in two productions. The productions are the Sydney Opera House’s 2022/3 staging of Peter Shaffer’s award-winning play Amadeus and OA’s 2022/23 national touring production of Rossini’s comedic opera, The Barber of Seville (“Barber”). These productions are high-profile and critically lauded examples of production design, and both benefitted from using iModel. This chapter commences with a background to multimodal, collaborative, and participatory design before examining research about their application in IVEs. Through this process, the chapter describes the types of interaction identified in past research and crucial technical and social barriers that might limit its effectiveness. In the following section, the chapter describes the aims of iModel, along with its implementation in stage productions. The chapter concludes with a review of potential opportunities and barriers to implementing an effective IVE for integrating staging and performance design into architecture.

116

M. J. Ostwald et al.

7.2 Background This section situates the present research in the context of past scholarly investigations into multimodal, participatory, and collaborative design. Thereafter, past research about design in IVEs is reviewed to identify barriers to effective multimodal design.

7.2.1 Multimodal, Participatory and Collaborative Design Three interconnected ideas, multimodal, participatory, and collaborative design, frame the content of this chapter. The first is “multimodal design”, a concept that describes the capacity of various types of human-to-human or human-to-computer interaction to add value during the design process (Kress and Selander 2012; Schultz and Bhatt 2012). In recent years, research about multimodality in design has had a growing focus on technological applications, often using Virtual Reality (VR) or Augmented Reality (AR) to support rich and creative ways of working together (Azofeifa et al. 2022; Goli et al. 2022). Research about multimodal methods has been conducted in a wide range of fields, including architecture (Bullinger et al. 2010), visual arts (Sedivy and Johnson 2000), and theatre and performance (Lavender 2020). Regardless of whether the desired outcome of this technologically mediated process is physical (for example, a building), virtual (a game environment) or temporal (a play or performance), multimodal design has been linked to the production of innovative, effective, and efficient results (Lee et al. 2020; van Leeuwen 2014). The second concept in this chapter is “participatory design”, which involves the engagement of diverse groups of “stakeholders” or “end users” in the design process (Bannon and Ehn 2013). The term is often used to describe a method wherein expert designers selectively work with non-designers to develop an outcome that meets the latter’s needs (Luck 2018). Participatory design has a long history and, like collaborative design, the third concept which frames the content of this chapter, grew out of a range of philosophical or political shifts that occurred around issues of empowerment, agency, and democracy in society (Correia and Yusop 2008). Participatory design has been applied in many fields, including architecture (Shanthi Priya et al. 2020) and narrative and theatre performance (Brandt et al. 2013). Although there are multiple differences between participatory and collaborative design (Kvan 2000), especially in terms of intentions, from the perspective of technical and social dimensions of design teams and the modes of communication used, there are clear similarities (den Otter and Emmitt 2007; Ostergaard et al. 2005; Salter et al. 2009). Because much past research about participatory and collaborative design in IVEs has been concerned with either involving end users or studying interactions between designers, respectively, relatively little research has been conducted about the engagement of diverse creative practitioners in a single project. More specifically, what types of barriers might exist that could diminish the performance of a design team with members drawn from diverse professional backgrounds? The following

7 Multimodal Collaborative Design in an Immersive Virtual Environment …

117

section reviews the literature published about barriers to different types of design collaboration in IVEs.

7.2.2 Collaborative and Multimodal Design in Immersive Virtual Environments An IVE can be defined as a “rich multisensory computer simulation” that evokes a “feeling of being mentally immersed or present” (Saeidi et al. 2018, p. 371). IVEs potentially offer “new ways of experiencing data, new ways of exploring, understanding, interacting with, and presenting information” (Bravo and Maier 2020, p. 1215). A basic premise underpinning most research about design in IVEs is that they “offer the user an active and real-time interaction with the design”, which is enabled by heightened “communication and collaboration” (Schnabel and Kvan 2002, p. 473). As such, definitions of IVEs tend to explicitly reference their capacity to support both multimodal interaction and diverse team types. While the earliest research about IVEs and design can be traced to the 1990s, it was primarily focused on developing visualisation tools rather than a capacity to support the design process or communication in teams (Schnabel 2011). One of the earliest examples of an experimental IVE for design teams was produced by Schnabel and Kvan (2002, p. 476), who demonstrated that such an environment could “support an instantaneous, direct, scaleless and intuitive control” in the design process. Working with early technology, with its innate barriers to performance— “clumsiness of gesturing and limited field of vision” (p. 476)—they were able to demonstrate multimodal design communication in an experiment. In a more advanced application, Schnabel and Kvan (2003) examined the capacity of IVEs to support effective collaboration. They concluded that while successful communication and collaboration in a design process is possible in an IVE, substantial technical barriers limited its effectiveness. Salter et al. (2009) describe an application of IVEs for participatory urban planning to evaluate its effectiveness for mixed teams with both experts and non-experts. While they found the IVE broadly effective for this purpose, they also identified the negative impacts of two types of cognitive lag, the first caused by unfamiliarity with the technology and the second with expert communication practices (3D models, drawings, and jargon). Their findings highlight the problematic role that professional language can play in a design team. Salter et al. (2009) also acknowledge that the lack of complete immersion in their IVE reduced the number of modes of communication available. Bullinger et al. (2010, p. 373) researched the potential for IVEs to support participatory design and found that they “excel at providing scaled and spatial representations and 3D real-time interaction in scenes.” Significantly, Bullinger et al. (2010) consider the challenges faced by non-cognate team members who join the process at a later stage, which is relatively common in large design teams. They conclude,

118

M. J. Ostwald et al.

however, that transferring traditional design communication modes to the IVE constitutes a particular type of barrier to overcome. Thus, they call for new types of collective IVE experience and interaction. Schnabel (2011, p. 177) echoes this when he notes that potentially, “the unique properties of immersive virtual environments can empower designers to express, explore and convey their imagination more easily”. Merrick et al. (2011) also examine the ways various functions of IVEs can support collaboration in small teams. They define three capacities an IVE requires to be effective for collaborative or participatory design. These are (i) design tools for creating and editing objects (ii) communication support systems, and finally, a capacity to accommodate cognitive design processes. Their research focused on the capacity of multi-user worlds, with team members depicted as avatars, to support synchronous and asynchronous communication using text, chat, voice, gestural cues, and 3D interactive forms. Merrick et al. (2011, p. 178) observe that even in cognate teams, it is “difficult for designers, who often come from a non-computing background, to master” interaction in multi-user worlds (for example, Second Life and Design World). They identified several technical barriers responsible for cognitive lag as well as social barriers (“digital native” versus “non-digital native”), which limited the number of communication modes used. The research of Heydarian et al. (2014) tests the assumption that IVEs can support effective (productive) and efficient (timely) communication in a diverse design team. To do this, they examine the involvement of team members in the later stages of the design process using an IVE. They observed that non-expert participants felt “a strong sense of ‘presence’ in the IVE’ (2014, p. 729), although “participants did appear to have some navigation related trouble” (2014, p. 735). In an expansion of this research, Heydarian et al. (2015, p. 116) tested the capacity of to create or enhance a sense of presence in such a way that it would allow team members to effectively assess alternative design options. The end users reported that the IVE provided a “moderately natural” simulacra of the experience of a real space. The researchers concluded that the IVE can be effective for supporting decision-making. While the definition of “collaboration” in Heydarian et al.’s (2014, 2015) work is limited to end-user preferences for elements in a small office space, their research confirms that even first-time IVE users were able to interact and communicate in a meaningful way. The research of Maftei and Harty (2015) also describes the use of an IVE to support collaborative design. They sought to develop new knowledge about “the effects of immersive technologies on construction design activities as used in concrete ‘reallife’ settings and as perceived by the practitioners involved.” (2015, p. 54). Their research emphasises the necessity of feeling both present and engaged in a design environment. The theoretical framing of their work is based on Schön’s model of the “reflective practitioner”, an idea that would require that an IVE enables oscillation between exploration and determination or divergence and convergence. While Maftei and Harty (2015) report the emergence of several modes of design communication in the IVE (3D modelling, gesture, vocal communication, laughter, etc.), their study does not investigate multimodal design or collaborative design in further detail.

7 Multimodal Collaborative Design in an Immersive Virtual Environment …

119

While virtual and augmented reality environments have become commercially available in recent years, most have only limited multimodal capacity, being focussed on human-to-computer rather than human-to-human interaction. Holth and Schnabel (2017) suggest that despite this limitation, increased availability of IVEs should reduce some of the technical and social barriers identified in previous research, with a growing number of people having experience in them. Moreover, Holth and Schnabel (2017) argue that the heightened spatial perception afforded by IVEs may support a deeper appreciation of the ways people interact with their environments. The concept of the IVE as a place for scenario modelling has been the subject of several research projects. For example, Mastrolembo Ventura et al. (2018) use IVEs to test design scenarios for pedestrian movement in healthcare buildings. They find that the IVE provides a means of visualising and communicating spatial conditions, which in turn facilitates user input in the design process. Similarly, Saeidi et al. (2018) consider the use of an IVE for producing and capturing projected user data during the design process. As such, both Mastrolembo Ventura et al. (2018) and Saeidi et al. (2018) use IVEs as spaces for simulation and data gathering. Two studies by Chowdhury and Schnabel (2019, 2020) investigate non-experts’ involvement in design teams using IVEs. Both studies examine communication modes and participant experiences. Through this research, Chowdhury and Schnabel propose a design communication framework that accommodates non-experts in team deliberations. Their research emphasises the importance of multimodal communication in IVEs, “because independent participants in the collaborative design process need to be able to coordinate and inform their activities through background or peripheral awareness of one another’s activities” (2019, p. 2). In their second study, Chowdhury and Schnabel (2020) examine barriers to participation and communication in design teams in IVEs. They postulate that in participatory design, “the design iteration stage suffers from a lack of shared perceptual understanding to generate collaborative design ideas” (2020, p. 451). In response, they test the use of an IVEbased design process for scenario modelling. Using protocol analysis, they identify key design communication modes such as “verbal communication, presence, and co-presence [and] the generation of 3D artefacts” (2020, p. 462). Past research described in this section reveals several assumptions, as well as a range of evidence and knowledge gaps about design and barriers to its effectiveness in IVEs. First, most studies position co-present, expert teams as the epitome of clear, transparent, and effective communication in a design team (Fig. 7.1a). While there are obvious flaws in this assumption—even co-present teams have social, cultural, and linguistic differences—it is partially responsible for early IVE research being focused on overcoming technical barriers (Fig. 7.1b). As technical issues have gradually been resolved or improved, and the desire to explore participatory and collaborative design in IVEs has grown, so too has the recognition of a range of social barriers that operate in parallel with and sometimes exacerbate the technical ones (Fig. 7.2a). Notably, research about non-cognate expert teams is largely absent in the literature (Fig. 7.2b).

120

M. J. Ostwald et al.

Fig. 7.1 Barriers to design collaboration and multimodality: (a, left), transparent communication in a co-present, cognate expert (EC ) team and (b, right) technical (T) barriers to communication in a cognate expert (EC ) team in an IVE

Fig. 7.2 Barriers to design collaboration and multimodality: (a, left), social (S), and technical (T) barriers to communication in a team combining experts (EC ) and non-experts (NE ) in an IVE and (b, right) complex or unknown barriers (?) to communication in a non-cognate expert (EN ) team

7.3 Methodology This chapter combines a critical review of the literature (Sect. 7.2) with the development of a framework identifying barriers to effective collaborative design in an IVE (Sect. 7.3), and a descriptive case study (Sect. 7.4). The case study provides information about an experience that addresses a knowledge gap in the field. The present section starts by classifying the barriers identified in the literature before introducing the case study.

7 Multimodal Collaborative Design in an Immersive Virtual Environment …

121

7.3.1 Barriers to Effective Design in IVEs Two recent systematic literature reviews have examined IVEs in design and various aspects of their application, modes of communication, and capacity to support team processes (Kalantari and Neo 2020; Yu et al. 2022). In the first of these reviews, Kalantari and Neo (2020, p. 28) commence with the idea that the IVE serves as a “prototyping tool in the interior design process, which helps to promote better design analysis and end-user involvement by presenting a realistic representation of a design concept”. Through their research, they synthesise four barriers identified in past research: (i) low levels of realism, (ii) low levels of immersion, (iii) low levels of sensory feedback, and (iv) time constraints, either externally imposed or associated with participants’ capacity to endure the IVEs. Kalantari and Neo (2020) also highlight that most research about IVEs is based on small experiments, few of which are associated with real-world projects. The second systematic review, by Yu et al. (2022, p. 2), has a greater focus on the use of IVEs for design collaboration across the architecture, engineering, and construction sectors, noting that multiple past studies have “emphasised that digital modalities and collaboration affect the quality, efficiency, and accuracy of design”. Yu et al. (2022) classify several categories of interaction in terms of platform or hardware types and then examine each for its capacity to support various modes of communication. One of their conclusions is that there remains a relative lack of research about design collaboration in IVEs. As such, future research should examine different types of teams, different modes of communication and ideation (creative visualisation) tools. Importantly, there is not only relatively little research available about these topics but very few cases have been described that address this knowledge gap. In general, regardless of the nature of the barriers identified in these systematic reviews, they tend to be linked to either “cognitive lag” or “cognitive load”. The former refers to a delay in the initialisation of mental resources, and the latter to the sustained encumbering of mental resources caused by an environment. To explain these two, it is often assumed that the use of a pencil to draw involves a direct connection between the mind and the paper (by way of hand and eye), and therefore, that representation, the creation of a drawing, is almost instantaneous. Conversely, drawing using a keyboard and mouse requires intermediary steps and processes, which have inbuilt delays or lags between thought and representation. A linguistic barrier (either speaking in a second dialect or communicating while learning a new professional language) could create a cognitive load that potentially reduces over time with practice and growing fluency. The barriers identified in past research can be summarised into two categories: technical and social factors (Table. 7.1). The first is associated with the capacity of hardware and software to provide a seamless, high-fidelity visual simulation of environmental and team co-presence. The more complex the technical interface, the greater the barrier to seamless engagement and the higher the cognitive load for users. It could be argued that the greatest inroads or developments in IVE research have

122

M. J. Ostwald et al.

Table 7.1 Barriers identified in the literature to effective collaborative design in an IVE Barrier

Category

Sources

Technical

• • • • •

Schnabel and Kvan (2002) Schnabel and Kvan (2003) Salter et al. (2009) Merrick et al. (2011) Heydarian et al. (2014) Maftei and Harty (2015)

Social

• The complexity of professional modes of representation • The complexity of professional modes of communication • Innate technical literacy levels (non-digital native)

Lack of visual/auditory immersion Low fidelity replication of the ‘real’ world Time and responsiveness delays Scale/weight/cost of hardware Software interoperability

Salter et al. (2009) Bullinger et al. (2010) Merrick et al. (2011) Holth and Schnabel (2017) Chowdhury and Schnabel (2019) Chowdhury and Schnabel (2020)

occurred in this first category, with the hardware and software interface becoming increasingly intuitive and unobtrusive. The second category of barriers, the social, are more complex and arguably also exist in traditional, co-present design teams. Two of the major social barriers relate to linguistic and cultural differences, which have the capacity to delay or confuse communication, although they are also linked to heightened creativity (Lee et al. 2020).

7.3.2 A Case Study in Operatic Design This research uses operatic design as an example of a type of non-cognate expert team design process. It presents iModel—a new networked modelling system—as a practical example of the operations of these types of teams in an IVE. iModel is used for this purpose because it is situated in the knowledge gap between collaborative and participatory design on the one hand and multimodal design on the other (Fig. 7.3). Operatic design is a type of participatory practice that requires coalescing expertise in fields as disparate as architecture, choreography, engineering, graphics, music, and textiles. Ideation, presentation, and communication in each of these domains follow conventions that translate into preferences for specific approaches and technologies. Consequently, design modalities and outputs present in various formats whose creative integration typically consumes energy and can divert focus. While, to a degree, capable of inspiring novel trajectories through creative friction, this disparity mostly increases cognitive load and produces inefficiencies across the overall design and production pipeline. iModel addresses this challenge by providing a modelling system that assimilates disparate conceptual modes into a special type of IVE, a 360-degree interactive cinematic visualisation system called AVIE networked to other desktop and mobile platforms. It is capable of animating design component interaction and provides real-time interaction capabilities. Hence as a tool, it enables the design and rehearsal

7 Multimodal Collaborative Design in an Immersive Virtual Environment …

123

Fig. 7.3 Case selection to address the knowledge gap associated with combining non-cognate expert teams and IVEs

of a full opera ahead of its physical translation onto the stage. That is, it provides the capacity to preview and modify the interplay between choreography, costumes, lighting, projections, props, set, and stage machinery, all articulated to the timing and structure of the musical score. As such, iModel’s objective is to facilitate a multimodal design process that operates as a symbiotic exchange between its noncognate users (e.g., designers, artists, directors, technical crew, and cast), as well as across the human–computer interface. In this way, it seeks to improve the creative scope and complexity of productions, workflows, and overall resource allocation. By visually integrating heterogenous design elements across the “ideation, testing, evaluation” pipeline, iModel was intended to enhance communication and understanding of creative visions across a diverse user group. By addressing technical and social barriers in this way, the intention is to catalyse a process of analysis and cross-pollination that surpasses the capabilities of existing sequential, isolated pipelines. The iterative collaborative assembly of draft components can facilitate rich dialogue arising from the identification of conceptual and material tensions between the stakeholders’ visions. By lightening cognitive load through rapid audio-visual manifestation, iModel can support the generation of more ambitious, coherent, and innovative operatic designs (Thurow et al. 2021).

124

M. J. Ostwald et al.

7.4 Findings This section commences with an overview of iModel, including the features that seek to limit the negative impacts of previously identified social and technical barriers. The implementation of iModel is then described—its hardware, software, and interface— along with its application in two operatic productions.

7.4.1 Implementing iModel The iModel project has been developed by an interdisciplinary team of artists, computer scientists, architects, theatre professionals, and 3D modelling experts at UNSW Sydney’s iCinema Research Centre in collaboration with OA, the nation’s largest performing arts organisation. It has been led by iCinema’s Executive Director, Laureate Professor Dennis Del Favero, and realised under the artistic guidance of Michael Scott-Mitchell, one of Australia’s most accomplished production designers working across opera, theatre, and event design. iModel consists of a software application developed for the Unity 3D Game Engine, which facilitates a shared virtual environment that can be accessed via diverse visualisation systems—ranging from desktop computers and tablets to 1:1scale 360° 3D cinematic CAVE (Cave automatic virtual environment) theatres. The latter provides the capability to articulate the visualisation to the human body, activating peripheral vision and prompting kinaesthetic responses, which support human sense-making capabilities beyond the solely visual. As such, they furnish inhabitable spaces that can viscerally evoke a design’s intended spatial and temporal dimensions—factors intended to reduce the social barriers to effective collaboration in an IVE. The iModel AVIE, is centred on a portable CAVE, measuring 10 m in diameter and 4.5 m in height, accommodating up to 30 users. As a mixed-reality theatre, it facilitates unconstrained full-body social collaboration between design teams. While the AVIE features an in-built motion-tracking system and supports a range of input controllers, the team opted for a keyboard-mouse interface since it best caters to the needs of precise input when adjusting complex designs on screen. In this way, designers can also remotely access the 3D environment via desktop computers or mobile devices to collaboratively work on their specific design pipelines while offsite. This proved particularly valuable during the extended Covid19 lockdowns when face-to-face collaboration was severely constrained. Users generate draft designs on a range of software applications (e.g., 3D models or Photoshop files) and upload these into iModel’s database. Available categories comprise choreography, costumes, set and props, directorial notes, lighting, musical score, stage robotics, and screen content. This heterogeneous data is funnelled through an online conversion pipeline that combines the Nginx webserver, Celery queuing software, and Django webapp to translate native files (e.g., .json, .dwg, or.fbx) into a bespoke format readable by the iModel application. This allows all

7 Multimodal Collaborative Design in an Immersive Virtual Environment …

125

design information to coalesce into a unified visualisation space, while reducing the need for local computing power. Furthermore, users can complement their data by drawing on iModel’s internal asset library, which contains, for example, commonly used props, simple shapes for early ideation, and a range of character avatars that can be “drag’n’dropped” into the modelling space. This space represents either a generic or a digitally twinned (i.e., real-world) theatre that already reflects the possibilities and constraints that a design team will have to take into consideration in their work (Thurow et al. 2023). For iModel’s initial implementation, the team digitally reconstructed the Sydney Opera House (SOH)’s Concert Hall, its Joan Sutherland Theatre, and the City of Dandenong’s Drum Theatre. These 3D models are rendered in high fidelity, capturing a wide range of detail (e.g., available stage machinery, floor, wall and ceiling textures, reconstructions of auditorium seats, etc.) to maximally approximate the atmospheric properties of the venue. Ray tracing simulates realistic lighting behaviour, such as reflection off shiny surfaces and casting of shadows. iModel’s interface is comprised of individual workspaces that cater to the needs of its distinct user groups, such as costume, set, or lighting designers. Functionalities are customised accordingly, extending to routine tasks and information needs. For instance, a set designer may want to overlay the stage view with a centre line or gridlines, indicate the pace and rotational direction of a revolve, or add, adjust, flip, and scale assets, such as props or set structures. All assets and their intended movements are recorded on a timeline “sequencer”, which is mapped to the musical score. Users can (de)couple assets and specify keyframes to streamline the editing of acts and scenes. A master window affords the capability to coalesce content from all (or selected) workspaces, layering it onto the sequencer—staging a virtual rehearsal. Users can play, pause, rewind, forward, and annotate keyframes (via text, drawing, or audio commentary), which can support in-situ discussion as well as remote collaboration among the creative and technical teams. The software also features an AI-assisted analysis tool that calculates points of collision, obstructions, and occlusions, which can be used to reduce potential hazards and to optimise actor path-tracing as well as audience sightlines.

7.4.2 Applications of iModel iModel has been iteratively developed through its use in two projects that involved members of the creative and technical teams of OA’s 2022/2023 production The Barber of Seville (dir. by Priscilla Jackman, set design by Michael Scott-Mitchell) and Sydney Opera House’s 50th-anniversary gala show Amadeus (dir. Craig Illott, set design by Michael Scott-Mitchell, lighting design Nick Schlieper). For these productions, both design teams drew on a full range of visualisation platforms, using desktop computers in combination with large wall-screens for initial drafting, porting designs into AVIE for 1:1-scale review, and lastly deploying laptops and tablets during rehearsals to finetune and annotate design components. The iModel system allowed

126

M. J. Ostwald et al.

both teams to unlock substantial efficiencies and synergies across the design workflow due to its capability to furnish seamless visual communication of the intended orchestration of cast members, set pieces, and props. The staging of both productions was to be carried by high degrees of dynamism and had to be realised in complex spatial settings: Barber was funded by the Opera Conference to tour at least 20 metropolitan and regional venues (differing widely in size and available infrastructure), while Amadeus was booked into a venue without a pre-existing theatrical stage and that was inaccessible for most of the rehearsal time due to refurbishment. These conditions meant that designs had to aesthetically resonate across large as well as small venues, provide a stage to anything from vast tableaux to intimate two-hander scenes, and that multiple technical teams were involved whose communicative needs had to be rapidly met to stay on schedule. The Barber team, for example, realised a modular set design whose components could be flexibly composed to accommodate available stage space across all 20 venues (Fig. 7.4) With the smallest venue as a baseline, iModel was used to create a malleable set design that could be expanded to accommodate larger venues by increasing the distance between staging elements, adjusting their angles, or adding non-essential modules to ensure it was experienced as a coherent production, regardless of the venue it was staged in. Cast members were choreographed to (de)construct set elements and props in tune with the musical score in full view of the audience. For Barber, Jackman and Scott-Mitchell used iModel as a rapid prototyping tool to block the stage space with simple shapes and to trace possible movement pathways for actors to manoeuvre across the stage. It enabled them to validate ideas, detect bottlenecks and pre-empt likely collisions, and to specify alternative routes, which fed an overall playful, twirling choreography that amplified the joyous pace of the musical score (Fig. 7.5). Performers were then able to virtually preview, understand, and safely rehearse their complex movements before entering the physical stage. The capability to preview the interaction of costumes within the set allowed for assessing their aesthetic effect and provided inspiration to adjust textures, detail, and colours. Furthermore, iModel proved expedient in preparing touring logistics for Barber, supplying the means to design custom road cases able to minimise the set elements’ space requirements. Amadeus is a stage play that features extensive orchestration along with segments from Mozart’s operas. For this production, the team drew on its capability to coordinate stage machinery and lighting design with cast and musical score. With its deepperspective set mounted on the SOH’s Concert Hall floor, multiple mobile screens were designed to divide the stage space both horizontally and vertically at varying intervals (Fig. 7.6). At the same time, two revolves downstage had to be coordinated with the large cast in opulent costumes moving about at frenetic pace (Fig. 7.7). Without props drawing focus on the otherwise bare stage, the lighting design had to set the atmospheric tone for each scene—hinging on its successful timing with the swift movements of and on the stage. iModel’s ability to preview animations of key scenes facilitated shared understanding among the team of the show’s intended aesthetic and mechanical realisation—greatly aiding in project planning and managing a show that reaped equal critical and popular acclaim (Fig. 7.8).

7 Multimodal Collaborative Design in an Immersive Virtual Environment …

127

Fig. 7.4 Example of set elements for The Barber of Seville (Opera Australia, 2022/3), visualised in iModel

Fig. 7.5 iModel sequencer for The Barber of Seville (Opera Australia, 2022/3)

7.5 Discussion iModel was conceptualised as meeting the needs of an operatic design team using the best IVE technology available. With this as its starting point, the design of iModel necessarily considered many of the barriers to collaborative, multimodal design.

128

M. J. Ostwald et al.

Fig. 7.6 Staging design for Amadeus (Sydney Opera House, Sydney 2022/3), visualised in iModel

Fig. 7.7 A team interacting with a stage set for Amadeus (Sydney Opera House, Sydney, 2022/3), visualised in the 360-degree AVIE. Note that the photograph of iModel project is without 3D vision correction

Nevertheless, as noted previously, social barriers, especially those relating to professional languages and cultures, also exist in co-present design teams and may even be valuable for supporting creative practices. As this chapter demonstrates, barriers to effective design collaboration in IVEs have been identified and examined in the past. For example, both Steed and Schroeder (2015) and Prabhakaran et al. (2022) consider the effectiveness of IVEs for supporting non-co-present teams. That is teams where not everyone is geographically present in a single space as well as the same IVE. iModel is a full immersion, co-present simulation where an entire group of users (up to 30 in number) can see both the operatic design and each other in a real space. This partially reflects Prabhakaran et al.’s (2022) argument that co-presence in an IVE could reduce cognitive, linguistic,

7 Multimodal Collaborative Design in an Immersive Virtual Environment …

129

Fig. 7.8 Rehearsal meeting with a final stage set for Amadeus (Sydney Opera House, 2022/3), projected in AVIE

and cultural barriers. iModel, however, also accommodates multiple simultaneous modes of interaction, many of which can occur outside the IVE. Paes et al. (2021) examined the cognitive benefits of IVEs, finding that increased immersion and realism may reduce cognitive lag. This is not, however, necessarily the same as increasing creative potential through the development of innovative tools because social communication modes must also be engaged. Hawton et al. (2018, p. 483) also highlight the need for integrated design tools to improve communication in IVEs. A catalyst for such research is the recognition that for an IVE to be effective, specialised modes of communication must exist that translate “design artefacts” in such a way as to support creativity. Unlike architectural design, where the product is largely static, operatic design calls for the delineation of both spatial and temporal components of a performance, which calls for the development of unique tools to support the design process. iModel’s sequencer is an example of the types of tools identified by Hawton et al. (2018), which are intended to overcome specialised cognitive barriers to effective production. The focus on theatre and operatic design is useful for this purpose, in the first instance because it has been repeatedly characterised as “a collaborative art” for noncognate experts. This is because “no one theatre artist works independently to create a performance” (Malloy 2014, p. 1). Secondly, theatre and operatic design is a type of multimodal, micro-design problem rare in architecture, but it also directly engages with similar spatial and representational concerns. Finally, theatre and operatic design occurs within the confines or limits of a building and negotiating the connections between the physical architecture and the temporal staging of a production is one crucial goal of the design team.

130

M. J. Ostwald et al.

7.6 Conclusion This chapter has two aims, first to identify and examine barriers to effective collaborative design in IVEs and second, to demonstrate how these barriers may potentially be overcome. For the first aim, a literature review was used to identify two types of barriers, the technical and the social, and eight sub-categories. For the second aim, a descriptive case study was used to show how some of these barriers might be accommodated or overcome in the design of iModel, a special type of IVE for operatic design. Both the case study description and the discussion section in this chapter link the barriers identified in the literature to aspects of iModel’s design, and the design of other IVEs. The combined method, literature review and descriptive case study, also addresses a defined identified knowledge gap (Yu et al. 2022), which is core to its contribution to future research. This contribution includes both new information about the technical implementation of a novel IVE, and a rare example of the use of an IVE to complete two major projects (rather than experiments or student projects). At a time when “global acceleration of remote working” has led to widespread plans to use IVEs to support design teams, examples of ways to overcome barriers to effective performance are urgently needed (Yu et al. 2022, p. 93). Finally, this research has several practical limitations. From a methodological perspective, a descriptive case study does not have the same validity as an empirical one. This case presented in this chapter, iModel, is still a work-in-progress, despite being already used by one of Australia’s foremost performing arts institutions to support the design of two major productions. In this chapter, it is positioned as a catalyst for a review of the complex, multi-dimensional barriers to communication that exist in diverse teams. There is not, however, detailed evidence available that iModel has overcome all the barrier types, and as previously noted, it should not be expected to. Complex social barriers exist in all team design processes, and technical solutions can never resolve all of these. Furthermore, the focus of this chapter has been on professional languages and the barriers they entail, but the members of a design team might also come from different geographic parts of the world and speak different vernacular dialects. As such, further research into multi-lingual and multicultural design teams in IVEs is clearly required. Acknowledgements This research has been supported by the Australian Research Council (ARC LP190100563). All figures are by the authors. Michael Scott-Mitchell was responsible for the set designs for both Amadeus and Barber of Seville.

References Azofeifa JD, Noguez J, Ruiz S, Molina-Espinosa JM, Magana AJ, Benes B (2022) Systematic review of multimodal human-computer interaction. Informatics 9(13). https://doi.org/10.3390/ informatics9010013

7 Multimodal Collaborative Design in an Immersive Virtual Environment …

131

Bannon LJ, Ehn P (2013) Design matters in participatory design. In: Simonsen J, Robertson T (eds) Routledge international handbook of participatory design. Routledge, New York, pp 37–63 Benetti S, Ferrari A, Pavani F (2023) Multimodal processing in face-to-face interactions. Front Hum Neurosci 2(17):1–9 Brandt E, Binder T, Sanders EB (2013) Tools and techniques: ways to engage storytelling, making and enacting. In: Simonsen J, Robertson T (eds) Routledge international handbook of participatory design. Routledge, New York, pp 145–181 Bravo A, Maier AM (2020) Immersive visualisations in design: using augmented reality for information presentation. In: International design conference—design 2020. Cambridge University Press, Cambridge UK, pp 1215–1224 Bullinger H-J, Bauer W, Wenzel G, Blach R (2010) Towards user-centred design in architecture based on immersive virtual environments. Comput Ind 61:372–379 Chowdhury S, Schnabel MA (2019) Laypeople’s collaborative immersive virtual reality design discourse in neighborhood design. Front Robot AI 6(97). https://doi.org/10.3389/frobt.2019. 00097 Chowdhury S, Schnabel MA (2020) Virtual environments as medium for laypeople to communicate and collaborate in urban design. Archit Sci Rev 63(5):451–464 Correia A-P, Yusop FD (2008) ‘I don’t want to be empowered’: the challenge of involving real-world clients in instructional design experiences. Proc PDC 2008:214–216 Di Benedetto S (2013) An introduction to theatre design. Taylor and Francis, London den Otter A, Emmitt S (2007) Exploring the effectiveness of team communication: balancing synchronous and asynchronous communication in design teams. Eng Constr Archit Manag 14(5):408–419 Goli A, Teymournia F, Naemabadi M, Garmaroodi AA (2022) Architectural design game: a serious game approach to promote teaching and learning using multimodal interfaces. Educ Inf Technol 27:11467–11498 Hawton D, Cooper-Wooley B, Odolphi J, Doherty B, Fabbri A, Gardner N, Hank Haeusler M (2018) Shared immersive environments for parametric model manipulation: evaluating a workflow for parametric model manipulation from within immersive virtual environments. In: Learning, adapting and prototyping, proceedings of the 23rd international conference of the association for computer-aided architectural design research in Asia (CAADRIA), pp 483–492 Heydarian A, Carneiro JP, Gerber D, Becerik-Gerber B, Hayes T, Wood W (2014) Immersive virtual environments: experiments on impacting design and human building interaction. In: Rethinking comprehensive design: proceedings of the 19th international conference on computer-aided architectural design research in Asia (CAADRIA), pp 729–738 Heydarian A, Carneiro JP, Gerber D, Becerik-Gerber B, Hayes T, Wood W (2015) Immersive virtual environments versus physical built environments: a benchmarking study for building design and user-built environment explorations. Autom Constr 54:116–126 Holth J, Schnabel MA (2017) Immersive virtual environments as a tool for exploring perceptional space. Int J Parallel Emergent Distrib Syst 32(1S):155–162 Kalantari S, Neo JRJ (2020) Virtual environments for design research: lessons learned from use of fully immersive virtual reality in interior design research. J Inter Des 45(3):27–42 Kress G, Selander S (2012) Multimodal design, learning and cultures of recognition. Int High Educ 15(4):265–268 Kvan T (2000) Collaborative design: what is it? Autom Constr 9:409–415 Lavender A (2020) Multimodal acting and performing. In: Elleström L (ed) Beyond media borders, vol 1. Springer, Cham, pp 113–140 Lee JH, Ostwald MJ, Ning G (2020) Design thinking: creativity, collaboration and culture. Springer, Cham Luck R (2018) Participatory design in architectural practice: changing practices in future making in uncertain times. Des Stud 59:139–157

132

M. J. Ostwald et al.

Maftei L, Harty C (2015) Designing in caves: using immersive visualisations in design practice. Int J Architectural Res 9(3):53–75 Malloy K (2014) The art of theatrical design: elements of visual composition, methods, and practice. Taylor and Francis, London Manzini E, Rizzo F (2011) Small projects/large changes: participatory design as an open participated process. CoDesign 7(3–4):199–215 Mastrolembo Ventura S, Hilfert T, Archetti M, Rizzi M, Spezia A, Tagliabue LC, Oliveri E, Ciribini ALC (2018) Evaluation of building use scenarios by crowd simulations and immersive virtual environments: a case study. In: 35th international symposium on automation and robotics in construction, np Merrick KE, Ning G, Wang X (2011) Case studies using multiuser virtual worlds as an innovative platform for collaborative design. J Inf Technol Constr 16:165–188 Ostergaard KJ, William Wetmore R, Divekarx A, Vitali H, Summers JD (2005) An experimental methodology for investigating communication in collaborative design review meetings. CoDesign 1(3):169–185 Paes D, Irizarry J, Pujoni D (2021) An evidence of cognitive benefits from immersive design review: comparing three-dimensional perception and presence between immersive and non-immersive virtual environments. Autom Constr 130:103849 Prabhakaran A, Mahamadu A-M, Mahdjoubi L, Boguslawsk P (2022) BIM-based immersive collaborative environment for furniture, fixture, and equipment design. Autom Constr 142:104489 Saeidi S, Chokwitthaya C, Zhu Y, Sun M (2018) Spatial-temporal event-driven modeling for occupant behavior studies using immersive virtual environments. Autom Constr 94:371–382 Salter JD, Campbell C, Journeay M, Sheppard SRJ (2009) The digital workshop: exploring the use of interactive and immersive visualisation tools in participatory planning. J Environ Manage 90:2090–2101 Schnabel MA (2011) The immersive virtual environment design studio. In: Wang X, Tsai JJ-H (eds) Collaborative design in virtual environments. Springer, Berlin, pp 177–191 Schnabel MA, Kvan T (2002) Design communication in immersive virtual environments: an initial exploration. In: Architectural information management. 19th eCAADe conference proceedings, pp 472–478 Schnabel MA, Kvan T (2003) Spatial understanding in immersive virtual environments. Int J Archit Comput 4(1):435–448 Schultz C, Bhatt M (2012) Multimodal spatial data access for architecture design assistance. AI EDAM 26(2):177–203 Sedivy J, Johnson H (2000) Multimodal tool support for creative tasks in the visual arts. KnowlBased Syst 13:441–450 Shanthi Priya R, Shabitha P, Radhakrishnan S (2020) Collaborative and participatory design approach in architectural design studios. Soc Sci Humanit Open 2:100033 Sleeswijk Visser F, Van Der Lugt R, Stappers PJ (2007) Sharing user experiences in the product innovation process: participatory design needs participatory communication. Creativity Innov Manag 16:35–45 Steed A, Schroeder R (2015) Collaboration in immersive and non-immersive virtual environments. In: Lombard M, Biocca F, Freeman J, Ijsselsteijn W, Schaevitz R (eds) Immersed in media. Springer, Cham, pp 263–282 Thurow S, Del Favero D, Wallen L (2021) Inhabitable models—immersive intelligent aesthetics for scenographic design. Theatre Perform Des 7(1–2):82–95 Thurow S, Del Favero D, Ostwald MJ, Grehan H (2023) Augmenting creative symbiosis using a cyber/physical aesthetic. In: Proceedings of the 28th international symposium on electronic art. ISEA, Paris, pp tbc

7 Multimodal Collaborative Design in an Immersive Virtual Environment …

133

van Leeuwen T (2014) Design, production and creativity. In: Jones R (ed) Discourse and creativity. Routledge, London, pp 133–142 Yu R, Ning G, Ostwald MJ (2021) Computational design: technology, cognition and environments. Taylor and Francis, CRC Press, London Yu R, Gu N, Lee G, Khan A (2022) A systematic review of architectural design collaboration in immersive virtual environments. Designs 6(93). https://doi.org/10.3390/designs6050093

Chapter 8

The Museum of Touch: Tangible Models for Blind and Low Vision Audiences in Museums Dagmar Reinhardt , Leona Holloway , Jane Thogersen , Eve Guerry , Claudio Andres Corvalan Diaz , William Havellas , and Philip Poronnik

Abstract Museums curate collections of artworks, artefacts, and specimens, often shielded by barriers that prohibit physical interaction. Consequently, these exhibits are effectively inaccessible to people with visual impairments, preventing meaningful engagement with these cultural treasures. This study explores the idea of touch models, facilitating a bridge between objects in these collections and everyone, irrespective of their visual abilities. This chapter discusses three primary areas: (a) implementation of 3D scanning and 3D printing techniques, enhancing the touch experience for blind and low vision audiences in museums; (b) integration of multiple modalities, particularly 3D and audio elements, designed to further enrich the engagement for these individuals; and (c) a potential for sharing and transferring these touch models through open platforms to extend accessibility and knowledge dissemination. By utilising datasets as the foundation, this research advocates for a multi-modal approach to museum exhibition practices. Ultimately, this study seeks to eliminate accessibility barriers in museums, empowering blind and low vision individuals to immerse themselves in the narratives and cultural treasures encapsulated within these institutions. Keywords Multimodal · Museum · Touch access · Blind and low vision · Photogrammetry · Scanning · 3D printing · Object-based learning D. Reinhardt (B) School of Architecture, Design and Planning, The University of Sydney, Camperdown, Australia e-mail: [email protected] L. Holloway Monash University, Melbourne, Australia J. Thogersen · E. Guerry Chau Chak Wing Museum, The University of Sydney, Camperdown, Australia C. A. C. Diaz · W. Havellas · P. Poronnik The Faculty of Medicine and Health Media Lab, School of Medical Sciences, The University of Sydney, Camperdown, Australia © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 J. H. Lee et al. (eds.), Multimodality in Architecture, https://doi.org/10.1007/978-3-031-49511-3_8

135

136

D. Reinhardt et al.

8.1 Introduction Museums provide sites of learning, social interaction and cultural discourse by enabling access to a rich array of artworks, artefacts and specimens through their collections and exhibitions. Often, these collections are fragile or light sensitive, heavy, sharp, toxic, or too large/small to be handled. Unique and irreplaceable exhibits are consequently kept behind glass or in temperature-controlled environments. Artefacts and specimens can only be viewed and are literally out-of-reach for a general public (Candlin 2004; Classen 2005), yet more importantly, don’t perceptually exist for blind and low vision (BLV) visitors. Accessibility to museums for blind and low vision audience is not a niche problem, since there are globally approximately 285 million individuals with visual impairments, of whom around 39 million are classified as legally blind. 65% of individuals with visual impairments are aged over 50, even though this age group constitutes only 20% of the world’s population (Vision Australia 2019; World Health Organisation 2019). Additionally, at least 2.2 billion people have a near or distance vision impairment (World Health Organisation 2019). As this paper argues, BLV communities can be significantly supported by introduction of multi-modalities for museums, by including touch and audio for a good museum experience. In this context, Object Based Learning (OBL) is a good example as this strategy uses museum artefacts, specimens and artworks to elicit, stimulate and question student interaction with their field of study, thus enabling transformative learning (Chatterjee and Hannan 2015). Here, ‘object’ commonly refers to 3-dimensional exhibits (artefacts, specimens, or sculptures), 2-dimensional items (paintings, photographs, documents or pressed-natural specimens), but museums can also employ 3D replica (Fig. 8.1).

Fig. 8.1 Multi-modal experience with oral explanation and 3D models and natural specimens, for increasing touch access experiences in a museum context. Displayed here an abalone shell with additional growth (left), and sequence of comparison objects during consultancy session

8 The Museum of Touch: Tangible Models for Blind and Low Vision …

137

Consequently, our research centres on the utilisation of 3D printed duplicates as surrogates for handling, offering a solution that enables visitors from the BLV community to physically interact with museum content while safeguarding delicate and valuable collections, such as the Natural History collection at the Chau Chak Wing Museum (CCWM). This approach aligns with the recent trend in museums digitising their collections, whereby 3D scans provide data for interactive 3D models, thus granting global access. By way of comparison, we have conducted scans of natural specimens with similar geometric, material, and expressive characteristics in collaboration with the Faculty of Medicine and Health (FMH) Media Lab. These scans provide us with a dataset that can be transformed into 3D printed replicas using SLA resin, replicas that can be explored and discussed by the BLV community. Moreover, we are actively exploring methods to share this 3D data with a broader audience. The CCWM digital museum database serves as a virtual gateway to the museum, allowing individuals to experience it from a distance. In our pursuit of creating a “museum of touch,” we have established a database and platform on Sketchfab, where anyone can access and download 3D models of natural specimens for home 3D printing. Additionally, these 3D replicas or models can be expanded upon to provide supplementary information. The concept of multi-modality is pivotal here, as our aim is not to replace other modes of engagement but to complement them and introduce a fresh channel for interaction. In this regard, multi-modality fosters a crucial skill for all learners: the ability to seamlessly transition between various formats and modes, enhancing their comprehensive understanding of the information associated with each artefact. We consider multimodality as a systematic approach that potentially can provide a meaningful museum visit and involve more than an opportunity to view the collections, offering audiences an invitation to interpret objects and exhibitions and to gain insights into a narrative or perspective. This requires both design and curatorial knowledge to construct narratives, reinterpret the object and introduce context and background. These are data that reach well beyond merely exhibiting the object towards highlighting details, comparative scales, biological principles or rules of geometry. We aim to explore how this can be achieved through the sense of touch enhanced with complementary audio. In the following, this chapter provides a background and discusses methodologies and practices to create a museum of touch. We report on design strategies for 3D printed replicas, derived from consultation with BLV people that enabled useful insights how to improve 3D models, interpretational support and audio content for BLV experiences.

8.2 Background This section presents the background context of this chapter, offering an overview of museum context and over touch experiences for BLV; strategies for object-based learning; and the current state-of-the-art for 3D models.

138

D. Reinhardt et al.

8.2.1 Touch Access for Museum Experiences Even more so than for a sighted audience, people who are BLV require accompanying information to support their interaction and understanding of materials such as museum artefacts. It should be noted here that touch access differs fundamentally from vision in several important ways. Firstly, less detail is available through touch compared with vision (Hatwell 2003; Heller and Gentaz 2014). Furthermore, touch requires direct contact with an object and must therefore be deliberate, whereas vision provides casual access to all objects within the individual’s field of view (Arnheim 1990). Lastly, touch access provides information only about one small area of contact at each point of time (Lopes 1997), whereas the whole field of view is available at once using vision. Tactile objects should therefore be introduced with an explanation and overview to facilitate understanding. Additionally, the BLV community is frequently perceived as a uniform entity, with attention primarily given to extreme visual impairment (Wilson et al. 2020). Nonetheless, visual impairment encompasses a wide spectrum of complexities and variations, ranging from partial sight, blurriness, tunnel vision, perception of light, to color blindness (Hayhoe 2013). Consequently, while museums often prioritise physical interaction or audio-based content, often leaning towards offering braille resources, it is advantageous to consider multiple sensory modalities for enhancing museum experiences for BLV individuals. In fact, BLV communities have expressed a growing demand for increased tactile accessibility in galleries and museums (Candlin 2003, 2004), which includes the provision of 3D models through the use of 3D printing (Anagnostakis et al. 2016; Borsotti and Bollini 2009; Cooper 2019; Fogle-Hatch 2020). Recent studies have begun to address accessibility and inclusion for BLV audiences in a museum context, reporting positive responses of BLV people to models, artefacts or replicas (Fuller and Watkins 2011; Race et al. 2023; Scianna and Filippo 2019; Vaz et al. 2021), including those that are 3D printed (Karaduman et al. 2022; Wilson et al. 2017). Touch serves as a crucial means for BLV individuals to acquire insights and knowledge, but it also offers benefits to the wider museum-going public (Comes 2016; Novak et al. 2020). Figure 8.2 illustrates an example that integrates tactile elements into the coin collection within the enclosed glass case.

8.2.2 Context: CCWM and Object Based Learning (OBL) The research context for this investigation into multi-modal experiences and 3D touch access is the Chau Chak Wing Museum (CCWM, The University of Sydney) and its three sub-collections; the Nicholson Collection (Ancient Near East, Egyptian, Greek, Roman, Cypriot and Indus Valley material culture); the University Art Collection; and the Macleay Collections (natural history, scientific instruments and ethnography). Access to these museum collections is facilitated through various means, including exhibitions, digital access avenues such as the museum’s website,

8 The Museum of Touch: Tangible Models for Blind and Low Vision …

139

Fig. 8.2 Precedent study for tactile objects in Museums: coin (Taler von Bern, 1679), simplified raised profile on upscaled replica (Historical Museum, Frankfurt am Main, Germany)

online collection search tools, podcasts. Furthermore, the museum offers educational programs designed to enhance engagement, such as Object-Based Learning initiatives, as well as programs tailored for schools and the general public (as depicted in Fig. 8.3). OBL refers to a pedagogical approach that engages learners through objects, with a focus on transferable skills including deep looking, critical analysis, creative problem solving, communication and learning (Hannan et al. 2013; Insulander and Selander 2009). OBL practices promote tactile engagement, although the use of gloves is necessary, creating a continued barrier to the full tactile experience (Fig. 8.3). While this approach effectively addresses the accessibility concerns for most collections, it encounters challenges when dealing with natural specimens. These specimens prove intricate to handle due to their inherent characteristics, agerelated fragility, brittleness, and susceptibility to damage. Additionally, the way in which animals or plants were originally collected, preserved, and stored adds to the complexity. Presently, most natural specimens are showcased behind protective glass, some housed in the original heritage cabinets generously donated by the Macleay family during the late nineteenth century. To put it in perspective, less than three percent of natural history specimens are available for public display, with the remaining vast majority residing in storage, and in select cases, accessible through the Object-Based Learning (OBL) Program. Exhibitions often strive to portray animals in their true contextual settings. For instance, beetles might be pinned with needles to simulate a mid-flight posture, while fishes, jellyfish, or frogs are enclosed in jars filled with formaldehyde or ethanol to mimic their anti-gravitational environments. Taxidermy specimens, including birds, reptiles, or mammals, are meticulously posed to represent their characteristic stances. Archiving these items through advanced scanning techniques emerges as a viable solution, offering the added advantage of harnessing the associated data, including the possibility of 3D printing. Furthermore, these digitised specimens can be shared through online platforms.

140

D. Reinhardt et al.

Fig. 8.3 Chau Chak Wing Museum (Sydney); the museum collections are accessible as physical/ spatial presence (top); in the digital web archive (bottom) and in the OBL sessions (right)

8.2.3 3D Scanning and State-Of-The-Art: 2.5-3D Models and Replicas 3D scanning is a methodology widely adopted for digital models of cultural heritage sites, artefacts and monuments and serves research, restoration planning, and virtual showcasing purposes. Museums use 3D scanning to create digital replicas of valuable artefacts which helps to preserve these objects and make them more accessible to the public (Kantaros et al. 2023). For micro and macro data capture, portable scanners or photogrammetry provide high accuracy and fidelity results and are versatile for use in different settings or environments, including for travelling between different museum sections or different sites (see Fig. 8.4).

Fig. 8.4 Data capture: scanning with handheld device and rotating disk, adopted for capturing objects or environments/scenarios, usable for different scales

8 The Museum of Touch: Tangible Models for Blind and Low Vision …

141

Scanner capture enables detailed 3D data efficiently and quickly and with a high level of detail, with no contact so that risk or damage to scanned objects can be reduced. Scanners can be used for objects of all sizes and types, as each scanner has a recommended scanning size range due to its specific scanning distance from the object. Whilst very efficient in scanning models, the initial costs of these scanners can be a limiting factor. Photogrammetry uses a camera or phone app to capture a series of photographs taken from different angles to reconstruct a 3D model using specific software. While commonly available, cost effective and well suited for environments and large structures, photogrammetry at high resolution requires a large number of photographs, with high time affordance. Other challenges and limitations might be complexity of shape, high detail, or reflection of surfaces. In contrast to photogrammetry, scanning using handheld 3D scanners is quick, efficient and accurate. However, some challenges exist as investment in the hardware software is costly and assembling the images can further be computationally intensive, requiring powerful software and hardware. 3D scanned data can be used for 3D prints, for digital museum interactions (eg. online), but also as virtual objects for immersive experiences such as virtual tours or interactive exhibits, explorations and research analysis. 3D printing produces tangible replicas or scaled versions of artefacts and cultural heritage items. 3D printing (additive manufacturing) provides a cost-effective, ubiquitous, and simple technique for producing 3D objects/replica and interpretations with a high degree of precision and detailing. 3D print technologies include Fused Deposition Modelling (FDM), Stereolithography (SLA), and Selective Laser Sintering (SLS). 3D models service a wide range of domains including educational, exhibition, and preservation purposes. In contrast to traditional methods of conservation often involving physically handling artefacts, which potentially initiates damage or deterioration over time, 3D imaging and printing in a museum context allows for preservation of fragile or endangered artefacts through creation of physical replicas. Physical replicas can also be employed for exhibitions and displays, allowing people to view and interact with artefacts and structures in a tangible way that can heighten the engagement and learning experience and making the museum accessible for a wider range of people. Recent research has explored natural history specimens. An overview is presented in Table 8.1. Natural specimens include: coral, shell, crab and bone from the Oxford University Museum of Natural History (Wilson et al. 2020) and a series of skulls differing in size and structure at the Buffalo Bill Center of the West (Heinrich et al. 2014). Wilson and colleagues (2020) noted that 3D printed objects are particularly valuable for museum visitors who are BLV because they may have less prior exposure to such objects.

142

D. Reinhardt et al.

Table 8.1 Recent research is increasingly investigating multi-modalities for touch access, which includes sculptures and replicas with 2.5D reliefs and 3D models and can also include audio labels Mode

Author

Original

Translation

3D

Ania (2020)

Bronze sculptures

Tactile models, shows the art production process through a series of tactile materials

2.5D

Eardley et al. (2016)

Tile panels

Raised tactile reliefs, replica created intentionally in white (no colour info)

3D

Henrich et al. (2014)

Buffalo Bill Ccenter of the West

Series of skulls differing in size and structure

3D

Montusiewicz et al. (2018)

Stone sculptures from the mid-twentieth century, depicting mainly human figures

3D scanned, edited, 3D printed in PLA. Details (such as eyebrows) enhanced. Portions selected (full sculpture not reproduced) and base added for stability

3D

Reichinger et al. (2012)

L-D converter

3D scanned to create 1:50 miniatures of large exhibits. 3D printed models were up to 16 cm in size

3D

Wilson et al. (2020)

Natural specimen, including: tortoise shell; brain coral; crab shell carapace; fossilised scallop shell; right femur of a red fox

3D scanned and 3D printed

3D+

Fogle-Hatch et al. (2020)

Stone projectiles

3D scanned and 3D printed, with QR code that releases audio info

2D+

Quero et al. (2018)

Painting (Vincent van Gogh, ‘Starry Night’)

2.5/3D model representation with conductive paint, coupled with Arduino board and audio feed

8.3 Methodology 8.3.1 Dataflow from Scan to Print Our research project aims to enable access to fragile specimen collections for BLV audiences, assembled as 3D printed replicas, comparative sets between natural and printed replica and paralleled with narratives that engage in descriptions, context and explanations for biological principles.

8 The Museum of Touch: Tangible Models for Blind and Low Vision …

143

Fig. 8.5 Natural comparative specimen that investigate different character attributes (geometry, details, spatiality, weight, size, ‘springiness’, etc.)

The scanning process explores two categories of objects: (a) specimens from the museum’s Macleay collection and (b) natural specimens gathered specifically for this project, designed to serve as comparative object groups in terms of geometry, weight, texture, haptic qualities, colours, size, and details. The latter category of specimens was scanned as non-museum items to facilitate their transportation and facilitate advice by a BLV control group, thereby assessing their fidelity and viability in comparison to the originals. These objects presented various challenges due to their varying shapes, colours, sizes, and compositions, with some being exceptionally fragile. Figure 8.5 illustrates comparison sets with natural specimens with shared ‘hard’ aspects, across marine and terrestrial environments. To address the diverse range of objects to be scanned, a strategic approach based on their size and composition was employed. For objects not exceeding approx. 20 cm, a Spider scanner (Artec3D® ) was used, given its ability to capture high-quality object details, mesh architecture, and texture. In instances involving larger objects where detailed capture was not the primary concern, a Leo scanner (Artec3D® ) was chosen as the most suitable option. When dealing with objects or specimens < 5 cm, a Micro scanner was employed, offering precision in capturing even the minutest of details. For objects with complex compositions, a mixed approach involving multiple scanners and subsequent merging of scans in software was adopted. Before initiating the scanning process, each object underwent meticulous preparation to ensure optimal results and prevent damage, especially for fragile items. Manipulation and support techniques were employed to minimise the risk of damage during scanning, thereby enhancing scan quality through stabilisation. In cases where complex objects presented challenges in scanning, a surface texturing spray was applied to facilitate visualisation by the scanner. Each object was positioned carefully to enable optimal

144

D. Reinhardt et al.

Fig. 8.6 Dataflow: Natural comparative specimen (oyster), scan, obj dataset, 3D print environment, print with scaffold

capture, and the scanning process was executed with specific parameters and configurations tailored to each scanner (Spider, Leo, or Micro). Resolution and precision settings were adjusted as per the specific requirements of each object. Once scanning was completed, data post-processing was performed. Specialised software (Artec Studio 17 Professional® ) was used to clean, align, and merge the point clouds generated by the scanners. This allowed obtaining accurate and complete three-dimensional models of each scanned object for preparation of (a) 3D printing; or (b) 3D viewing. Data to be viewed in 3D form were textured using the scanners cameras, with file types provided as OBJ for the 3D scan data, JPG for the texture image and MTL for the material file linking the 3D data with the texture image. Scans to be 3D printed were exported as a STL file which contains data of the 3D mesh. Figure 8.6 illustrates the workflow and file sets from natural specimen to 3D replica. The 3D data sets compiled by scan were then revised in a 3D modelling environment and using 3D printing programs (makerbot, CURA). The 3D files were fabricated as a series of scaled replica or 3D models in different materials, including commonly plastic filament (FLS) and resin (SLA), depending on the desired final properties for the printed object, including weight, strength, flexibility, and surface texture and finish. SLA 3D printing was chosen as a method that uses photopolymerisation to produce objects whereby the printer uses a light source to solidify the liquid resin. The 3D object is constructed layer by layer as the light source hardens the resin. Translation from natural object to representation can thus be tested on the 3D model. Figure 8.7 illustrates a range of natural specimen with varying characteristics, including three-dimensionality (branch); geometrical complexity and surface contrast between smooth and sharp (oyster); and dry-wet contrast (sponge) (Fig 8.7). Datasets were further collected as .obj files and shared on Sketchfab and Thingiverse (see: https://sketchfab.com/museumfortouch), so that information becomes

8 The Museum of Touch: Tangible Models for Blind and Low Vision …

145

Fig. 8.7 3D printing files based on high fidelity scans of various marine and terrestrial specimen that demonstrate natural growth principles, here: seed pods (top), oyster shell (mid) and sea sponge (bottom). Comparison to actual objects to demonstrate geometry, scale, detail

accessible and shareable for BLV audiences, museums, educators and a general public.

8.3.2 Enhancing Characteristics Developing touch access models also means that 3D data might not be printed 1:1, but that data is re-interpreted and enhanced for better tactility. The true significance of this project lies in this re-interpretation of collection items to go beyond reproducing exact replicas and to allow audiences to explore a narrative or particular aspect of the object, through touch. For example, the specimen Ostracion cubicus (Box Fish, NHF.1398) is mounted fish skin that represents challenges in terms of age related damage (fins), but also finesse of tectonic scale pattern. We explored how properties should be changed/varied so touch modalities would be improved. This could include

146

D. Reinhardt et al.

Fig. 8.8 Increasing tactile criteria through high fidelity scan, 3D print and geometrical conversions for better tactility for BLV: Ostracion cubicus (Box Fish) skin. Collected Fair Cape, QLD. NHF.1398, SLA printing, 2023. Natural History—Chau Chak Wing Museum

repairing broken elements (rear fin) or changing fins from embedded in body surface to facing outwards to render them more recognisable. As illustrated in Fig. 8.8, this process involves enhancing the tactility of the distinctive hexagonal pattern covering the fish skin by translating it into a tactile profile, effectively elevating the pattern by an exaggerated 2 mm to facilitate touch recognition.

8.4 Results Datasets derived from scans of museum exhibits can enable a touch experience for BLV audiences and enrich engagement for these individuals by further enabling acting as keystones for narrative, storytelling and audio elements, as is discussed in the following.

8.4.1 Museum Specimen and Comparison Collection Tangible interaction is not a unisensory task. Research into sensory psychology (Heller and Ballesteros 2006; Lederman and Klatzky 2004; Spence 2018) and neuroscience (Lacey and Sathian 2014; Sathian 2005; Voss 2016) refers to our senses being integrated within our perceptual systems. The more senses are integrated, the wider the perceptual experience, hence including multiple modalities is an important factor for museum engagement. We used an expanded set of 3D models to further investigate how 3D models can not only become available replica that display most accurately the objects themselves, but also explore how series of 3D models can help construct an understanding of natural principles, such as growth (seed pods), evolution (shells) or specialisation (Darwin’s Finches), see Fig. 8.9. Significantly,

8 The Museum of Touch: Tangible Models for Blind and Low Vision …

147

Fig. 8.9 Selection of touch access model series for natural specimen, exploring food specialisation (Darwin’s Finches), sorting shells (void/solid comparison), identifying gradient patterns (shells) and understanding growth (seed pods and detail)

for people who are congenitally blind (i.e., since birth), their passive or incidental experience of the world refers to touch, sounds, smell, and language; with touch requiring more deliberate exploration and being limited to that which is on a human scale and available in the immediate environment. They may therefore lack some of the concepts that sighted people take for granted, such as the movement of a bird’s wings (Franco 1982; Recchia 1997). While this adds motivation for the need for touch access to museum objects, it also means that additional explanations may be needed to understand some objects and their context. As part of the study, we therefore also explore how kinetics or movement can be explained, such as a shark moving. We define this aligning of tactile and audio elements as a narrative kit, which ideally should include series of objects that can be compared to demonstrate a principle or phenomenon or differences, such as a comparison between species (e.g. tooth kit with humans and sharks, Fig. 8.10), but also audio cues and audio prompts that help to explain the context and story of the objects/species. Multimodality here means that a series of related objects are introduced and explained (such as Darwin’s finches which stand for evolutionary principles of food specification), so that the experiences is formed through internal ‘visualisation’ (mental image-forming) while listening and experiencing. This also requires narrative sequencing, detailed explanations, and potentially audio add-ons of ambient sound (as part of the species environment).

148

D. Reinhardt et al.

Fig. 8.10 Multi-sensory experience: combining models and story in the narrative set/kit for touch access in museums—comparison study of predator/prey, shark/human with a focus on movement (left) and teeth (right). Objects partially fragments (human) and details with scales increased

8.4.2 BLV Consultation As part of the scoping for this study and to frame-up the project, we consulted with BLV representatives in Australia and overseas regarding their preferences for the creation of tactile models for museum engagement. A summary of design considerations based on consultation for enhancing the tactile experience of objects includes: • Prioritise Real Touch: Whenever feasible, prioritise providing a real tactile experience for objects rather than relying on alterations. • Accurate Replicas: Use accurate replicas of objects instead of making alterations to them, ensuring users can explore the object as it truly appears. • Full Representations: Present complete representations of objects, avoiding excerpts or partial views that may not provide the full context. • Relief Focus: Highlight object features like patterns or textures in a 2.5D relief format, placed alongside the object to prevent distortions of its form and make it easier for users to understand. • Consider Weight, Geometry, Dimensions, Temperature, and Texture: Take into account the importance of factors such as weight, geometry, dimensions, temperature, and texture, as these play a crucial role in assessing objects, aligning with previous research findings (Gentaz and Hatwell 2003; Hatwell et al. 1990; Katz 1925; Klatzky et al. 1985). • Scale Representation: Present objects to scale or include a scale reference alongside the object to allow users and testers to assess their size accurately.

8 The Museum of Touch: Tangible Models for Blind and Low Vision …

149

• Scaled-Up Details: Add scaled-up details or features to objects to make them more accessible through touch and interpretation. • Distinguishable Limbs/Appendixes: Ensure that limbs and appendages (e.g., head/ ears, arms/legs, fins) are clearly distinguishable from the torso/body and ideally oriented outward instead of being aligned with the body. • Address Geometrical Complexity: Provide explanations for the logical organisation of geometrically complex objects to aid users in understanding their structure. • Audio Descriptions: Include audio descriptions as an assistive solution, offering a description of the object itself and providing background/context or extended explanations when needed.

8.5 Discussion 8.5.1 3D Touch Access Guidelines and Kits Scopignio et al. (2015) explored the options and issues involved in 3D printing for museum heritage and called for guidelines on issues such as how to prepare a model for 3D scanning and how much editing should be done to ensure that the models are tactually distinct for BLV individuals. Prior research into the use of 3D models for museum visitors who are BLV has identified that: full round models are preferred to bas reliefs (Fuller and Watkins 2011); scaled-up 3D models enable better detection of details (Karaduman et al. 2022; Wilson et al. 2020); realism is desired, for example in terms of weight and thermal properties (Wilson et al. 2017) but is not of primary importance (DiFranco et al. 2015). Importantly, a preference for aligning/providing touch and sound modalities has been indicated, such as descriptions (Karaduman et al. 2022; Scianna and Filippo 2019) and accompanying information that explains how the model differs from the original, for example in terms of scale and materiality (Wilson et al. 2017). However, while studies have begun into materiality for objects, scale and patterns, little is known about how multi-modal information of objects could be combined and presented so as to constitute a good touch experience. The previously described strategy is a multi-modal narrative kit combines 3D models that deliver comparative attributes, for example varying in size, pattern, body posture, position of limbs, with added different modes of access e.g. sound recording (audio description and/or soundscape), braille description, touch object, etc. More research is needed here for interaction and media design.

150

D. Reinhardt et al.

8.5.2 Digital Datasets: Sharing Opportunities Museums increasingly digitise their collections and develop digital datasets and digital archives. This digitisation of objects, artefacts and specimens holds multiple benefits; including archiving, tracing, assessing, engaging, replication and interpreting based on the scan and resulting digital datasets. • Archiving: By scanning originals, museums can produce a ‘digital twin’, a dataset that can be further used to store, share and investigate the original. • Tracing: This dataset the allows for tracking any changes in condition or state of preservation and for the design of bespoke and object-specific storage and display solutions (Wachowiak and Karas 2009). • Accessing: In the digital version, users have access with 360° ‘viewability’ of objects on display, which also includes remote access, and the potential for scaling of objects to be smaller or larger. • Engaging: Digital archives generate possibilities for profound and sustained engagement both before and after visits. • Replicating: Datasets can be 3D printed to scale or enlarged, including focusing on specific parts for higher resolution and attention. • Interpreting: The manipulation of digital objects enables researchers or audience to enhance features or attributes, section or otherwise connect the original to an intended narrative. In addition, sharing datasets in publicly accessible, open web platforms (Thingiverse or Sketchfab) connects museum collections, artefacts and items on a virtual level across the world. More importantly, this allows communities and cultural institutions to download and print 3D models for the blind and partially sighted, including children, adults, educators and teachers (Fig. 8.11).

8.5.3 Expanded Modalities: Braille and Audio While this paper focuses on creating accurate and engaging 3D printed replicas with explanatory sets, for full inclusion, additional modalities should be considered. During museum visits, visitors typically view objects on display and read labels. Some may encounter longer text panels or audio guides that provide deeper context. However, these methods may not cater to all audiences, including those who are too young, have language barriers, or prefer audio or tactile experiences. Embracing universal design for learning principles benefits all learners, and studies into the use of 3D models of museum artefacts for people who are BLV suggest that extended descriptions are essential for understanding (Karaduman et al. 2022; Reichinger et al. 2012; Scianna and Filippo 2019).

8 The Museum of Touch: Tangible Models for Blind and Low Vision …

151

Fig. 8.11 Museum of Touch website at online platform (Sketchfab), with examples of natural specimens, and computationally modelled types

To make 3D touch models more accessible, methodologies can increase multimodality, including audio, braille, and electronic documentation. These are discussed in the following short overview. Braille: Braille is recommended for literacy support and communication for people who are blind and deafblind. It can be accessed in hard copy or through a refreshable tactile display, which can also be read aloud using a screen reader. Audio: Audio is a versatile format for drawing attention to items, providing information, and offering multisensory stimulation. Audio description translates visual aspects for the benefit of visually impaired individuals. Museums commonly provide audio guides for both sighted and BLV visitors, however, should also describe objects and explain differences from the original, such as scale and materiality (Wilson et al. 2017). Audio Labeling: Complex surfaces of 3D models limit the use of braille labels, leading to exploration of audio labelling options (Butler et al. 2021). The most popular techniques for adding audio labels to 3D printed models have been the use of computer vision response to tags on the user’s hands (e.g., Coughlan et al. 2020; Shi et al. 2017) or embedded electronic circuits (Chopra and Jain 2014; Ghodke et al. 2019; Reinders et al. 2023). These approaches, along with locational beacons, are gaining popularity for labelling 3D touch models in museums. Several companies offer assistance for implementing technology for audio labelling in museums, including smart ring that interacts with 3D surfaces, returning audio information

152

D. Reinhardt et al.

about the area being touched (Tooteko); interactive tactile museum exhibits using touch buttons with embedded electronics or 3D models on touch screens (TouchGraphics); or computer vision with user-friendly software for attaching audio labels and interactive activities related to parts of a model placed on a base device (Inventivio, Tactonom). In summary, based on datasets, 3D prints and initial kit strategies, our research advocates for a multi-modal approach to museum exhibitions, involving technologies like braille displays, tactile graphics, laser-cut layered graphics, 3D printed models, and soundscapes. Ultimately, our research aims to remove accessibility barriers, empowering blind and low vision individuals to engage fully with the narratives and cultural treasures housed within museums and thereby benefitting all visitors universally.

8.6 Conclusion This chapter has discussed the inaccessibility of museum exhibits to BLV individuals due to barriers preventing physical interaction. It introduced touch models as a solution, bridging the gap between these objects and people regardless of their visual abilities. The chapter covered three key aspects: (a) enhancing touch experience for blind and low vision audiences through the application of 3D scanning and 3D printing techniques in museums; (b) enriching engagement for these individuals by incorporating various modalities, including 3D and audio elements; and (c) facilitating accessibility and knowledge sharing by potentially distributing these touch models via open platforms. Multi-sensory experiences can then be enhanced for all learners through multimodality, particularly when audiences are interacting with the physical object, high res 2D image, 3D digital model or 3D print of the digital model; and when additional dimensions are being added, such as audio- and audio-visual items. Solutions are 3D printed models and explanatory sets, which can be further enhanced by expanding through more modalities, such as assistive technologies like braille displays, tactile graphics, audio labelling and soundscapes that can provide alternative forms of access for BLV audiences to museum content. Through these many modalities that speak to a maximum of sense, access, engagement and learning in a museum context can be highly expanded. Acknowledgements “Understanding Museum Exhibits through Touch” is a research project led by A/Prof. Dagmar Reinhardt (The University of Sydney), with Dr. Jane Thogersen and Dr. Eve Guerry (CCWM), Leona Holloway (Monash University), in collaboration with the Health Media Lab, USyd, Nextsense Australia and IBOS Denmark. Supported by the Alastair Swayn Foundation, International Research Grant, Round Two, 2022 Grant ID: INT004. Research support: Dane Jensen (scripting) and Farisa Adi (3D printing). Scans by The Faculty of Medicine and Health Media Lab (Prof. Philip Poronnik, Claudio Andres Corvalan Diaz, William Havellas), The University of Sydney. 3D prints by DMaF, ADP, The University of Sydney.

8 The Museum of Touch: Tangible Models for Blind and Low Vision …

153

References Anagnostakis G, Antoniou M, Kardamitsi E, Sachinidis T, Koutsabasis P, Stavrakis M, Vosinakis S, Zissis D (2016) Accessible museum collections for the visually impaired: combining tactile exploration, audio descriptions and mobile gestures. In: Proceedings of the 18th international conference on human-computer interaction with mobile devices and services adjunct. Association for Computing Machinery, Florence, Italy, pp 1021–1025 Ania M-J (2020) Art nouveau & ecology. VIP action: how to make heritage accessible for blind & visually impaired people. Réseau Art Nouveau Network, Brussels Arnheim R (1990) Perceptual aspects of art for the blind. J Aesthetic Educ 24(3):57–65. https:// doi.org/10.2307/3332799 Borsotti M, Bollini L (2009) Reshaping exhibition and museum design through digital technologies: a multimodal approach. Int J Virtual Reality 8(3):25–31 Butler M, Holloway L, Reinders S, Goncu C, Marriott K (2021) Technology developments in touchbased accessible graphics: a systematic review of research 2010–2020. In: International CHI conference on human factors in computing systems (Yokohama, online) Candlin F (2003) Blindness, art and exclusion in museums and galleries. Int J Art Des Educ 22(1):100–110 Candlin F (2004) Don’t touch! hands off! Art, blindness and the conservation of expertise. Body Soc 10(1):71–90 Cavazos Quero L, Bartolomé JI, Cho J (2021) Accessible visual artworks for blind and visually impaired people: comparing a multimodal approach with tactile graphics. Electronics 2021 10:297. https://doi.org/10.3390/electronics10030297 Chatterjee H, Hannan L (2015) Engaging the senses: object-based learning in higher education, pp 1–230 Chopra K, Jain A (2014) Acoustics in 3D printing. ACM SIGSOFT Softw Eng Notes 39(4):1–7. https://doi.org/10.1145/2632434.2632441 Classen C (2005) Touch in the museum. In: Classen C (ed) The book of touch. Berg, Oxford and New York, pp 275–286 Comes R (2016) Haptic devices and tactile experiences in museum exhibitions. J Ancient Hist Archeol 3(4). https://doi.org/10.14795/j.v3i4.205 Cooper C (2019) You can handle it: 3D printing for museums. Adv Archaeol Pract 7(4):443–447. https://doi.org/10.1017/aap.2019.39 Coughlan JM, Shen H, Biggs B (2020) Towards accessible audio labeling of 3D objects. J Technol Pers Disabil 8 Di Franco, Di Giuseppantonio P, Camporesi C, Galeazzi F, Kallman M (2015) 3D printing and immersive visualization for improved perception of ancient artifacts. Presence 24:243–264 Eardley AF, Mineiro C, Neves J, Ride P (2016) Redefining access: embracing multimodality, memorability and shared experience in museums. Curator Museum J 59:263–286. https://doi.org/10. 1111/cura.12163 Fogle-Hatch C (2020) Tactile access and the 3D-printing process. https://museumsenses.org/tactileaccess-and-the-3d-printing-process/ Fogle-Hatch C, Nicoli J, Winiecki D (2020) Designing a portable museum display of native American stone projectile points (arrowheads) to ensure accessibility and tactile quality. J Blindness Innov Res 10(2) Franco KJ (1982) Concept formation and development in the congenitally blind child. Honours thesis, University of Richmond, Paper 485 Fuller R, Watkins WR (2011) Exhibit design relating to low vision and blindness: research on effective use of tactile exhibits with touch activated audio description for the blind and low vision audience (white paper). Indiana University, Bloomington, IN Gentaz E, Hatwell Y (2003) Haptic processing of spatial and material objects. In: Hatwell Y, Steri A, Gentaz E (eds) Touching for knowing: cognitive psychology of haptic manual exploration. John Benjamins Publishing Company, pp 123–159

154

D. Reinhardt et al.

Ghodke U, Yusim L, Sowmya S, Coppin P (2019) The cross-sensory globe: participatory design of a 3D audio-tactile globe prototype for blind and low-vision users to learn geography. In: Proceedings of the 2019 on designing interactive systems conference (DIS ‘19), Association for Computing Machinery, San Diego, CA, USA Hannan L, Duhs R, Chatterjee H (2013) Object-based learning: a powerful pedagogy for higher education. In: Museums and higher education working together Hatwell Y, Orliaguet JP, Brouty G (1990) Effects of object properties, attentional constraints and manual exploratory procedures on haptic perceptual organization: a developmental study. In: Bloch H, Bertenthal B (eds) Sensory-motor organizations and development in infancy and early childhood. Klumer Academic Publishers, pp 315–335 Hatwell Y (2003) Touch and cognition. In: Hatwell Y, Streri A, Gentaz E (eds) Touching for knowing: cognitive psychology of haptic manual perception. John Benjamins Publishing Company, Amsterdam, Netherlands, pp 1–16 Hayhoe S (2013) Expanding our vision of museum education & perception: an analysis of three case studies of independent, blind, arts learners. Harv Educ Rev 83(1):67–86 Henrich G, Cleveland FQ, Wolverton E (2014) Case studies from three museums in art beyond sight’s multi-site museum accessibility study. Mus Soc Issues 9(2):124–143. https://doi.org/10. 1179/1559689314Z.00000000023 Heller MA, Ballesteros S (eds) (2006) Touch and blindness: psychology and neuroscience. Lawrence Erlbaum Associates Publishers Heller MA, Gentaz E (2014) Psychology of touch and blindness. Psychology Press, New York, NY, USA Insulander E, Selander S (2009) Designs for learning in museum contexts. Des Learn 2(2):8–21. https://doi.org/10.16993/dfl.21 Kantaros A, Ganetsos T, Petrescu FIT (2023) Three-dimensional printing and 3d scanning: emerging technologies exhibiting high potential in the field of cultural heritage. Appl Sci 13(8):4777. https://doi.org/10.3390/app13084777 Karaduman H, Alan Ü, Özlem Yi˘git E (2022) Beyond “do not touch”: the experience of a threedimensional printed artifacts museum as an alternative to traditional museums for visitors who are blind and partially sighted. Univers Access Inf Soc. https://doi.org/10.1007/s10209-022-008 80-0 Katz D (1925) Der Aufbau der Tastwelt (The world of touch) Klatzky RL, Lederman SJ, Metzger VA (1985) Identifying objects by touch: An “expert system”. Percept Psychophy 37(4):299–302. https://doi.org/0.3758/BF03211351 Lacey S, Sathian K (2014) Visuo-haptic multisensory object recognition, categorization, and representation. Front Psychol 5. https://doi.org/10.3389/fpsyg.2014.00730 Lederman SJ, Klatzky RL (2004) Multisensory texture perception. In: Calvert GA, Spence C, Stein BE (eds) The handbook of multisensory processes, pp 107–122 Lopes DMM (1997) Art media and sense modalities: tactile pictures. Philos Q 47(189):425–440 Martiniello N, Haririsanati L, Wittich W (2022) Enablers and barriers encountered by workingage and older adults with vision impairment who pursue braille training. Disabil Rehabil 44(11):2347–2362. https://doi.org/10.1080/09638288.2020.1833253 Montusiewicz J, Miłosz M, K˛esik J (2018) Technical aspects of museum exposition for visually impaired preparation using modern 3D technologies. IEEE global engineering education conference (EDUCON), Santa Cruz de Tenerife, Spain Novak M, Phelan S, Lewalter D, Schwan S (2020) There is more to touch than meets the eye: haptic exploration in a science museum. Int J Sci Educ 42(18). https://doi.org/10.1080/09500693.2020. 1849855 Race L, D’Souza S, Flouty R, Igoe T, Hurst A (2023) Understanding accessible interpretation through touch object practices in museums. Int J Inclusive Mus 16(1):75–98. https://doi.org/10. 18848/1835-2014/CGP/v16i01/75-98

8 The Museum of Touch: Tangible Models for Blind and Low Vision …

155

Recchia SL (1997) Play and concept development in infants and young children with severe visual impairments: a constructionist view. J Vis Impairment Blindness 91(4):401–406. https://doi.org/ 10.1177/0145482X9709100408 Reichinger A, Schröder S, Löw C, Sportun S, Reichl P, Purgathofer W (2016) Spaghetti, sink and sarcophagus: design explorations of tactile artworks for visually impaired people. In: NordCHI, Gothenburg, Sweden Reinders S, Ananthanarayan S, Butler M, Marriott K (2023) Designing conversational multimodal 3D printed models with people who are blind. In: ACM designing interactive systems (DIS ’23), association for computing machinery. Pittsburgh, PA, USA. https://doi.org/10.1145/3563657. 3595989 Sathian K (2005) Visual cortical activity during tactile perception in the sighted and the visually deprived. Dev Psychobiol 46(3):279–286. https://doi.org/10.1002/dev.20056 Scianna A, Di Filippo G (2019) Rapid prototyping for the extension of the accessibility to cultural heritage for blind people. In: International archives of the photogrammetry, remote sensing and spatial information sciences, Ávila, Spain Scopignio, R, Cignoni P, Pietroni N, Callieri M, Dellepiane M (2015) Digital fabrication technologies for cultural heritage (STAR). In: Proceedings of the Eurographics workshop on graphics and cultural heritage, Wiley Sheffield R, D’Andrea F-M, Morash V, Chatfield S (2022) How many braille readers? Policy, politics, and perception. J Vis Impairment Blindness 116(1). https://doi.org/10.1177/014548 2X211071125 Shi L, Zhao Y, Azenkot S (2017) Markit and Talkit: a low-barrier toolkit to augment 3D printed models with audio annotations. In: 30th annual ACM symposium on user interface software and technology, Association for Computing Machinery Sketchfab. https://sketchfab.com/feed Spence C (2018) Multisensory perception. In: Wixted J, Serences J (eds) The Stevens’ handbook of experimental psychology and cognitive neuroscience (4th ed, vol 2). Wiley, pp 1–56 Thingiverse. https://www.thingiverse.com/ Ucar E (2015) Multisensory MET: touch, smell, and hear art. The metropolitan Museum of Art. https://www.metmuseum.org/blogs/digital-underground/2015/multisensory-met Vaz R, Freitas D, Coehlho A (2021) Visiting museums from the perspective of visually impaired visitors. Int J Inclusive Museum 14(1). https://doi.org/10.18848/1835-2014/CGP/v14i01/71-93 Vision Australia (2019) Global facts on blindness and low vision. https://visionaustralia.org/news/ 2019-08-23/global-facts-blindness-and-low-vision Voss P (2016) Auditory spatial perception without vision. Front Psychol 7. https://doi.org/10.3389/ fpsyg.2016.01960 Wachowiak MJ, Karas BV (2009) 3d scanning and replication for museum and cultural heritage applications. J Am Inst Conserv 48(2). https://doi.org/10.1179/019713609804516992 Wilson PF, Griffiths S, Williams E, Smith PM, Williams MA (2020) Designing 3-D prints for blind and partially sighted audiences in museums: exploring the needs of those living with sight loss. Visitor Stud 23(2):120–140. https://doi.org/10.1080/10645578.2020.1776562 Wilson PF, Stott J, Warnett JM, Attridge A, Smith PM, Williams MA (2017) Evaluation of touchable 3D-printed replicas in museums. Curator Museum J 60(4):445–465. https://doi.org/10.1111/ cura.12244 World Health Organization (2019) World report on vision. World Health Organization. https://www. who.int/publications/i/item/9789241516570

Chapter 9

How to Enhance Architectural Visualisation Using Image Gen AI Jin-Kook Lee , Hyun Jeong , Youngchae Kim, Suhyung Choi, Hayoung Jo, Sumin Chae, and Youngjin Yoo

Abstract This research explores the applicability of image generation artificial intelligence (image gen AI) techniques for diverse design visualisation within the field of architecture. In architecture, images of building exteriors and interior spaces are commonly used as reference images for design and communication purposes, particularly in the early stages of design planning. However, generating a single image involves a complex process and requires significant time, economic and human resources. To address this challenge, this chapter proposes an approach that efficiently generates reference images for interior spaces, building facades, and building forms using image-generation (“image gen”) AI. Based on the image gen AI, the process of this study consists of two main stages: (1) Intensive Test of the Default Model (2) Model Fine-Tuning Process. Within this framework, the architectural focus of this research covers four aspects: (1) Generating indoor space images with diverse design styles, (2) Designing bathroom spatial layouts based on users’ physical and medical characteristics, (3) Creating facade designs that capture regional characteristics, and (4) Generating housing images that reflect various renowned architects’ design styles. Through these efforts, the research demonstrates the potential of AI in the field of architecture and contributes to the advancement of architectural image generation research. Keywords Architectural design · Visualisation · Image generation AI · Model fine-tuning

9.1 Introduction Visualisation plays a crucial role in conveying and facilitating the understanding of ideas and designs through visual means in the field of architecture and interior architecture (Chiu 1995). During the process of designing spaces, a vast amount J.-K. Lee (B) · H. Jeong · Y. Kim · S. Choi · H. Jo · S. Chae · Y. Yoo Department of Interior Architecture and Built Environment, Yonsei University, Seoul 03722, Republic of Korea e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 J. H. Lee et al. (eds.), Multimodality in Architecture, https://doi.org/10.1007/978-3-031-49511-3_9

157

158

J.-K. Lee et al.

of architectural visual information is generated in various forms such as handdrawn sketches, building drawings, photographs, videos, and text (Kalay 2004; Phare et al. 2018). Visual information in architecture is highly preferred by non-experts, including clients, as it provides a clear representation of design information and concepts. Traditionally, photorealistic images have been used to depict and identify buildings in a conventional manner throughout the history of architecture (Borden 2007). However, manual labour was primarily required to create visual information, involving multiple stages of work to express architectural or interior design concepts. This process was time-consuming and labour-intensive, with limited flexibility and efficiency, especially when design changes or modifications were necessary. To overcome these challenges, automated methods for generating visual information using image generation artificial intelligence (“image gen AI”) have received significant attention. Recent advancements in image gen AI platforms such as Diffusion-based Midjourney (Oppenlaender 2022), DALL-E 2 (Ramesh et al. 2022), and Stable Diffusion (Rombach et al. 2022) have sparked great interest among researchers. These large language models (LLMs) based image gen AIs have demonstrated the ability to generate purposeful images through fine-tuning. In this context, we focus on architectural visualisation using Image gen AI and aim to explore design alternatives that align with specific objectives and visualise them. Particularly, our research aim is to fine-tune a pre-trained default model and propose an architectural visualisation approach that incorporates the visual characteristics of the learned elements. Through this research, we intend to offer insights into the use of generative AI for architectural visualisation and contribute to enhancing the efficiency of architectural design stages.

9.2 Methodology 9.2.1 Overall Process Since the emergence of gen AI (Vaswani et al. 2017), AI platforms capable of generating images based on text have rapidly advanced in fields such as design, art, and gaming (Vimpari et al. 2023). This study analysed the image generation performance of three well-known AI platforms: Stable Diffusion, DALL-E2, and Midjourney. Based on the analysis, it used Stable Diffusion (SD) that demonstrated the most stable generation capability. SD utilises diffusion techniques to maintain the stability of pixel values during the image generation process, resulting in high-quality and consistent image outputs while minimising noise and ambiguity. Additionally, SD is an open-source platform, making it easily accessible for users to utilise in their studies and experiments. Based on SD, the process of this study consists of two main stages. Figure 9.1 illustrates the overall research process.

9 How to Enhance Architectural Visualisation Using Image Gen AI

159

Fig. 9.1 Overall research process

1. Intensive Test of Default Model: Conduct an intensive test using the default model. The understanding of an architectural topic is assessed based on the generated results, and subjects with low understanding are selected as the training targets. 2. Model Fine-Tuning Process: Perform fine-tuning on the selected training targets to generate a fine-tuned model. The process of fine-tuning is divided into data preparation, hyperparameter optimisation, and training stages. The following equation can also represent the overall process addressed in this study. genT arget(target, AI M, Param) → G I

(9.1)

The genT arget () function takes inputs such as target, image gen AI default model (AI M), and parameters (Param) to produce generated images (G I ). If most of G Ii does not belong to the target image (T I ) set, AI M is transformed into a FineTuned model (F T M) capable of generating T I . This process can be defined as the get T I M() function, which utilises information about T I to propose default models developed by other researchers or the need for fine-tuning. This study is focused on implementing F T M through the method of fine-tuning. / T I )then, i f (Most o f G Ii ∈

(9.2)

get T I M(target ∨ T I ) → F T M

(9.3)

genT arget(target, F T M, Param) → T I

(9.4)

160

J.-K. Lee et al.

Table 9.1 The standard format of prompts used for image generation Prompts

Positive

Description prompts

Target, target requirements, keyword None elements, specific scene description

Negative

Image quality prompts

Professional photograph, photorealistic rendering, realistic, enhance-detail, v ray rendering, full HD, masterpiece, highly detailed, high quality, 8k, full shot, deep depth of field, f/22, 35 mm, high-key lighting, natural lighting, realistic shadows

Bad proportion, Low quality, awkward shadows, unrealistic lighting, pixelated textures, Worst, noisy, unrealistic reflections, normal quality, watermark, bad perspective, confusing details, blurry textured, blurry, noise, cloudy, faint, text, tacky, crowded, signature

9.2.2 Intensive Test of Default Model There are two methods of image generation in SD; image-to-image (img2img) and text-to-image (txt2img) (Saharia et al. 2022). We primarily focused on the txt2img approach to proceed with the intensive test of the default model. In the txt2img approach, text prompt engineering was integrated to generate images that adhere to predefined criteria. The prompts consist of two main components: (1) Description prompts for the target and (2) Image quality prompts. The description prompts provide detailed text descriptions that are used to generate specific target images, guiding the model to understand the desired content and context for image generation. The image quality prompts focus on generating high-resolution images and systematically classify both positive and negative aspects that reflect both reflective and non-reflective elements of the target image. Table 9.1 showcases some template prompts used in the image generation process. The widely downloaded default model, SD1.5V checkpoint, was used for image generation. The essential configurations, including the sampling step, CFG scale, and image size, were individually adjusted for each architectural topic. Each image generation took an average processing time of approximately 5 s. The results of image generation demonstrated consistent functionality in generating high-quality images across most targets using the default open model. However, there were limitations in accurately representing specific targets due to a low level of recognition for those specific targets. Fine-tuning for specific targets is required to overcome these limitations and improve the accuracy of image generation.

9.2.3 Model Fine-Tuning Process Model fine-tuning was conducted on the default SD model for the target topics. All processes conducted in this study were performed on a local PC equipped with an RTX A6000 GPU model, which had 47.5 GB of memory. The fine-tuning

9 How to Enhance Architectural Visualisation Using Image Gen AI

161

Fig. 9.2 Three FT processes (FT 1, 2, and 3)

(FT) process involves the following steps. FT (1) Data Preparation (including Preprocessing): The first step involved preparing and pre-processing the data specific to the target. This included collecting reference images, organising the dataset, and ensuring consistency in the data format. FT (2) Hyperparameter Optimisation: The next step focused on optimising the hyperparameters of the model to enhance its performance in generating images of the target. This involved FT parameters such as learning rate, batch size, and network architecture to achieve better results. FT (3) Training: The final step entailed training the model using the prepared dataset and optimised hyperparameters. The model was trained to learn the specific characteristics and nuances of the target, enabling it to generate images that align more closely with the target. This study adopts the Low-rank Adaptation (LoRA) method (Hu et al. 2021), which allows for few-shot learning. Figure 9.2 represents three FT processes. FT 1: Fine-tuning requires high-quality training dataset with consistent representation (Kim and Lee 2020). Training Dataset consists of two components: Training Image Data and Training Text Data (Ramesh et al. 2021). To ensure high-quality and consistent representations for the target, this research uses appropriate collecting tools for each topic, such as journal or magazine, BIM Modelling, street view, etc. Before using these extracted images, we perform a pre-processing step that includes resizing to ensure compatibility and uniformity within the dataset. The overview of step FT 1 is shown in Fig. 9.3. FT 2: We conducted fine-tuning using various combinations of hyperparameters based on the previously constructed training dataset. The goal was to optimize the hyperparameters and improve the quality of the LoRA-based fine-tuned model. To evaluate the performance of each hyperparameter combination, the corresponding loss value was extracted. Hyperparameters encompass a range of different settings, but this study focused on optimizing the train batch size, epochs, learning rate, learning rate scheduler, and learning rate warmup. These hyperparameters help enhance the quality of the fine-tuned model. Table 9.2 explains the main roles of each hyperparameter.

162

J.-K. Lee et al.

Fig. 9.3 Overview of FT (1) Data Preparation

Table 9.2 The main roles of each hyperparameter Hyperparameters

Role

Train batch size

Number of examples processed together in each training iteration

Epoch

Complete iteration through the entire training dataset

Caption extension

Augmenting or generating additional captions to expand the training data for better performance

Learning rate

Controls the step size during model parameter updates

Learning rate scheduler

Specifies how the learning rate changes over the course of training

Learning rate warmup

Gradually increasing the learning rate at the beginning of training to stabilize the optimization process

FT 3: Training was conducted using the training data and hyperparameter settings to generate the target model. The training time varied for each model. After the FT completion, a LoRA model with a size of only 144 MB in (.safetensors) format was generated. The generated fine-tuned model is used in conjunction with the default SD model during the further image generation process. The overview of step FT 3 is shown in Fig. 9.4.

9.3 Findings This section showcases various architectural visualisation images generated by finetuned models. The architectural topics covered in this chapter are as follows: (1) Mixmatch visualisations of interiors by 26 style keywords, (2) Remodel visualisation of bathrooms for elderly people, (3) Commercial building facade design of local identity, and (4) Perspectives of housing designs by renowned architects’ style. These topics are illustrated in Table 9.3.

9 How to Enhance Architectural Visualisation Using Image Gen AI

163

Fig. 9.4 Overview of FT (3) Training

Table 9.3 Topics of architectural visualisation Visualisation method

Space boundary Architectural topics

3D Indoor Photorealistic rendering

(1) Mix-match visualisations of interiors by 26 style keywords (2) Remodel visualisation of bathrooms for elderly people

Outdoor

(3) Commercial building facade design of local identity (4) Perspectives of housing designs by renowned architects’ styles

9.3.1 Mix-Match Visualisations of Interiors by 26 Style Keywords Interior design encompasses various styles and trends based on different design elements, colour choices, and materials used. Additionally, each space has its own individual style influenced by regional, cultural, and user preferences. This allows for the emergence and evolution of new design styles and variations. However, the default SD model may not always reflect the latest trends in interior design. Therefore, this chapter utilised a fine-tuned model based on 26 interior design styles to generate interior space images. Table 9.4 represents some image results of a few interior design styles generated using the fine-tuned model. Furthermore, by combining various styles and trends in innovative ways, it can stimulate creativity for new design ideas that haven’t been experienced before. Carefully selecting and harmonizing elements such as design keywords, material finishes, and colour schemes, we aimed to surpass the limitations of conventional designs and visualise unique design spaces through the fusion of diverse styles. By applying mix and match of interior design using fine-tuned models, which considers user preferences, we can efficiently provide customised designs. The generated image data

164

J.-K. Lee et al.

Table 9.4 Generated images of various interior styles using the fine-tuned models Input

Output

Zen

Retro

Brutalism

Junk

Sustainable

Kinfolk

can be used as reference material in the field of interior design, aiding in exploring new design styles (Kim et al. 2019). Additionally, archiving this data can serve as a resource for training other gen AI models and for creative image generation purposes. Table 9.5 represents the results of image generation through mix and match.

9.3.2 Remodel Visualisation of Bathrooms for Elderly People This section shows the images using fine-tuned models on safety equipment considering the physical aging of elderly people. The safety equipment that was specifically trained in this research is suitable for musculoskeletal aging users. As the aging society rapidly progresses worldwide, the concept of rental elderly welfare housing and silver towns is increasing in the field of architecture (Lee et al. 2020). However, most residential spaces, excluding buildings specifically designed for the elderly, are not suitable for the physical aging process. For this reason, this research focused on users with musculoskeletal aging (including those with the use of assistive devices) and generated images of bathroom with safety equipment. The safety equipment

9 How to Enhance Architectural Visualisation Using Image Gen AI

165

Table 9.5 The results of image generation through mix and match

for creating an elderly-friendly bathroom space was produced using a modelling program based on BIM (Building Information Modelling) (Lee et al. 2016; Shin and Lee 2019). Table 9.6 presents the showcased safety equipment, categorized for both Musculoskeletal Aging Users and Musculoskeletal Aging Users who need assistive devices. By using fine-tuned models based on the bathroom safety equipment listed in the table above, it is possible to obtain bathroom images for the elderly. For instance, in the generated bathroom images for individuals with musculoskeletal aging, appropriate placement of shower curtain, folding shower chair, washbasin with chair, nonslip floor, and toilet wall grab bar is ensured. The generated images are shown in Table 9.7.

166

J.-K. Lee et al.

Table 9.6 Bathroom safety equipment for musculoskeletal aging users

Bathroom Safety Equipment

Musculoskeletal Aging Users

Both

Musculoskeletal Aging Users who need Assistive Devices

Folding Shower Angle Adjustment Chair Mirror

Toilet Side Grab Bar

Walk-In Bathtub

Nonslip Floor

Washbasin with Chair

Shower Curtain

Bathtub/Shower Grab Bar

Washbasin with a Wide Top

Toilet Wall Grab Bar

Draw-Out Faucet

Bathtub Chair

Flush Button

Height-Adjustable Height-Adjustable Washbasin Toilet Grab Bar

Table 9.7 The results of image generation for musculoskeletal aging users Input

Output

Musculo skeletal Aging Users Musculoskeletal Aging User who needs Assistive Devices

If additional equipment installation is desired for the user’s bathroom image generated through Txt2img, the image can also be modified using the img2img and inpaint methods. By providing a text prompt related to the additional equipment and specifying the desired installation area, it is possible to generate the desired image. Figure 9.5 provides an example of image generation with the addition of a grab bar next to the toilet using in-paint.

9 How to Enhance Architectural Visualisation Using Image Gen AI

167

Fig. 9.5 Example of using in-paint (grab bar next to the toilet)

9.3.3 Commercial Building Facade Design of Local Identity Among the various physical elements in cities and regions, building facades play a significant role in people’s aesthetic perception and understanding of the local environment (Sun et al. 2022). This section utilised fine-tuned models to generate building facade images based on the characteristics of Seoul, South Korea: especially Seongsu-dong. This approach can serve as a tool for expressing the distinctive identity of buildings when designing structures in specific locations (Sharifi Noorian et al. 2020). As a result of learning the overall neighbourhood facilities images, entire images of the building reflecting the “Seoul style” and “Seongsu-dong style” were created. Table 9.8 summarizes the output images generated by fine-tuned models. Additionally, this approach leverages the canny edge detector feature of the ControlNet extension available in the SD Web UI. It enables the processing of real building images requiring remodelling due to a change in use and generates multiple design alternative images. The comparison between input and output images of façade design using fine-tuned models is shown in Table 9.9. Table 9.8 The result of image generation using the fine-tuned models Input

Seoul Style

Seongsu-dong Style

Output

168

J.-K. Lee et al.

Table 9.9 Facade image generation using img2img approach

INPUT

A Building Type Region

Prompt

B Neighbourhood Facilities

The whole of Seoul

Seongsu-dong in Seoul

Design Style of Seoul Neighbourhood Facilities, Front View of Neighbourhood Facilities Building in Seoul

Neighbourhood facility of Seongsu, Storefront design style of Seongsu, Facade design style of Seongsudong

OUTPUT

Image

Generated Image

9.3.4 Perspectives of Housing Designs by Renowned Architects’ Style This section utilised fine-tuned models based on the design styles of 20 architects to conduct image generation. We selected 20 architects with high international recognition, such as recipients of the Pritzker Prize, also known as the Nobel Prize of Architecture, or those who have had a significant design influence, and focused on generating images that capture their respective styles by using our fine-tuned models. Table 9.10 shows some image results of architects’ design style using the fine-tuned model. By combining various architects’ design styles (Merkel 2008; Vandenbulcke 2013) and manipulating the elements through operations like addition, subtraction, multiplication and division, architects can creatively achieve outcomes of new architectural styles that have not previously existed. This approach enables efficient support for providing clients with diverse design alternatives during the initial design phase (Farsäter and Olander 2019). Table 9.11 represents the template prompts

9 How to Enhance Architectural Visualisation Using Image Gen AI

169

Table 9.10 The results of image generation based on architects’ design style Input

Output

Ando Tadao

Luis Barragan

I. M. Pei

Louis Kahn

Le Corbusier

Renzo Piano

SANAA

Shigeru Ban

for architects’ design style operation, and Table 9.12 shows the results of image generation for architects’ design style operation.

9.4 Discussion This research delves beyond conventional architectural visualisation methods to explore an AI-based approach for automated architectural image generation. While various case studies and empirical evidence have demonstrated AI’s potential in the field of architecture, utilizing existing Image Gen AI for precise aspects like shadows

170

J.-K. Lee et al.

Table 9.11 The template prompts for architects’ design style operation Operation Addition: A + B

Prompt Positive

Negative

“Architect A” and “Architect B” inspired-residential house

None

Subtraction: A − B “Architect A” inspired-residential house

“Architect B” design features

Multiplication: () (to enhance)

If its additional trained model is available, put a numeric extent from 0 to 1 after colon. If its additional trained model is not available, use (parenthesis) on the prompt

None

Division: [] (to diminish)

Use [brackets] on the prompt

Use (parenthesis) on the prompt

and numerical representation remains limited. The contributions of the proposed methodology in architectural visualisation approach are as follows: 1. Enhanced Visualisation: AI models can create vibrant and realistic visualisations from textual descriptions, aiding architects, clients, and stakeholders in better comprehending and conveying design concepts. 2. Efficiency and Iteration: AI-generated visualisations expedite design iteration, providing architects with instant feedback on changes for faster decision-making. 3. Innovation and Creativity: AI-generated images can inspire architects by offering unique visualisations that spark creativity and innovation. As AI technology advances, Image Gen AI’s potential to revolutionize architectural visualisation is significant. It’s poised to become a crucial tool for bridging the gap between text-based ideas and lifelike visual representations in architecture. This integration aligns with the overarching trend of AI enhancing creativity across diverse industries. While this study has focused on four scenarios, the methodology can be expanded to accommodate various requirements, such as 2D images like floor plans, light analysis simulation images, and indoor 360-degree panoramic images, thus exploring the applicability and performance in a broader range of architectural contexts. Furthermore, in the subsequent stages of the research, the future goal is to develop user-friendly applications that offer real and experiential usage, thereby enhancing the potential of AI-supported visualisation in architectural design.

9 How to Enhance Architectural Visualisation Using Image Gen AI Table 9.12 The results of image generation for architects’ design style operation Input SANAA + Luis Barragan

SANAA + Shigeru Ban

Antoni Gaudi + Frank Lloyd Wright

Frank Gehry + Shigeru Ban

Frank Gehry + Tadao Ando

Zaha Hadid + Shigeru Ban

SANAA – Zaha Hadid

Louis Kahn – Tadao Ando

Luis Kahn (Antoni Gaudi)

Luis Barragan (Frank Gehry – Zaha Hadid) SANAA (Zaha Hadid + Luis Barragan)

Output

171

172

J.-K. Lee et al.

9.5 Conclusion This research proposes a novel approach to various visualisations within the field of architecture by leveraging image gen AI. While the scope of empirical validation in this study was limited to reflecting famous architects’ design styles in residential design, various interior spaces reflecting different design styles, bathroom remodelling for the elderly, and commercial building facade design reflecting regional identity, the results of this study hold significance in validating the potential of image gen AI as a visualisation tool in the field of architecture. Although AI applications in the field of architecture are in their early research stages, the emerging topic of generative AI is expected to have a substantial impact within the architectural domain. In this study, we have discussed the direction of utilizing generative AI technology in the field of architecture at the current technological level. Future research will concentrate on developing practical, user-friendly applications for real-world implementation. Acknowledgements This work was supported in 2023 by the Korea Agency for Infrastructure Technology Advancement (KAIA) grant funded by the Ministry of Land, Infrastructure and Transport (Grant RS-2021-KA163269). This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MIST) (No. NRF-2022R1A2C1093310).

References Borden I (2007) Imaging architecture: the uses of photography in the practice of architectural history. J Archit 12(1):57–77. https://doi.org/10.1080/13602360701217989 Chiu M-L (1995) Collaborative design in CAAD studios: shared ideas, resources, and representations. In: Proceedings of international conference on CAAD future, pp 749–759 Farsäter K, Olander S (2019) Early decision-making for school building renovation. Facilities 37(13/ 14):981–994. https://doi.org/10.1108/F-10-2017-0102 Hu EJ, Shen Y, Wallis P, Allen-Zhu Z, Li Y, Wang S, Wang L, Chen W (2021) Lora: low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685. https://doi.org/10.48550/ arXiv.2106.09685 Kalay YE (2004) Architecture’s new media: principles, theories, and methods of computer-aided design. MIT press Kim J-S, Lee J-K (2020) Stochastic detection of interior design styles using a deep-learning model for reference images. Appl Sci 10(20):7299. https://doi.org/10.3390/app10207299 Kim J-S, Choi J-S, Lee J-K (2019) Approach to design reference management using auto-recognition system of room and design style. Int J Eng Technol (UAE) 8(1.4):56–64. https://doi.org/10. 14419/ijet.v8i1.4.25133 Lee H, Shin J, Lee J-K (2016) BIM-enabled definition of a path object and its properties to evaluate building circulation using numerical data. J Asian Archit Build Eng 15(3):425–432. https://doi. org/10.3130/jaabe.15.425 Lee J-K, Shin J, Lee Y (2020) Circulation analysis of design alternatives for elderly housing unit allocation using building information modelling-enabled indoor walkability index. Indoor Built Environ 29(3):355–371. https://doi.org/10.1177/1420326X18763892

9 How to Enhance Architectural Visualisation Using Image Gen AI

173

Merkel J (2008) SANAA’s new museum of contemporary art, New York. Archit Des 78(3):98–101. https://doi.org/10.1002/ad.684 Oppenlaender J (2022) The creativity of text-to-image generation. In: Proceedings of the 25th international academic mindtrek conference, pp 192–202 Phare DM, Gu N, Ostwald M (2018) Representation in design communication: meaning-making in a collective context. Front Built Environ 4:36. https://doi.org/10.3389/fbuil.2018.00036 Ramesh A, Pavlov M, Goh G, Gray S, Voss C, Radford A, Sutskever I (2021) Zero-shot text-to-image generation. In: International conference on machine learning. PMLR 139:8821–8831 Ramesh A, Dhariwal P, Nichol A, Chu C, Chen M (2022) Hierarchical text-conditional image generation with clip latents 1(2):3. arXiv preprint arXiv:2204.06125 Rombach R, Blattmann A, Lorenz D, Esser P, Ommer B (2022) High-resolution image synthesis with latent diffusion models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 10684–10695 Saharia C, Chan W, Saxena S, Li L, Whang J, Denton EL, Ghasemipour K, Lopes G, Raphael KA, Burcu S, Tim H, Jonathan F, David J, Norouzi M (2022) Photorealistic text-to-image diffusion models with deep language understanding. Adv Neural Inf Process Syst 35:36479–36494 Sharifi Noorian S, Qiu S, Psyllidis A, Bozzon A, Houben G-J (2020) Detecting, classifying, and mapping retail storefronts using street-level imagery. In: Proceedings of the 2020 international conference on multimedia retrieval, pp 495–501 Shin J, Lee J-K (2019) Indoor walkability index: BIM-enabled approach to quantifying building circulation. Autom Constr 106:102845. https://doi.org/10.1016/j.autcon.2019.102845 Sun C, Zhou Y, Han Y (2022) Automatic generation of architecture facade for historical urban renovation using generative adversarial network. Build Environ 212:108781. https://doi.org/10. 1016/j.buildenv.2022.108781 Vandenbulcke B (2013) Concretion, abstraction: the place of design processes in today architecture practice. Case study: Sanaa. International conference on architecture and urban design Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. Advances in neural information processing systems, p 30 Vimpari V, Kultima A, Hämäläinen P, Guckelsberger C (2023) “An adapt-or-die type of situation”: perception, adoption, and use of text-to-image-generation AI by game industry professionals. arXiv preprint arXiv:2302.12601. https://doi.org/10.48550/arXiv.2302.12601

Chapter 10

Development of Collective Intelligence for Building Energy Efficiency Peichun Xiao, Lan Ding , and Deo Prasad

Abstract Most studies of energy efficiency in buildings have focused on technical systems and lacked considerations on cooperative solutions by technical systems and occupant behaviour. This chapter presents an innovative collective intelligence model to fill this research gap, where occupants and building energy systems are represented using a society of intelligent agents and Genetic Algorithm (GA) is integrated with it to enhance self-organization, optimisation and cooperation of energy systems actions and occupant behaviour. The utilisation of the collective intelligence model is demonstrated using an apartment building in Australia. The results show that building energy efficiency can be significantly improved through the collective intelligence model. This approach links humans and collective intelligence with building energy systems to tackle the building energy efficiency problem. This chapter advances cross-disciplinary knowledge about the development of artificial intelligence technologies for maximising the potential of energy efficiency in buildings. Keywords Collective intelligence · Building energy efficiency · Multi-agent model · Genetic algorithm (GA)

10.1 Introduction Energy operation in buildings accounts for 30% of global energy consumption (IEA 2023), therefore, it is a key challenge for the building sector to improve energy efficiency while the world has a net zero emissions target by 2050. Most studies of building energy efficiency have focused on technical systems and lacked considerations on cooperative solutions by technical systems and occupant behaviour. P. Xiao Country Garden Property Development, Foshan, Guangdong, China L. Ding (B) · D. Prasad School of Built Environment, The University of New South Wales, Sydney, NSW 2052, Australia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 J. H. Lee et al. (eds.), Multimodality in Architecture, https://doi.org/10.1007/978-3-031-49511-3_10

175

176

P. Xiao et al.

Building energy efficiency refers to energy savings while achieving the standard level of occupant thermal comfort. Multi-objectives are usually defined for building energy efficiency including reduction in energy demand (e.g., heating and cooling demand, energy use in lighting, hot water and plug-in appliances), occupant thermal comfort, and costs. Building energy efficiency is affected by many factors, which include not only technical systems such as construction types and energy systems, but also occupant behaviour and complex interactions among them (Xiao et al. 2014; Xiao 2018). For example, building energy efficiency could be affected by interacting actions and behaviour across heating and cooling system, control of shading system, and spatial activities of occupants. Occupant behaviour makes a significant contribution to building energy efficiency (Samaratunga 2021). Occupant behaviour is often a combination of individual actions, and family and social actions, influenced by many factors including attitudes, preferences, the local climate, and energy related policies and regulations (Samaratunga 2021). Most of existing studies of building energy efficiency have dealt with building envelopes, insulations, heating, cooling and ventilation (HVAC) systems, renewable energy systems, etc., but lacked the research into how cooperative actions and behaviour across occupants and technical systems can achieve the optimal energy efficiency in buildings. Two research questions are addressed in this chapter through the development of a collective intelligence model: (1) how can energy systems and occupants achieve optimal energy efficiency through cooperative actions and behaviour? (2) how can collective intelligence be developed to enable such cooperative actions and behaviour across energy systems and occupants to optimise building energy efficiency? This chapter aims to develop an innovative collective intelligence model to fill this research gap, where occupants and building energy systems are represented using a society of intelligent agents and GA is integrated with it to enhance selforganization, optimisation and cooperation of energy systems actions and occupant behaviour. The utilisation of this model is demonstrated using a residential building as an example. The remaining of this chapter is organised as follows: Sect. 10.2 introduces a theoretical background to establish the model, which includes collective intelligence, multi-agent modelling and GA; Sect. 10.3 presents the methodology for the development of a conceptual framework of the collective intelligence model for building energy efficiency; Sect. 10.4 presents prototype implementation of the model and analysis results; Sect. 10.5 discusses the optimal solutions generated from the model, limitations and future study; and Sect. 10.6 provides an overall conclusion.

10.2 Background Collective intelligence is a key topic in Complex Systems Science, which can be developed for managing complexity in the built environment. It exhibits emergence properties through self-organization. Collective intelligence often consists of individuals with intelligence that has the ability of perception, cooperation, reasoning

10 Development of Collective Intelligence for Building Energy Efficiency

177

and actions for problem solving. The outcomes of collective intelligence include the emergent collective behaviour resulting from the interactions among individuals (Matari´c 2000; Zhang et al. 2023). Collective intelligence in this chapter refers to the ability of humans (e.g., occupants) and machines (e.g., energy systems) to achieve the optimal building energy efficiency through cooperative actions and behaviour. A modelling approach to collective intelligence is developed for building energy efficiency which integrates multi-agent modelling with Genetic Algorithm (GA). Multi-agent modelling has been developed for dealing with complex challenges in many disciplines. A framework for concept formation of intelligent agents in design was established in 2002 (Gero and Fujii 2000). A muti-agent solution was developed for managing distributed energy systems including agent communication ontology and the agent cooperation strategy in 2013 (Ren et al. 2013). Collective intelligence was regarded as a phenomenon such as aggregating the knowledge or collaboration among individuals with a common goal, which leads to a kind of collective intelligence as complex outputs (Maher et al. 2011). Collective intelligence can emerge from multi-agent cooperation and competition (Chen et al. 2023). The collective intelligence model developed in this chapter is built upon multi-agent cooperation, integrated with Genetic Algorithm (GA) for an evolutionary process. Genetic Algorithm (GA) provides a computational model of the Darwinian principle of evolution. A typical GA consists of the following phases: self-production, competition and selection (Katoch et al. 2021). Self-production is to produce a population of solutions through self-organisation and genetic operations including crossover and mutation. Competition refers to competitions among individual solutions towards defined targets (i.e., fitness functions). Selection is a process of selecting individuals (i.e., parents) from a population of solutions in order to produce the next generation of solutions (offspring). In each generation, two parent chromosomes that map onto high fitness functions are selected for crossover and mutation operations to generate the next generation. After a certain number of generations, it can converge to the chromosomes that represent optimal solutions to the defined problems. GA has been integrated with multi-agent systems for various applications (Goodarzi et al. 2011; Honglin et al. 2018; Yasinthara and Dharshana 2020).

10.3 Methodology This section presents a conceptual framework of a collective intelligence model developed for improving building energy efficiency, which integrates multi-agent and GA to achieve cooperative actions and behaviour among energy systems and occupants. A society of intelligent agents are developed to represent occupants and building energy systems, while GA is integrated into intelligent agents to empower self-organization, coordination and evolution of cooperative actions and behaviour. Optimal solutions will emerge from the interactions and coordination of the society of intelligent agents and evolve over time.

178

P. Xiao et al.

10.3.1 Definitions A building comprises building elements, energy systems including appliances, and occupants. Intelligent agents are defined as autonomous software modules that act on behalf of building energy systems and occupants. They are capable of sensing the environment and taking actions in response to the local and global goals through self-organisation and cooperation. A society of intelligent agents refer to a group of intelligent agents with the social ability in a common or shared environment. They can share information and align with each other about energy-related actions and behaviour. Cooperative solutions represent aggregate contributions from a society of intelligent agents that meet a global goal such as building energy efficiency, which can emerge from the interaction and cooperation of intelligent agents. Therefore, collective intelligence for building energy efficiency is defined as the ability of a society of intelligent agents that represent energy systems and occupants to achieve optimal building energy efficiency through cooperative actions and behaviour. Complex interactions among energy systems and occupants include interactions within energy systems, interactions within occupants, and interactions between energy systems and occupants, as well as their interactions with the building environment.

10.3.2 A Conceptual Framework of the Collective Intelligence Model for Building Energy Efficiency A conceptual framework of the collective intelligence model is illustrated in Fig. 10.1, which consists of: (1) state space of the building environment, (2) a society of intelligent agents developed through intelligent agent modules, (3) the self-organisation and evolution mechanism of intelligent agents developed through the integration of intelligent agents and GA, (4) interactions and cooperation developed through the alignment and aggregation mechanisms, and (5) the optimal cooperative solutions consisting of emergent cooperative actions and behaviour from the society of intelligent agents. The key components of the conceptual framework are explained in detail in the following sub-sections.

10.3.2.1

State Space

The state space stores the data of building components, energy systems and occupants for sharing by the society of intelligent agents. The data can be collected from various sources including rea-time air temperature sensors and electricity metering devices. The state space is described as follows and illustrated with example datasets in Fig. 10.2:

10 Development of Collective Intelligence for Building Energy Efficiency

179

Fig. 10.1 Illustration of a conceptual framework of the collective intelligence model for building energy efficiency

S = {S B , S E , So , S P , C}

(10.1)

where, • • • • • •

S refers to a state space S B refers to building characteristics S E refers to active energy system actions So refers to energy-related occupant behaviour S P refers to energy performance data C refers to building environmental data.

10.3.2.2

A Society of Intelligent Agents

A society of intelligent agents is developed to represent building energy systems and occupants, which are distributed and self-organised, aimed at achieving a global goal which is building energy efficiency. The global goal of building energy efficiency refers to energy savings such as reduction in peak electricity reduction, energy costs, and a standard level of indoor thermal comfort of occupants. Human agents represent occupants which can sense the building environment, receive and send messages to other agents, evolve their behaviour in relation to energy use, take interventions on energy system operations, provide feedback on thermal comfort, and share experience with other human agents (Fig. 10.3). Similar to human agents, energy system agents represent various energy systems in buildings,

180

P. Xiao et al.

Fig. 10.2 Illustration of the state space with example datasets that will be shared by the society of intelligent agents

which can detect the building environment and real-time energy use (Fig. 10.4). Energy system agents can interact with each other and also interact with human agents to obtain feedback on thermal comfort, sense spatial activities of occupants, accept interventions from human agents and so on. Each individual intelligent agent consists of six sub-modules: M-1: sensing the environment, such as the indoor air temperature via sensors; M-2: communicating with other agents with agent messages; M-3: detecting situation changes, such as the change in direction by the façade shading agent to respond to solar heat gain; (4) identifying performance targets and evaluation criteria; (5) conducting an evolutionary process through GA-based self-organisation and optimisation to generate optimal solutions; and (6) providing optimal collaborative actions and behaviour for building energy efficiency (Figs. 10.3 and 10.4). An application example of the society of intelligent agents is presented in Fig. 10.5. The society of intelligent agents is presented in hierarchy consisting of agent organisations (or clusters) and individual human and energy system agents within each organisation. Namely, an agent organisation comprises human agents and energy systems agents in each dwelling, and a group of agent organisations present the intelligent agents in all dwellings within the apartment building which can interact and cooperate with each other to achieve optimal energy efficiency of the entire apartment building.

10 Development of Collective Intelligence for Building Energy Efficiency

181

Fig. 10.3 Illustration of a human agent module

Fig. 10.4 Illustration of an energy system agent module

Interactions and cooperation across intelligent agents are crucial in the collective intelligence model. Cooperation across intelligent agents refers to a process of coordinating and aligning individual agents’ actions and behaviour in response to the building environment to achieve a common goal. Cooperation of intelligent agents is developed through the alignment mechanism and aggregation mechanism which will be presented in the following section.

182

P. Xiao et al.

Fig. 10.5 An application example of a society of intelligent agents in an apartment building

10.3.2.3

Integration of Intelligent Agents and GA

GA is integrated with intelligent agents to enable self-organisation through an evolutionary process, where the alignment mechanism is developed for coordinating selection operations and evaluation criteria in the selection phase to ensure cooperative actions and behaviour to be achieved towards a common goal. The alignment mechanism is built upon the experience of intelligent agents and shared knowledge, and communicates through agent messages. The aggregation mechanism is part of the evolutionary process which ensures the aggregate optimal cooperative solutions to be achieved. Figure 10.6 provides an illustration including: (1) an evolutionary process within each individual intelligent agent that evolves actions and behaviour over time, (2) the coordination across individual intelligent agents by the alignment mechanism that happens in the selection phase, and (3) the aggregation mechanism that enables to aggregate optimal cooperative solutions.

10 Development of Collective Intelligence for Building Energy Efficiency

183

Fig. 10.6 Illustration of the integration of multi-agent and GA including alignment and aggregation mechanisms

10.4 Prototype Implementation and Analysis Results The prototype of the colective intelligence model was implemented basd on the MASON system (Luke et al. 2005). An apartment building in Sydney was employed as an application example for demonstration of the results. The apartment building has 25 stories (Fig. 10.7) and its contrstuction type is double bricks. The apartment building consistds of a mix of two and three-bedroom dwellings, where reverse cycle ducted air conditioning systems are instaed in all dwellings. Four dwellings (denoted as Apt 1, Apt 2, Apt 3 and Apt 4) in this apartment building were selected to demonstrate the application of the collective intelligence model.

10.4.1 Real-Time Monitored Indoor Environments and Energy Use Sensors were installed in the four dwellings in the apartment building to monitor the real-time air temperatures and humidities with an interval of 15 min, while metering devices were installed to monitor the energy use of each energy item in a dwelling. Figures 10.8 and 10.9 present examples of the monitored indoor air temperatures and humidities of the four selected dwellings during a day respectively, while Fig. 10.10 presents the monitored daily energy use of each dwelling in a warm month. They indicate that, the indoor air temperature range was from 23.5 to 30.5 °C in summer,

184

P. Xiao et al.

Fig. 10.7 Application example of an apartment building in Sydney for demonstration of the results

the indoor relative humidity was between 43 and 63%, and the average daily energy consumption of a dwelling was 17.03 kWh.

10.4.2 The Key Variables of Occupant Behaviour and Energy Systems Actions and Their Genetic Representations in GA The key variables of occupant behaviour comprise spatial activities of occupants, schedules for use of appliances, and interventions of occupants on air-conditioning systems in dwellings when needed (Table 10.1). The key variables of energy systems actions include smart management and control of air-conditioning systems and lighting systems (Table 10.2). These key variables are used for the demonstration of the prototype implementation, which can be extended to incorporate a range of comprehensive variables related to occupant behaviour and energy system actions in future. The key variables of occupant behaviour and energy system actions in each dwelling were encoded into initial genes, which then formed the genotypes of the agent organisations. Figure 10.11 presents an example of the genotype string that

10 Development of Collective Intelligence for Building Energy Efficiency

185

Fig. 10.8 Monitored indoor air temperatures of the four dwelings by sensors

Fig. 10.9 Monitored indoor humidities of the four dwellings by sensors

consists of a set of the key variables representing occupant behaviour and energy systems actions. The genetic representaiton enables an evolutionary process to be carried out by the inteligent agents.

186

P. Xiao et al.

Fig. 10.10 The daily energy use of the four dwellings in a warm month monitored by metering devices

10.4.3 Agents Classes The inteligent agents were implemntd using the agent unified modelling language (UML) classes, see an example in Fig. 10.12. The organisation class presents the agent organisation within each dwelling consisting of human and energy system agents. A set of organisaton classes presents all agent organisations in the apartment building. Occupants and energy system agents worked cooperatively through an evoutionay process within each agent organisation. Optimal coorperative solutions then emerged from the aggregate cooperative actions and behaviour of the agent organisations in the apartment building. An evolutionary process of the intelligent agents was implemented through submodules representing GA phases. Figure 10.13 illustrates the GA classes in the intelligent agents. In the evolutionary process, an initial population size was set to be 10 presenting an initial set of human behaviour and energy system actons, and the number of generations was set to be 50 that meant the evolutionary process ended at the generation 50.

10.4.4 Optimal Cooperative Solutions for Building Energy Efficiency The results from the prototype implementation of the collective intelligence model demonstrated that the intelligent agents of occupants and energy systems had the ability to respond to the changing environment, interact with each other and search

10 Development of Collective Intelligence for Building Energy Efficiency

187

Table 10.1 Examples of the key variables of occupant behaviour in the apartment building Occupant behaviour

Key variables

Representation

Value

Planning for space activities

Number of spaces available

n sa

{1, 2, 3, 4, 5}

Start time

tsa

{2–8 pm, 8–10 pm, 10 pm–7 am, 7 am–2 pm}

Duration

dsa

{0.5 h, 1 h, 1.5 h, 2 h, 3 h, 4 h, 5 h}

Thermostat setting

Tts

{21 °C, 22 °C, 23 °C, 24 °C, 25 °C, 26 °C, 27 °C, 28 °C}

Start time

tts

{2–8 pm, 8–10 pm, 10 pm–7 am, 7 am–2 pm}

Duration

dts

{0.5 h, 1 h, 2 h, 3 h, 4 h, 5 h, 6 h, 7 h}

Start time

tcw

{2–8 pm, 8–10 pm, 10 pm–7 am, 7 am–2 pm}

Duration

dcw

{30 min, 40 min, 50 min, 1 h}

tdw

{2–8 pm, 8–10 pm, 10 pm–7 am, 7 am–2 pm}

Duration

ddw

{0.5 h, 1 h, 1.5 h, 2 h}

Start time

thw

{2–8 pm, 8–10 pm, 10 pm–7 am, 7 am–2 pm}

Duration

dhw

{5 min, 10 min, 15 min, 20 min}

Spaces

sw

{1, 2, 3, 4, 5,6}

Open or closed

stw

{0, 1}

Planning for thermostat setting intervention of air-conditioning

Planning for clothes washing

Planning for dish washing Start time

Planning for shower

Window operation

for optimal cooperative solutions. The results indicated that the convergence of optimal solutions occurred around the generation 15 and the peak electricity demand decreased rapidly by the generation 15. Figure 10.14 presents examples of cooperative solutions of the four dwellings at 7:45 am under an indoor air temperature of 27.94 °C. As shown in Fig. 10.14, the energy-efficient occupant behaviour comprised a shower around 21:00, washing clothes at 19:00 and washing dishes at 8:30 am to reduce the peak electricity demand and associated costs. Energy-efficient energy systems actions included the automated set-points of air-conditioning systems to be 24 °C to enable the thermal comfort of occupants. Figure 10.15 illustrates examples of cooperative solutions, where occupant behaviour such as clothes washing was scheduled for the evening time to avoid

188

P. Xiao et al.

Table 10.2 Examples of the key variables of energy systems actions in the apartment building Smart energy system actions

Key variables

Representation

Value

Smart air-conditioning (A/C) system action

Space or zone to run A/C

sac

{1, 2, 3, 4, 5}

Indoor thermal comfort level setting

L tc

{− 3, − 2, − 1, 0, 1, 2, 3}

Start time

tdw

{2–8 pm, 8–10 pm, 10 pm–7 am, 7 am–2 pm}

Duration

dac

{0, 0.5 h, 1 h, 2 h, 3 h, 4 h}

Space

sl

{1, 2, 3, 4, 5}

Lighting level setting

Ll

{0–200 lux, 200–500 lux, 500–1000 lux, 1000–2000 lux}

Start time

tl

{2–8 pm, 8–10 pm, 10 pm–7 am, 7 am–2 pm}

Duration

dl

{0, 0.5 h, 1 h, 2 h, 3 h, 4 h, 5 h, 6 h, 7 h}

Smart lighting system action

Fig. 10.11 Examples of the genotypes that encode the key variables of occupant behaviour and energy systems actions for the agent organisations

10 Development of Collective Intelligence for Building Energy Efficiency

189

Fig. 10.12 Illustration of intelligent agents as UML classes

the peak electricity cost, while energy systems actions enable energy savings while maintaining a comfortable indoor thermal environment for occupant activities. Examples of cooperative air-conditioning system actions in the four dwelling are shown in Fig. 10.16a–d, where actions on the set points of air conditioning systems evolved over time to respond to the changing indoor environment (i.e., a temperature increase to 27 °C) and the feedback on thermal comfort from occupants. The optimal cooperative actions of occupants and energy systems achieved significant energy savings and costs as well as thermal comfort of occupants (Fig. 10.17). Figure 10.18 shows energy savings from the dwellings 1–4 reached 36.2%, 33.6%, 30.8% and 16.7% respectively compared to the conventional energy use in the apartment building without utilisation of the collective intelligence model.

190

P. Xiao et al.

Fig. 10.13 Illustration of the GA classes in an evolutionary process of intelligent agents

10.5 Discussion The results show that the overall optimal energy efficiency of the apartment building was achieved rapidly through the collective intelligence model although the convergence process of each dwelling was different. Different optimal cooperative solutions were achieved for different dwellings towards a common goal that is building energy efficiency covering peak electricity reduction, cost reduction, and thermal comfort of occupants at the same time. As a result, the whole apartment building achieved 29.6% energy savings compared to the conventional energy consumption of the apartment building without application of the collective intelligence model. Although the prototype implementation of the collective intelligence model was demonstrated using an apartment building, it can be applied to other types of buildings

10 Development of Collective Intelligence for Building Energy Efficiency

191

Fig. 10.14 Examples of cooperative solutions of the four dwellings at 7:45 am under an indoor air temperature of 27.94 °C

Fig. 10.15 Examples of cooperative solutions of the four dwellings to reduce peak electricity demand and costs while achieving thermal comfort of occupants

192

P. Xiao et al.

(a)

Dwelling 1

(b) Dwelling 2

(c)

Dwelling 3

(d) Dwelling 4

Fig. 10.16 Optimal set points of air-conditioning systems in dwellings 1–4

(a)

Dwelling 1

(b) Dwelling 2

(c) Dwelling 3

(d) Dwelling 4

Fig. 10.17 Illustration of the convergence of the optimal cooperative solutions to peak electricity reduction in the dwellings 1–4

10 Development of Collective Intelligence for Building Energy Efficiency

193

Fig. 10.18 Energy savings from the dwellings 1–4 reached 36.2%, 33.6%, 30.8% and 16.7% respectively compared to the conventional energy use in the apartment building without utilisation of the collective intelligence model

such as commercial and industrial buildings. Furthermore, occupant behaviour can be extended to incorporate social networks of occupants to allow collective learning.

10.5.1 Limitations The collective intelligence model has integrated a society of intelligent agents and GA to achieve optimal cooperative solutions for building energy efficiency. However, learning functions have not been included in the model. Advanced multi-agent reinforcement learning methods have been developed in recent years to resolve complex domain problems (Mei et al. 2023; Zhang et al. 2022). Learning from the past experience of occupants and energy systems and social networks of occupants can further accelerate the process and outcomes of building energy efficiency. The prototype implementation demonstrated exemplar interactions across occupants’ behaviours and energy system actions. Complex interactions across occupants and energy systems can be further modelled in future work.

10.5.2 Future Study Future work can incorporate the collective learning mechanism in the collective intelligence model. Collective learning can shape human cognition that can be affected by social networks and collective behaviour (Momennejad 2021). The development

194

P. Xiao et al.

Fig. 10.19 Example of a cross-reality environment to bridge a 3D virtual world and a real world to empower collective intelligence

of the collective learning mechanism will enable intelligent agents to learn from each other through human social networks including social medias about the successful experience to accelerate the evolutionary process of intelligent agents. A cross-reality environment (Ziker et al. 2021) or a cyber-physical-social system (Gong et al. 2022) can be developed to empower collective intelligence in the built environment, for example, it bridges a 3D collaborative virtual environment and a real world through networked sensors. A virtual social environment (Gu and Maher 2014) or other extended realty environments (Casini 2022) will allow participants from different spaces to communicate and learn from each other the energy-efficient behaviour and respond to energy system actions that happen in the real-world. Such a cross-reality environment will enhance collective intelligence across humans and technical systems (Fig. 10.19).

10.6 Conclusion This chapter presents an innovative collective intelligence model for building energy efficiency. GA is integrated with intelligent agents to enable self-organisation and evolution of energy systems actions and occupant behaviour. The emergence of the optimal cooperative solutions that achieve energy efficiency targets was demonstrated

10 Development of Collective Intelligence for Building Energy Efficiency

195

using an apartment building in Sydney. The results indicate that the innovative collective intelligence model creates new forms of cooperation of building energy systems and people for improving building energy efficiency which is a key challenge to reach net zero carbon. This chapter contributes to the development of cross-disciplinary knowledge about the development of artificial intelligence technologies for maximising the potential of energy efficiency in buildings. It demonstrates both theoretical implications and practical application of a collective intelligence model in a built environment. It also provides possible directions for future work including collective learning through human social networks including social medias and an advanced cross-reality environment to empower collective intelligence across humans and technical systems to achieve net zero carbon outcomes in buildings.

References Casini M (2022) Extended reality for smart building operation and maintenance: a review. Energies 15(10):3785. https://doi.org/10.3390/en15103785 Chen H, Tao S, Chen J, Shen W, Li X, Yu C, Cheng S, Shu X, Li X (2023) Emergent collective intelligence from massive-agent cooperation and competition. arXiv preprint arXiv:2301.01609 Gero JS, Fujii H (2000) A computational framework for concept formation for a situated design agent. Knowl-Based Syst 13(6):361–368 Gong K, Yang J, Wang X, Jiang C, Xiong Z, Zhang M, Guo M, Lv R, Wang S, Zhang S (2022) Comprehensive review of modeling, structure, and integration techniques of smart buildings in the cyber-physical-social system. Front Energy 16:74–94. https://doi.org/10.1007/s11708-0210792-6 Goodarzi M, Radmand A, Nazemi E (2011) An optimized solution for multi-agent coordination using integrated GA-fuzzy approach in rescue simulation environment. In: Bai Q, Fukuta N (eds) Advances in practical multi-agent systems. Springer, Berlin, Heidelberg, pp 377–388 Gu N, Maher ML (2014) Designing adaptive virtual worlds. De Gruyter Open Honglin B, Qiqige W, Wolfgang B (2018) Evolution of cooperation through genetic collective learning and imitation in multiagent societies. In: Proceedings of the ALIFE 2018: the 2018 conference on artificial life, Tokyo, Japan, pp 436–443 IEA (2023) Tracking clean energy progress 2023. IEA, Paris. https://www.iea.org/reports/trackingclean-energy-progress-2023 Katoch S, Chauhan SS, Kumar V (2021) A review on genetic algorithm: past, present, and future. Multimed Tools Appl 80(5):8091–8126. https://doi.org/10.1007/s11042-020-10139-6 Luke S, Cioffi-Revilla C, Panait L, Sullivan K (2005) MASON: a new multi-agent simulation toolkit. In: Proceedings of the 2004 swarmfest workshop Maher ML, Paulini M, Murty P (2011) Scang up: from individual design to collaborative degin to collective design. In: Proceedings of design computing and cognition’10, pp 581–599 Matari´c MJ (2000) From local interactions to collective intelligence. In: Cruse H, Dean J, Ritter H (eds) Prerational intelligence: adaptive behavior and intelligent systems without symbols and logic, Volume 1, Volume 2 prerational intelligence: interdisciplinary perspectives on the behavior of natural and artificial systems, Volume 3. Studies in cognitive systems, vol 26. Springer, Dordrecht. https://doi.org/10.1007/978-94-010-0870-9_61 Mei Y, Zhou H, Lan T, Venkataramani G, Wei P (2023) MAC-PO: multi-agent experience replay via collective priority optimizationar. arXiv:2302.10418v2 [cs.LG]

196

P. Xiao et al.

Momennejad I (2021) Collective minds: social network topology shapes collective cognition. Phil Trans r Soc B 377:2020031520200315. https://doi.org/10.1098/rstb.2020.0315 Ren F, Zhang M, Sutanto D (2013) A multi-agent solution to distribution system management by considering distributed generators. IEEE Trans Power Syst 28(2):1442–1451 Samaratunga M (2021) Development of an evidence-based post-occupancy behaviour framework for energy consumption in BASIX-compliant dwellings in Sydney, Australa. PhD thesis, UNSW Sydney Xiao P (2018) Development of collective intelligence for building energy efficiency. PhD thesis, UNSW Sydney Xiao P, Ding L, Prasad D (2014) Modelling adaptive building energy systems and human behavior: an agent-based modelling approach. In: Proceedings of grand renewable energy 2014, Tokyo, Japan Yasinthara M, Dharshana K (2020) Incorporating strategy adoption into genetic algorithm enabled multi-agent systems. In: 2020 IEEE congress on evolutionary computation (CEC). IEEE Press, pp 1–8. https://doi.org/10.1109/CEC48606.2020.9185502 Zhang B, Hu W, Ghias AMYM, Xu X, Chen Z (2022) Multi-agent deep reinforcement learningbased coordination control for grid-aware multi-buildings. Appl Energy 328:120215 Zhang J, Qu Q, Chen XB (2023) A review on collective behavior modeling and simulation: building a link between cognitive psychology and physical action. Appl Intell. https://doi.org/10.1007/ s10489-023-04924-7 Ziker C, Truman B, Dodds H (2021) Cross reality (XR): challenges and opportunities across the spectrum. In: Ryoo J, Winkelmann K (eds) Innovative learning environments in STEM higher education. SpringerBriefs in statistics. Springer, Cham. https://doi.org/10.1007/978-3-030-589 48-6_4

Part III

Education

Chapter 11

Multimodality and Architectural Education Mi Jeong Kim , Ju Hyun Lee , and Michael J. Ostwald

Abstract Advances in information and communication technologies have been the catalyst for substantial transformations in the field of architecture and in design education. In response to the pandemic, architectural education rapidly embraced digital design techniques, incorporating multimodal tools like virtual reality, online collaborative platforms, design management systems, and even social networking services (SNSs). In this context, this chapter introduces and explores the multifaceted perspectives of architectural education in the digital era. Specifically, in Part III of this book, multimodal architectural education is examined in conjunction with a new teaching model, intelligent design evaluation, computer-mediated design education, and a theoretical discourse. This chapter introduces Part III and provides an overview of multimodality in architectural education. Two multimodality models introduced in this book, namely the ‘modality–functionality’ and ‘modality–efficacy’ models, are then used to further discuss the contributions in Part III. Keywords Architectural education · Design education · Digital design · Multimodality

11.1 Introduction In recent years, architectural education has embraced digital design environments for teaching and learning, and the use of such environments continues to evolve in response to the changing landscape of information and communication technology (ICT). Consequently, the way teachers and students communicate and interact has evolved to align with the changes in both social and technical aspects (Kim et al. M. J. Kim (B) School of Architecture, Hanyang University, 222 Wangsimni-Ro, Seongdong-Gu, Seoul 04763, Republic of Korea e-mail: [email protected] J. H. Lee · M. J. Ostwald School of Built Environment, Faculty of Arts, Design and Architecture, The University of New South Wales, Sydney, NSW 2052, Australia © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 J. H. Lee et al. (eds.), Multimodality in Architecture, https://doi.org/10.1007/978-3-031-49511-3_11

199

200

M. J. Kim et al.

2012). Furthermore, during the COVID-19 pandemic, teachers and instructors were forced to adapt, by transitioning their traditional physical classrooms, either entirely or partially, into virtual learning environments, marking a notable shift in architectural education (Cho et al. 2023; Lee et al. 2023). Part III of this book explores these multimodal approaches in architectural education, focusing on a new teaching model, intelligent design evaluation, computer-mediated design education, and theoretical discourse. Every model, method, platform, and theory in education comes with its own set of advantages and disadvantages. For example, live online classes enable synchronous interaction between teachers and students, enhancing a sense of social presence. However, they are susceptible to technical glitches that can disrupt the learning experience. In contrast, pre-recorded video lectures offer flexibility, allowing students to access resources at their own pace, but they lack immediate feedback and spontaneity. Such advantages and disadvantages should be considered when designing and implementing educational strategies and systems to ensure effective and meaningful learning experiences for students. In this regard, Part III of this book addresses ‘multimodality’ and technology adoption in teaching and learning, emphasising its relevance to architectural education. The first contribution presented in Part III demonstrates that incorporating computing into design education can stimulate students’ innovative thinking in the design process. The flexibility of a virtual platform plays a pivotal role in facilitating this learning process by enabling the sharing of online information sources among students. The second contribution introduces an innovative approach that combines advanced vision technology with machine learning to predict students’ perceived stress levels in university campus design. This intelligent design evaluation underscores the potential of artificial intelligence (AI) in shaping educational environments. The third research contribution examines a promising application for collaborative work. An empirical study highlights how the multimodality of digital tools like Miro can enhance both the sense of social presence and spatial presence, fostering a stronger sense of a “learning community”. The final contribution explores the potential of pattern languages for design education and practice. This research is based on a qualitative review of a famous architectural theory, along with its past applications in a range of fields. The chapter begins with an overview of multimodality in architectural education. It then proceeds to introduce the four research contributions in Part III of the book. Finally, this chapter concludes by discussing the contributions in the context of the modality–functionality model and modality–efficacy model presented in Chaps. 1 and 6, respectively.

11 Multimodality and Architectural Education

201

11.2 Multimodality in Architectural Education Research has explored the impact of digital technologies on people’s thought processes and design activities in higher education settings. Some of this research is situated within the domain of design cognition, a field that deals with topics like “tool neutrality” and potential cognitive side effects associated with these technologies. In the concept of tool neutrality, technology is considered separate from cognitive processes, and digital tools or technologies are viewed as merely another type of writing or drawing instrument, similar to a pencil. However, Cross (2006) suggests that the multimodality of digital technologies affects human cognition and social interaction. Kim and Maher (2008), for example, provide evidence that tangible user interfaces, compared to graphical user interfaces, can have a significant impact on designers’ spatial cognition and contribute to a more creative design process. As such, the adoption of multimodal design technologies has the potential to foster effective education that enhances students’ capacity for design thinking. From a practical standpoint, the architectural design process has seen substantial changes since the 1980s, largely due to the transition to Computer-Aided Design (CAD) (Gero 1983). The advancement in CAD-related technology has paved the way for a variety of innovative digital design approaches (Lee et al. 2023). Similarly, digital technologies such as mobile computing, social media, the Internet of Things (IoT), and big data have introduced novel approaches to business processes (Markus and Loebbecke 2013). Collaborative conceptual design in the digital age also presents significant challenges and opportunities—e.g., collaborative conceptual design modelling and data sharing, product-centric design methodology, knowledge management, distributed design project management, and implementation of virtual design studios (Wang et al. 2002). Furthermore, it is “supported by artificial intelligence, and fuelled by information technologies” (Wang et al. 2002, p. 981). In this context, design education should be grounded in digital design thinking, which is “non-typological and non-deterministic in supporting and preferring the discrete and differentiated over the generic and the typological” (Oxman 2006, p. 262). Furthermore, the combined terminology of “digital” and “thinking” naturally evokes multimodality in the context of technology and cognition. That is, it not only encompasses a wide range of technologies, including computers, smartphones, software, and the internet, but also pertains to the cognitive processes that occur in the human mind, including reasoning, problem-solving, decision-making, creativity, and more. Furthermore, this intersection highlights the diverse ways in which digital tools can support and shape human cognition, leading to a multimodal approach to design thinking. As such, Part III of this book focuses on digital design thinking processes in education settings.

202

M. J. Kim et al.

11.2.1 The IC-PBL Model on a Virtual Platform The Industry Coupled Problem-Based Learning (IC-PBL) model emphasises the relationship between academia and industry so that the focus is the enhancement of students’ problem-solving in a real context. In “Urban and Architectural Design Education Using the IC-PBL Model on a Virtual Platform”, Yang and Kim focus on the potential of design computing curricula combined with the IC-PBL model in architectural design education. To demonstrate this potential, they applied the model in design computing classes on a virtual platform in a graduate course. They collected observation data from the classes and analysed it qualitatively to understand students’ problem-solving processes. The starting assumption was that integrating design computing into urban and architectural design would improve students’ capability of multimodality in design practice. The scenario given in the classes was the development of a design for a universal city for vulnerable people, using geospatial information from a design computing perspective. Three sub-scenarios were further developed: (i) organisation of a list of geospatial information by the types of vulnerable groups, (ii) collection of site-based geospatial information on a selected test bed and (iii) development of alternatives for geospatial information services using smart technologies. In this context, students were required to act as employees in a spatial information company and develop smart solutions using spatial information as a novel approach to the universal city design. In their research, Yang and Kim demonstrate that design computing using the ICPBL model can catalyse a synergistic response, improving students’ abilities to solve real-world problems. The problem-solving process, emphasising user experiences, allowed students to apply multimodality in the built environment within urban and social contexts, leading to an empirical approach to architectural environments. The virtual platform supported flexible interactions among students and instructors in the classes since various online sources of information, such as YouTube, websites, and online files, are easily shared. When combined with IC-PBL, it produced better collaboration and computing performance in urban and architectural design education.

11.2.2 An Intelligent Approach for University Campus Design A significant amount of research has investigated the relationships between environments and human perception using labour-intensive research methods such as surveys or interviews. In contrast, Chap. 13 focuses on the computational measurement and prediction of the relationship between visual properties and psychological responses in university campus design. In “The Impact of Visual Character on Perceived Stress Levels: An Intelligent Approach Applied to University Campus Design”, Li, Zu, Lee,

11 Multimodality and Architectural Education

203

and Ostwald identify the lack of investigation into the relationship between the visual character of campus streetscapes and perceived stress levels. They then computationally assess the impact of street view imagery (SVI) properties on perceived stress in campuses. Seven universities in China were selected for the examination of visual and psychological variables in campus streetscapes. The strength of the research lies in its combining of a computer vision technique and a random forest (RF) machine learning algorithm—automatic semantic segmentation using SegNet and human–machine adversarial scoring using RF. SegNet can segment each SVI into visual elements and calculate their visual element proportion (VEP). As for the human–machine adversarial model, 20 undergraduate students scored a random sample of 100 SVIs using a stress perception scale from 1 to 6. Li et al. examined the impact of three visual features—the ‘nature, ‘enclosure’ and ‘connectivity’ of the SVIs—on the perceived stress scores (PSSs) and identified the impacts of visual elements on the PSSs. The research findings suggest that, to reduce students’ perceived stress on campus, designers should focus on VEPs such as trees and sidewalks, and also that this computational approach is an effective method for supporting designers. The research presents quantitative results considering multimodality in visual environments, but there are limitations to the visual complexity of campus streetscapes. SVIs may have inconsistencies in time, season and weather because of collection method. The human–machine adversarial model also needs to be further validated using more cases in other regions to generalise the results. Non-geometric design elements, such as colour and seasonality, must be considered in future research.

11.2.3 Computer-Mediated Education Environments Computer-mediated environments have been widely adopted in schools and universities to provide students with more effective educational experiences. During the COVID-19 pandemic, online learning environments, mediated by computing technologies became an essential aspect of architecture education. Chap. 14 explores the potential of computer-mediated environments for online classes to support students’ learning experiences and improve problem-solving skills in design studios. In “Design Education and Learner Experiences in a Computer-Mediated Environment”, Cho and Kim present the results of two case studies, one of which investigates students’ experiences of online classes during the pandemic through a questionnaire survey; the other is an experimental study exploring students’ design process in the computer-mediated platform Miro. The first study identified problems associated with classes using a learning management system (LMS), by analysing architecture students’ learning experiences (Cho et al. 2023). LMSs are a widely adopted platform used in domestic universities to provide management functions such as the sharing of lecture content and management of student attendance. To investigate students’ learning experience, Cho and Kim used the Community of Inquiry (CoI) framework consisting of

204

M. J. Kim et al.

teaching, social, cognitive presence and spatial presence. The CoI is a valid metric for measuring educational experience in an online environment. The results show that the low social and spatial presence in the LMS resulted in few social exchanges among teachers and students, leading to a reduced or limited sense of community. According to this first study, a more strategic approach to computer-mediated environments is necessary for building a community of learners in a cooperative learning environment. In their second study, Cho and Kim explored web-based tools for design education focusing on multimodality and selected Miro as a mediated environment for the experiment. The results show that rich multimodality in a computer-mediated environment can build the learners’ community by enhancing social and spatial presences. Miro enables a fluid transition among various electronic tools and flexible interaction for remote team members. Further, it enables a space-based collaboration asynchronously by leaving comments through the “post-it” function, thereby supporting a sense of spatial presence.

11.2.4 Pattern Languages and Design Education Relatively little research has developed new pattern languages in the architecture domain in comparison with computing and programming areas. The focus of Chap. 15 is Christopher Alexander’s pattern language. In “Pattern Languages: Concepts and Applications in Design for Ageing and Design Education”, Dawes emphasises the potential of pattern language in architectural practice and education. The flexible features of the pattern languages allow users to extend languages by sharing sociospatial information and research insights. To explore the potential of the new applications of architectural pattern language, he conducted a qualitative and non-exhaustive literature review. The review identified the strengths and weaknesses of new pattern languages. Developing new patterns requires iterative development focusing on adaptability, and patterns need to be published in order to be used collaboratively. By selecting pattern content carefully, the impact of the patterns can be increased when implemented. Dawes presents the application of new pattern languages in architectural education, where concise pattern formats can provide students with a holistic understanding of the design process, including problem framing and reformulation. He proposes that patterns can be used in designing guidelines and principles for an ageing population since patterns represent generic solutions to frequent problems in the built environment and link multiple solutions. The significance of this chapter lies in its discussion of the potential of the concept of pattern languages in the context of the built environment, which identifies their strengths, weaknesses, and potential lessons. Pattern languages are an effective strategy for adopting multimodality in our built environment because they can provide generic solutions to given problems in terms of technological aspects.

11 Multimodality and Architectural Education

205

New pattern languages can be associated with computer programming for the implementation of the built environments. Further, by incorporating the pattern languages concept and theory in design education, students can be trained to consider various multimodal approaches to designing the built environment.

11.3 Discussion Digital design thinking processes and their multimodalities in the four contributions of Part III can be characterised and examined by both the modality–functionality model in Chap. 1 and the modality–efficacy model in Chap. 6. While the two models in Fig. 11.1 primarily address collaboration and technology, respectively, they can both illustrate the multifaceted capabilities and effectiveness of digitised design education. Furthermore, the multimodality of digital technologies refers to their ability to present information and interact with users through multiple modes and channels. These technologies have significantly impacted various aspects of human cognition, social interaction, and education. As illustrated in the left chart of Fig. 11.2, Yang’s and Kim’s IC-PBL model on a virtual platform highlights that digital technologies allow for interactive and immersive learning experiences through virtual reality (VR), augmented reality (AR), 3D modelling, mobile and simulation applications, and even big data. Thus, it enables “opportunistic” creation and decision that is rarely considered in the modality–functionality model (Fig. 11.1a). Students, in addition to numerous interactions with instructors and industry experts (‘diversity’ collaborators), can virtually explore architectural designs which enhances their understanding of geospatial relationships and design concepts. If society becomes involved in the education system, as described in the model, individuals can gain access to various online platforms, video conferencing, and collaborative design software that enable ‘anytime global’ collaboration and sharing of ideas, fostering a more multimodal learning environment. Digital tools also help in integrating environmental and human perceptions into architectural design processes. Simulation and analysis software can assess the environmental impact of designs, encouraging sustainable practices from the early stages of education. As shown in Fig. 11.2b, Li et al.’s human–machine adversarial model involves “participatory design” as well as AI collaboration. Furthermore, it can be expanded to incorporate a digital platform that that facilitates ‘anytime local’ collaboration during the early design phase. In such a scenario, AI can function as a multitude of public participants for design evaluation, and the outcomes of this collaborative effort are employed for future design creation. If it is integrated into a design-decision platform, the model can be expanded to achieve a higher level of multimodality by enabling ‘anytime global’ collaboration and facilitating ‘collective decision’. The modality–efficacy model in Fig. 11.1b provides a way of visualising and tracking the structure of a system in terms of its input modes, their cumulative effectiveness, and barriers to achieving the desired outputs. The model is represented

206

M. J. Kim et al.

Fig. 11.1 The modality–functionality model and the modality–efficacy model

graphically with an x-axis for efficacy and a y-axis for modality, and where a correlation between the two—growth in x potentially supports growth in y—is a possible consequence of a well-designed and implemented system. In the model, modes are numbered (1, + 2, + 3, + 4, …), increasing levels of efficacy are alphabetical (a → b → c → d …), and barriers are located relative to the modes or outcomes they affect. The two main types of barriers are social (S) and technical (T ), although other types are also possible. Importantly, the model accommodates the identification of limits, as neither modality nor efficacy is infinitely extendable. The inclusion of too many modes may lead to information overload while effectiveness at higher levels may be difficult to conceptualise or measure. Thus, some of the core lessons of the

11 Multimodality and Architectural Education

207

Fig. 11.2 The multimodalities of the IC-PBL Model and the human–machine adversarial model

modality–efficacy model are that the relationship between the two (x, y) parameters must be critically tested, and there are limits to both. As an example of the model in an educational context, the verbal content of a lecture (1), may be coupled with visual content (2) and an interactive sketching exercise (3), potentially raising the pedagogical efficacy of the system from a low level (a, awareness of a concept) to a higher level (c, a capacity to employ a concept). However, if we imagine that the interactive sketching mode is supported by a software platform, then there may be barriers to people using it. These could, for example, arise from network bandwidth limitations (T ) or software accessibility and affordability (S). In this way, multimodal systems can be thought of as comprising inputs (1, + 2, + 3, + 4, …), outputs (a → b → c → d …) and operations (overcoming S and T ). The modality–efficacy model provides a way of conceptualising Cho’s and Kim’s research in Part III about learner experiences in computer-mediated environments. In that research, Cho and Kim analysed students’ perceptions of the effectiveness of an online LMS platform (1), to which they added three additional modes: video conferencing (2), social networking (3), and online access (4). Barriers to the efficacy of the system were mostly social, although underpinned by technology, such as a reduced capacity for interaction and information exchange. Ultimately, students felt that a cooperative learning environment was not created using these modes. Cho’s and Kim’s research shows why the relationship implied in the modality–efficacy model must be tested, as the assumption that more modes would lead to more success was not supported in this case. As a means of overcoming some of the barriers in the original educational system, Cho and Kim examined the use of Miro as a shared workspace with the capacity for simultaneous interaction (5). Indeed, students identified the effectiveness of three modes in Miro, drawing, speaking and spatialisation, with evidence suggesting that it increased (a → c). While Cho and Kim do not argue that Miro can overcome all the social and technical barriers, the two halves of their research can be viewed as complementary from a pedagogical perspective. Thus, a range of barriers inhibited

208

M. J. Kim et al.

Fig. 11.3 Conceptualising the creation of a cooperative learning environment using the modality– efficacy model

the effectiveness of the original system, but some of these can potentially be overcome by adding a digital platform with additional functionality (Fig. 11.3). The central premise of Christopher Alexander’s pattern language is that individual design patterns from a larger language become more “complete”—what he described as “whole” or “alive”—when combined with related patterns in the language. Thus, for example, in the language of domestic architecture, the pattern for the front door (a mode of entry), is improved when coupled with the pattern for a porch (a transition zone) and a canopy (to shelter visitors from the weather). The structure of the language also suggests that the porch pattern will be more effective if the pathway pattern leads to it, and the door pattern will be enhanced by the inclusion of an entry alcove pattern. The greater the number of connected patterns or modes in this system, in theory, the more effective the output. In Part III, Dawes’ chapter explores the origins and applications of design pattern languages before focussing on several aspects of educational design. There are two parallel educational dimensions to Dawes’ reading, the first of which is that pattern languages may be useful educational tools to assist novice designers in conceptualising a building as a rich network of connected decisions. The second is that pattern languages have been developed for designing educational environments. Whereas these uses are valuable, it is Dawes’ critique of the complexity of the pattern language system and its assumptions that is informative in the context of multimodality. If Alexander’s pattern language is treated as a system, then each connected pattern could be considered part of a modal set, with the degree of connectedness contributing to an increased level of composite wholeness. However, every modal set which has no connection to a related mode would detract from the wholeness of the system. This is significant, as it suggests there is both robustness and fragility in the system, depending on the combination of modal sets used. Whereas additional modes are normally treated in the model as either positive or neutral (if barriers limit their effectiveness), in Alexander’s system, they can actively undermine wholeness.

11 Multimodality and Architectural Education

209

Fig. 11.4 Conceptualising the presence of conflicting inputs in a system using the modality–efficacy model

Consider a system where a modal set (11 , 12 , 13 ) collectively enhances wholeness (a → a2 → a3 ), and a second modal set (21 , 22 , 23 ) also shapes wholeness but in a contrary or negative way (−a1 → − a2 → − a3 ). The outcome could be a multimodal system, with negligible net wholeness or efficacy (∅) as positive and negative influences compete. Such an example isn’t about information overload but rather conflicting information undermining the quality of the output (Fig. 11.4). 1

11.4 Conclusion This chapter has explored multimodality in architectural education, providing an overview of the four contributions in Part III of this book. It introduces various approaches to the adoption of digital design thinking in education settings. Indeed, the integration of digital technologies into our daily lives has brought about significant shifts in various fields, including education and architecture. As such, the multimodality of digital technologies in architectural education has revolutionised the way students learn, collaborate, and develop essential skills. It enhances the learning experience by providing interactive, immersive, and personalised opportunities in various digital platforms. Thus, the four contributions in Part III can be discussed in conjunction with of the research in Part I (Collaboration) and Part II (Technology).

210

M. J. Kim et al.

References Cross N (2006) Designerly ways of knowing. Springer, London Cho ME, Lee JH, Kim MJ (2023) Identifying online learning experience of architecture students for a smart education environment. Journal of Asian Architecture and Building Engineering 22(4):1903–1914. https://doi.org/10.1080/13467581.2022.2145216 Gero JS (1983) Computer-aided architectural design—past, present and future. Archit Sci Rev 26(1):2–5. https://doi.org/10.1080/00038628.1983.9697249 Kim MJ, Maher ML (2008) The impact of tangible user interfaces on spatial cognition during collaborative design. Des Stud 29(3):222–253. https://doi.org/10.1016/j.destud.2007.12.006 Kim MJ, Maher ML, Gu N (2012) Mobile and pervasive computing: the future for design collaboration. In: Anumba CJ, Wang X (eds) Mobile and pervasive computing in construction. Wiley-Blackwell, Chichester, pp 169–188 Lee JH, Ostwald MJ, Arasteh S, Oldfield P (2023) BIM-enabled design collaboration processes in remote architectural practice and education in Australia. J Archit Eng 29(1):05022012. https:// doi.org/10.1061/JAEIED.AEENG-1505 Markus ML, Loebbecke C (2013) Commoditized digital processes and business community platforms: new opportunities and challenges for digital business strategies. MIS Q 37(2):649–653 Oxman R (2006) Theory and design in the first digital age. Des Stud 27(3):229–265. https://doi. org/10.1016/j.destud.2005.11.002 Wang L, Shen W, Xie H, Neelamkavil J, Pardasani A (2002) Collaborative conceptual design—state of the art and future trends. Comput Aided Des 34(13):981–996. https://doi.org/10.1016/S00104485(01)00157-9

Chapter 12

Urban and Architectural Design Education Using the IC-PBL Model on a Virtual Platform Hyungmo Yang

and Mi Jeong Kim

Abstract Problem-solving skills have been considered an important aspect of education. For that reason, many areas of education have attempted to apply the problem-based learning model. However, many curricula within urban and architectural design education still approach this using the conventional method of design practice, which commonly focuses on an analysis of the surrounding physical environment rather than design problems. Thus, this chapter explores whether the convergence of design computing curriculum in urban and architectural education and the problem-based learning model can bring positive educational benefits to students. First, the chapter introduces a design computing class that applied the industrycoupled problem-based learning model. This postgraduate course was conducted on a virtual platform in the School of Architecture at Hanyang University, South Korea. Second, this chapter analyses the outcomes of the class in addition to feedback from students. From the analysis of project outcomes and student feedback, this research identifies a potential beneficiary of the convergence of design computing curricula with a problem-solving design approach on a virtual platform for delivering better urban and architectural design education. Finally, this chapter discusses the potential of design computing on a virtual platform using the industry-coupled problem-based learning model for delivering better urban and architectural design education. Keywords Design computing · IC-PBL model · Design education · Virtual platform

H. Yang · M. J. Kim (B) School of Architecture, Hanyang University, 222 Wangsimni-Ro, Seongdong-Gu, Seoul 04763, Republic of Korea e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 J. H. Lee et al. (eds.), Multimodality in Architecture, https://doi.org/10.1007/978-3-031-49511-3_12

211

212

H. Yang and M. J. Kim

12.1 Introduction Past studies have highlighted both design computing and education (Goldman and Zdepski 1987; Marx 2000; Silva 2001). Despite the various purposes of design computing education (Silva 2001), there has been a common feature of design computing—the use and development of computational models and digital media to support various aspects of design activities (Gero 1998). Incorporating design computing into urban and architectural design education is not a new approach. There are two common focuses of design computing education in urban and architectural design curricula worldwide, including Australia, which has been a pioneer in this education area. For example, the School of Architecture, Design and Planning at the University of Sydney in Australia, which was one of the first groups to use the term “design computing,” has operated a design computing studio to improve the ability of students to resolve real-world problems and explore potential, which is enabled by advanced technologies (University of Sydney 2021). However, the University of New South Wales in Australia has developed a curriculum for a communication and visualisation course that uses a synthesis of technical and theoretical knowledge of design computing for students who are majoring in architectural design (University of New South Wales 2023). The two types of curricula allow students to develop additional skills within design computing and beyond for better urban and architectural design (University of New South Wales 2023; University of Sydney 2021). According to Dieter Rams, an industrial designer, a good design can make a useful and understandable output through innovative, aesthetic, durable, and environmentally friendly approaches (Jong et al. 2021). A good design can also support the resolution of social problems (McCann 1983). Given that an efficient approach to resolving social problems resulted from a poor design, problem-solving skills have been frequently discussed. Problem-solving skills are core abilities, specifically for the era of the fourth industrial revolution and therefore have been considered a significant aspect of design education (Oh et al. 2021). To develop problem-solving skills, problem-based learning (PBL) has been considered an effective educational model across academic disciplines (Hmelo-Silver 2004). However, in the field of urban and architectural design education, many curricula still approach this topic using the conventional method of design practice, which commonly focuses on an analysis of the surrounding physical environment. This phenomenon limits the practice of architecture to being empirically engaged with the rapidly changing urban environment from the perspective of human experience. According to Lee and Chung (2008), urban and architectural education engages students in learning by solving structured and unstructured problems, which is a distinct characteristic from other educational areas. Thus, practical architectural education based on a problem-solving approach is essential. The use of a problemsolving-driven curriculum, such as the PBL model for design education, allows students to devise potential design solutions for better human experiences.

12 Urban and Architectural Design Education Using the IC-PBL Model …

213

Moreover, urban and architectural design education is traditionally conducted face-to-face (Lee et al. 2021), which tends to be inevitably accompanied by limitations of time and physical spaces. In terms of this, many universities have discussed the necessity of conducting education on a virtual platform (Cortiz and Silva 2017). However, the integration of online education into higher education systems is still not common although it has recently increased because of the COVID-19 pandemic (Cho et al. 2022). This chapter focuses on the case of a design computing class on a virtual platform that applied the industry-coupled problem-based learning (IC-PBL) model. The class was conducted as a graduate class in the School of Architecture at Hanyang University, South Korea. The IC-BPL is one of the educational strategies that was developed by Hanyang University to achieve positive educational results. Through this, students explore context-rich problems that occur in real-life fields partnered with industry and society. This chapter explores the potential of the convergence of design computing and the IC-PBL model in the architectural design curriculum and the results that would be led by that curriculum. For this, the chapter first draws problem-based learning (i.e., the PBL model and the IC-PBL model). The chapter then introduces the design computing class that applied the IC-PBL model for those majoring in urban and architectural design, which was conducted on a virtual platform. The chapter then analyses the class outcomes to explore the potential of a problem-solving approach to deliver a better urban and architectural design education curriculum. Finally, this chapter discusses the potential of design computing on a virtual platform through the IC-PBL model for delivering better urban and architectural design education.

12.2 Problem-Based Learning 12.2.1 PBL Model PBL is an educational method that allows students to improve skills for resolving problems that are likely to occur in the real world. PBL was initially developed for medical education at McMaster Medical School in Canada in the early 1970s (Barrows 1996). However, since then, PBL has been considered an effective strategy in other educational fields (Hmelo-Silver 2004) because of the significance of the process of understanding the problems within the broader context. PBL specifically deals with real-life issues instead of knowledge acquired from a lecture in a class (Barrows 1985). Thus, the key in PBL is designing problems well (Duch 2001; Hung 2006; Weiss 2003). In terms of this, Duch (2001) stressed that the interest from students, complexity, and open-ended are important characteristics of PBL problems. Weiss (2003) specifically suggested criteria for a problem design for PBL, which are “appropriate for students,” “ill structured,” “collaborative,” “authentic,” and “promotes lifelong and self-directed learning.” Thus, PBL

214

H. Yang and M. J. Kim

problems should be designed by considering the ability and knowledge of students (Duch 2001). In addition, to provide a logical solution to a given problem that students could encounter, students need to proactively search for the background and thoroughly understand the problem within the contextual aspect. PBL engages with the practical aspects of life-based experiences, which expands the perspectives of students beyond the theoretical curriculum held in the classroom (Jung and Lee 2021; Oh et al. 2021). Design computing education using a design approach was introduced in the past 37 years (Silva 2001). However, most of the education has been derived from focusing largely on programming (Guzdial 2015; Silva 2001) in which the learner-centered and design-project aspects are not included. In other words, the process of searching for a problem and its reasoning context has been missed, which limits the opportunities for students to engage with a wider perspective toward empirical processes that are conducted in real life. Therefore, this chapter indicates the importance of introducing the PBL model into the design computing curriculum.

12.2.2 IC-PBL Model The IC-PBL model developed by Hanyang University is an innovative teaching and learning strategy. The IC-PBL model emphasises the relationship between the university and the real world, such as industry, to promote field-oriented talent who have practical problem-solving skills (Hanyang University IC-PBL Center for teaching and learning 2023). The collaboration with industry (the field experts) within the PBL model is an important feature of the IC-PBL model (see Fig. 12.1). The main instructors in the PBL model tend to be more familiar with theoretical knowledge than practical knowledge. However, they are given multiple roles in designing problems, operating class, and evaluating the students’ outcomes. However, as noted earlier, the PBL model aims to solve real-world problems (Barrows 1985), and therefore in which designing practice-oriented problems is important (Duch 2001; Hung 2006; Weiss 2003). Thus, in order to deliver better PBL classes, co-instructors who are familiar with practical knowledge to design problems and evaluate students’ outcomes from a practical point of view would be needed, and the IC-PBL model satisfies this.

Fig. 12.1 IC-PBL model

12 Urban and Architectural Design Education Using the IC-PBL Model … Table 12.1 Comparison of the four types of the IC-PBL model using the relationship between problem presenting and evaluation

Type

Problem presenting

Evaluation

M(erge)

Industry; society

Industry; society

E(valuate)

Instructor

Industry; society

C(reate)

Instructor

Instructor

A(nchor)

Industry; society

Instructor

215

To deliver better PBL classes, the IC-PBL model includes four types: M(erge), E(valuate), C(reate), and A(nchor). The instructors could select the four types of IC-PBL according to the characteristics of the problem being dealt with in class, specifically, using two main factors: “(1) whether the topic of problem or project is provided by a field or an instructor, and (2) whether the outcomes is evaluated by a field or an instructor” (Hanyang University IC-PBL center for teaching and learning n.d.). For example, M(erge) is a class type in which real society and a university are the most organically linked to problems. In M(erge), the field experts present problems and evaluate the outputs of the class. In the E(valuate) type, the instructors develop problems that need to be solved in real society and then, the field experts evaluate and give feedback on the class outcomes. In the C(reate) type, the instructors not only develop problems that require problem-solving in real society but also evaluate the outputs of the class. In the A(nchor) type, the problems that have occurred or have been dealt with in a real society are discussed, and the instructors evaluate the outputs of the class. Table 12.1 summarises the comparison of the four types of the IC-PBL model using the relationship between problem presenting and evaluation. In traditional education, students are given the information that is required to be memorised and then, they are assessed according to the given information for learning (Hanyang University IC-PBL center for teaching and learning 2023). However, in the IC-PBL approach, students are presented with practical problems and asked to determine potential solutions to the problems (Hanyang University IC-PBL center for teaching and learning 2023). In this way, the use of IC-PBL during design computing classes for architecture education can be seen as a significant benefit in terms of expanding the perspectives of students beyond the technological aspect to the physical aspects and embracing the context, having a focus on solving real problems.

12.3 Methodology A case study was conducted to explore the potential of the convergence of design computing and the PBL model in real urban and architectural design curricula. A case study is a qualitative method of research, occurring through observation (Gerring 2006; Yin 1994), which can provide an in-depth and detailed understanding of cases within a real-world context (Feagin et al. 2016). Data was collected through the observation of project outcomes to empirically investigate the impact of design computing

216

H. Yang and M. J. Kim

Fig. 12.2 Class on a virtual platform (Source Authors)

class applied the IC-PBL model on urban and architectural design education. The authors developed the class curriculum and systematically observed the progress of the class and students’ outcomes while operating the class as instructors. The following subsections illustrate the details of the class case. It includes participants and class mode, class composition and procedure, and IC-PBL scenarios.

12.3.1 Participants and Class Mode Twenty students majoring in architectural design and theory for Masters and PhD courses participated in a design computing course for one term (over 15 weeks). This class was conducted online using the Blackboard system of Hanyang University for non-face-to-face lectures because of the coronavirus disease (COVID-19) pandemic (see Fig. 12.2). Through the Blackboard system, an instructor was able to create a room that allowed students and the field experts to talk with each other and share related materials effectively.

12.3.2 Class Composition and Procedure The class was divided into two stages. The first stage (Weeks 1–7) was a recorded lecture on the relevant theories led by an instructor, which aimed to enhance students’ understanding of the issues raised by industries. The second stage (Weeks 8–15) was a student-led presentation and discussion class that applied the M(erge) type of the IC-PBL model. For the second stage, the 20 students were divided into five teams of four students. The presentations on the subtheme given in the previous week by industry (for more detail on the subthemes, see Table 12.2) were made weekly (weeks 9–13) by each team, and feedback was received from industry experts in real-time through a virtual platform. Table 12.2 shows the weekly course schedule and details for IC-PBL (second stage: Weeks 8–15).

12 Urban and Architectural Design Education Using the IC-PBL Model …

217

Table 12.2 Weekly course schedule and details for IC-PBL (second stage: Weeks 8–15) Week

Subject

Activity

8

Introduction

The specific guidance of the IC-PBL class in the second stage is introduced by an instructor. In addition, a lecture on universal design living laboratory by a field expert is provided to offer basic knowledge about the problems presented in this class

9

Characteristics of vulnerable groups and supporting technologies

Each team presents on the subtheme raised in Week 8 (What is a universal city?) and the field experts offer feedback. A lecture on the concept of geospatial data, the cases in which geospatial data are used, and the method for collecting geospatial data, which relate to the subtheme of Week 10, is provided

10

List of geospatial information for each type of vulnerable group

Each team presents on the subtheme raised in Week 9 (What is spatial information and how is it used?) and the field experts offer feedback. A lecture on vulnerable groups, focusing on safety issues, which relates to the subtheme of Week 11, is provided

11

Selection of test bed and data collection

Each team presents on the subtheme raised in Week 10 (What are the types of vulnerable groups? What are the constraints for them in city life?) and the field experts offer feedback. To support students to understand the main IC-PBL scenario (for more detail on the IC-PBL scenario, see Sect. 12.3.3), a lecture on technology for solving the constraints of cities for vulnerable groups and the test bed selection is provided

12

Selection of test bed and data collection

Each team presents on the subtheme raised in Week 11 (Where are the test bed for your projects?) and the field experts offer feedback. Empirical feedback from the field experts on the selection of test bed and data collection is provided

13

Geospatial information service alternatives using various smart technologies

Each team presents on the subtheme raised in Week 11 (What kind of geospatial information service alternatives will you suggest?) and the field experts offer feedback. Empirical feedback from the field experts on the problem-solving solutions by each team is provided

14

Final presentation and assessment

Final presentations are delivered and feedback from the field experts is provided

15

Submission of final outcomes on geospatial information service

Students submit the final outcomes

12.3.3 IC-PBL Scenarios: Problem Design The class was conducted using hands-on learning of the IC-PBL scenario presented by an industry. The IC-PBL scenario was:

218

H. Yang and M. J. Kim

Derive the geospatial information service alternatives to deliver a universal city to encompass vulnerable social groups.

The problem of current urban spaces that do not sufficiently provide accessibility for vulnerable groups was considered. The students were assumed to be employees of a spatial information modelling company. The students needed to seek smart approach solutions using spatial information beyond the physical alternatives that are traditionally used in the fields of urban and architectural design. To achieve this scenario, the sub scenarios of (1) “Organise a list of geospatial information to be built by the types of vulnerable groups,” (2) “Collect site-based geospatial information through the selection of a test bed,” and (3) “Derive alternatives for geospatial information services using diverse smart technologies” were used. These PBL scenarios developed for the second stage of this class aimed to understand the concept of a universal city and cultivate the ability of students to suggest the direction of smart city regeneration using problem-solving skills.

12.4 Results As mentioned in the methodology section, during the second stage of the IC-PBL class, subthemes related to the IC-PBL scenarios were provided by the field experts in each week (weeks 9–13) and presentations on them were delivered by each team. Table 12.3 illustrates the given subthemes and the approach of the students in responding to them. As shown in Table 12.3, overall, the students have more frequently reviewed census data, news, and real cases rather than books and academic papers to respond to the given themes (see weeks 9–12 in Table 12.3). The students have focused on a field survey to respond to the question related to a potential solution for resolving actual problems (see week 13 in Table 12.3). These approaches of students could be understood as the effect of the IC-PBL class in urban and architectural design education. The IC-PBL class case focuses on the problems in the real world, thus naturally, students tried to find potential solutions based on practical data reviews (such as actual news and census data reviews) rather than theoretical literature reviews (such as book and academic paper reviews). In other words, the IC-PBL class in urban and architectural design curriculum enables students to focus on more practical applications than theoretical knowledge in order to seek potential solutions to real problems. Through the approaches shown in Table 12.3, the overall outcome of the student projects proposed potential solutions driven from the perspective of design computing for delivering universal and inclusive design for all inhabitants in the urban environment, including vulnerable groups who have been excluded from being able to conveniently use given environments in daily urban life. This principal result indicates the importance of a convergence of design thinking that embraces social inclusion within the urban environment and design computing that uses the application of smart technology for all human beings to be able to sustain their life and well-being.

12 Urban and Architectural Design Education Using the IC-PBL Model …

219

Table 12.3 Approach of the students in responding to the subthemes Week

9

Subtheme

Approach of the students in responding to the subthemes

What is a universal Review of regulations city? Review of books Review of academic papers

Team 1 √

Case study

What is spatial information and how is it used?

√

0 3 2

√

√

√

√

√

√

√

√

4 4 0

Field survey

0

Persona development

0

Review of regulations Review of books Review of academic papers

√

√

0

√

3

√

1

√

√

√

√

√

√

√

√

4 √

5

Review of Census data

0

Review of Big data

0

Field survey

0

Review of regulations

√

√

0 2

Review of books

0

Review of academic papers

0

Review of news articles

√

Case study Review of Census data

√

√

√

√

√

√

√

5 0 3

Review of Big data

0

Field survey

0

Persona development Where are the test bed for your projects?

√

Review of Big data

Persona development

12

√

5

0

Case study

What are the types of vulnerable groups? What are the constraints for them in city life?

4

Review of Census data

Review of news articles

11

Note: frequency 3

√

Review of news articles

10

2

Review of regulations

√

0 1

Review of books

0

Review of academic papers

0 (continued)

220

H. Yang and M. J. Kim

Table 12.3 (continued) Week

Subtheme

Approach of the students in responding to the subthemes Review of news articles Case study Review of Census data Review of Big data

Team 2

3

4

5

√

√

√

√

√

√

√

√

√

√

√

√

√

√

√

Field survey Persona development 13

What kind of geospatial information service alternatives will you suggest?

Review of regulations

Note: frequency

1

√

5 0 5 3 0 3

√

1

Review of books

0

Review of academic papers

0 √

Review of news articles Case study Review of Census data Review of Big data Field survey Persona development

√

2 0

√

1

√ √

√

√

√

√

1 5 0

Note: The bold highlight indicates that the frequently used reference types which consequently show that practical data were more used to find potential solutions than theoretical literature

To be specific, through this design computing class’s application of a problemsolving approach, the students developed human experience-focused checklists or a toolkit as design considerations in accordance with the analysis of geospatial data (see Fig. 12.3). Through the process of developing human experience-focused checklists or a toolkit, the students had an opportunity for expanding their views on the design of physical built environments, specifically, that an inclusive understanding of the perspectives of vulnerable groups in society could enable the design of better user-friendly built environments. The checklists or toolkit suggested by students comprised two main factors: diagnostic items and evaluation, as shown in Fig. 12.3. This outcome, which was developed through an analysis of the test bed sites and experiences of vulnerable groups in using those physical built environments, can provide empirical data for delivering a better user-friendly living environment. For example, Team 1 selected the elderly as a vulnerable group and a particular location for a test bed in accordance with an analysis of academic references and census data. The team then developed a checklist for the elderly in accordance with the geospatial information analysis of the selected site, having a focus on their experiences. Team 4 chose women as a vulnerable group and a test bed site that had the highest level

12 Urban and Architectural Design Education Using the IC-PBL Model …

221

1*: Adequate, 2*: Common, 3*: Inadequate

Fig. 12.3 Examples of the checklists developed by Team 1 (top) and Team 4 (bottom)

222

H. Yang and M. J. Kim

of risk of sexual violence in Seoul, according to media and officially announced statistical data. The students of Team 4 developed checklists in accordance with an on-site survey and graphical analysis using a geographic information system (GIS) and space syntax techniques. Furthermore, the students proposed geospatial information services using various smart technologies for delivering a more convenient and safer city for diverse people (see Fig. 12.4). For example, the students of Team 1 suggested a sensor system for controlling traffic for the safety of vulnerable road users, such as the elderly. The students of Team 4 considered a mobile application announcing safer routes for women who return home late at night. These solutions can be understood as enhancing the value of the built environment beyond itself. The students learned how to provide a better design by deriving solutions to problems in the built environment through the systematic analysis of geospatial data. The students positively experienced the design computing approach using advanced technologies to analyse the problems and suggest potential solutions as a part of urban and architectural design. These project outcomes were derived from the curriculum of design computing, applying the IC-PBL model to show specific examples of the potential of information

Fig. 12.4 Examples of solutions suggested by Team 1 (top) and Team 4 (bottom) derived from geospatial information services using smart technologies

12 Urban and Architectural Design Education Using the IC-PBL Model …

223

technology combined with urban and architectural design to solve real social problems and, as a result, achieve a liveable built environment for various people. Thus, design computing using the IC-PBL model accompanied by architectural design studios can create a synergy effect on the improvement of students’ abilities to solve real-world problems from the human perspective.

12.5 Discussion This chapter has revealed the potential of design computing curricula that apply the PBL model for architectural design education. The problem-solving approach that uses human experiences (perspective) for designing urban environments and architecture can provide an empirical function beyond the physical built environment within urban and social contexts. Understanding the built environment through the problem-solving approach using spatial information analysis by technologies has a positive impact on designing better human-friendly built environments. Thus, the design computing approach using geospatial information-related technologies can synergise with the traditional approach for urban and architectural design studio education to explore problems and solve them. This empirical finding can also be understood from the existing literature. According to Alexander (1977), a design pattern of urban environments and architecture describes a problem and therefore can offer potential solutions. A good design (patterns) can be created by design thinking, which is a process of critical thinking to discover creative solutions through problem-solving (Dorst 2011) that uses a humancentric approach (IDEO n.d.). However, design thinking has a limitation of a weakness for structured problems that can be effectively resolved using computational thinking (Kelly and Gero 2021). For example, the challenges of wayfinding in a contemporary city could be resolved using the software that is available for GPSenabled smartphones and geographical data about Google Maps (Kelly and Gero 2021). For this reason, Kelly and Gero (2021) suggested a dual-process model of design thinking and computational thinking to address two types of design challenges (unstructured and structured problems). In other words, the convergence of design computing curricula that apply computational thinking and the IC-PBL model that uses design thinking can bring comprehensive results of learning performances for designing better urban environments and architecture from the perspective of users. This chapter identifies the benefits of design computing curricula that apply the IC-PBL model on a virtual platform as a part of urban and architectural education, which are outlined as follows: From the Perspective of Learners: • Expansion of students’ views to design user-friendly urban environments and architecture:

224

H. Yang and M. J. Kim

Students expanded their views to design better urban environments and architecture. Students suggested potential solutions in accordance with the problems from using an inclusive understanding of vulnerable groups in city life that enable the delivery of a universal city. • Understanding design computing for urban and architectural design: Students understood the potential of design computing approaches for designing urban environments and architecture. Students conducted an analysis of geospatial data using advanced technologies and proposed potential solutions for designing better cities using the analysis results. • Improvement in the visual expression of data: Students’ skills for data analysis, visual storytelling, and presenting on a virtual platform were improved because students needed to communicate with others online about work. The ability to perform data visualisation is an essential skill in the fourth industrial era because it can support effective decision-making and collaboration processes (Allen et al. 2021). • Ease and efficiency of sharing information: A variety of online information from YouTube links, website links, and online files was easily shared by not only the instructor and field experts but also students. This supported the initial stage of the project for a clear understanding of the concept. Besides, Students could effectively share their weekly outcomes through a virtual platform, which was then reviewed by the field experts and students received prompt feedback correspondently. This shows the positive advantage of a curriculum implemented by the IC-PBL model on a virtual platform. From the Perspective of Educators: • Possibility of interaction between students (university) and field experts (industry) without any space and time restrictions: Industry field experts participated in the classes every eight weeks, rather than one-time class participation, to provide continuous feedback on the problem-solving process of students without any attempt to visit the university. This allowed for the efficient provision of feedback and comments from industry, which is an important feature of the IC-PBL model. • Student-led discussion-driven learning: The psychological stability of a non-face-to-face system led to a better active discussion by students. Unlike face-to-face classes, students’ presentation times were longer, and they were more active in expressing their opinions about the instructor’s comments. This is the opposite of the finding of prior studies (Grimes 2002; Yang and Cornelius 2004; Prodgers et al. 2023), which have discussed the lack of a sense of belonging to the class as a critical disadvantage of online education.

12 Urban and Architectural Design Education Using the IC-PBL Model …

225

• Ease of management from a supervising perspective as a clear comparison of the development and performance among every group: From the perspective of an instructor, it was possible to quickly determine whether the class was progressing well toward the goal according to the contents each week and it was easy to compare students’ performances over time. Recent studies have explored how online education was implemented and how it performed in PBL learning activities during the COVID 19 pandemic (Elzainy et al. 2020; Ping et al. 2020; Khandakar et al. 2022). In addition, they have commonly concluded the successful achievement of a virtual platform class using the IC-PBL model for teachers and students. Likewise, this chapter could provide an insight into delivering better online education in the digital era.

12.6 Conclusion The application of technology using a problem-solving design approach has enhanced the significance of urban and architectural design by elevating human experiences. A design computing curriculum driven by problem-solving approaches can provide significant levels of support beyond traditional design education for students and instructors. Specifically, a virtual platform enables flexibility in the interactions between a university (students and instructors) and an industry (field experts), which can result in a better performance of design computing curricula that apply the ICPBL model in urban and architectural design education. Thus, this chapter shows the potential of a design computing curriculum that uses the IC-PBL model on a virtual platform in design education. However, online classes using the IC-PBL model could potentially restrict students’ engagement levels. Nonetheless, the IC-PBL model has unique characteristics of designing real-world problems and solving them through the active interaction between students and field experts. In addition, for design computing classes in urban and architectural design education, it is important to effectively communicate through visualisation. This chapter contributes to this body of knowledge regarding how virtual platforms promote interaction and collaboration among all participants in a class, including learners, instructors, and industry experts. Acknowledgements This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF-2022S1A5A2A01038779). We would like to thank Sooyeon Han, a field expert in this design computing class who applied the IC-PBL model, for her time and enthusiasm as well as her comments on the class.

References Alexander C (1977) A pattern language: towns, buildings, construction. Oxford University Press

226

H. Yang and M. J. Kim

Allen L, Atkinson J, Jayasundara D, Cordiner J, Moghadam PZ (2021) Data visualization for Industry 4.0: a stepping-stone toward a digital future, bridging the gap between academia and industry. Patterns 2(5):100266 Barrows HS (1985) How to design a problem-based curriculum for the preclinical years. Springer Publishing Company Barrows HS (1996) Problem-based learning in medicine and beyond: a brief overview. New Drctns Teach & Learn 1996(68):3–12 Cho ME, Lee JH, Kim MJ (2022) Identifying online learning experience of architecture students for a smart education environment. Journal of Asian Architecture and Building Engineering 1–12 Cortiz D, Silva JO (2017) Web and virtual reality as platforms to improve online education experiences. In: 2017 10th international conference on human system interactions (HSI), IEEE de Jong C, Klemp K, Mattie E, Maan J (2021) Ten principles for good design: Dieter Rams. Prestel Dorst K (2011) The core of ‘design thinking’and its application. Design Studies 32(6):521–532 Duch BJ (2001) Writing problems for deeper understanding. The Power of Problem-Based Learning 47–58 Elzainy A, Sadik AE, Abdulmonem WA (2020) Experience of e-learning and online assessment during the COVID-19 pandemic at the College of Medicine, Qassim University. Journal of Taibah University Medical Sciences 15(6):456–462 Feagin JR, Orum AM, Sjoberg G (2016) A case for the case study. UNC Press Books Gero JS (1998) Research in design computing: an artificial intelligence framework. In: International conference on artificial intelligence for Engineering, Citeseer, HUST Press, Wuham Gerring J (2006) Case study research: principles and practices. Cambridge University Press Goldman G, Zdepski S (1987) Form, color and movement. In: Proceedings of ACADIA 1987 conference, Raleigh Grimes EB (2002) Student perceptions of an online dental terminology course. Journal of Dental Education 66(1):100–107 Guzdial M (2015) Learner-centered design of computing education: research on computing for everyone. Synthesis Lectures on Human-Centered Informatics 8(6):1–165 Hanyang University IC-PBL Center for Teaching and Learning (2023) A handbook for Hanyang University professors Hanyang University IC-PBL Center for Teaching and Learning (n.d.) IC-PBL connect & share. http://hyicpbl.hanyang.ac.kr/eng/pbl/pbl_about.php. Accessed 23 June 2023 Hmelo-Silver CE (2004) Problem-based learning: what and how do students learn? Educational Psychology Review 16(3):235–266 Hung W (2006) The 3C3R model: a conceptual framework for designing problems in PBL. Interdisciplinary Journal of Problem-based Learning 1(1):6 IDEO (n.d.) Design thinking defined. https://designthinking.ideo.com/. Accessed 23 June 2023 Jung Y, Lee Y (2021) A study on the educational effect of the history of fashion and costume through a comparison of general lecture and IC-PBL (Industry-coupled Problem-based Learning). Journal of Fashion Business 25(2):98–109 Kelly N, Gero JS (2021) Design thinking and computational thinking: a dual process model for addressing design problems. Design Science 7:e8 Khandakar A, Chowdhury ME, Khalid MS, Zorba N (2022) Case study of multi-course projectbased learning and online assessment in electrical engineering courses during COVID-19 pandemic. Sustainability 14(9):5056 Lee DY, Chung HM (2008) Investigating constructivist approach to design studio courses in universities of USA and UK. The Architectural Institute of Korea Planning & Design 24(6):149–160 Lee JH, OstwaldMJ GuN, Roberts M (2021) Linguistic and cultural perspectives on globalised design education. International Journal of Technology and Design Education 31:165–181 Marx J (2000) A proposal for alternative methods for teaching digital design. Automation in Construction 9(1):19–35

12 Urban and Architectural Design Education Using the IC-PBL Model …

227

McCann JE (1983) Design guidelines for social problem-solving interventions. The Journal of Applied Behavioral Science 19(2):177–189 Oh H, Yoon SK, Lim G-G (2021) Exploring blended IC-PBL model & strategy for course based PBLS in university: using a case study in engineering education. PBL, Sustainability and Digitalisation 2021:13 Ping Z, Fudong L, Zheng S (2020) Thinking and practice of online teaching under COVID-19 epidemic. In: 2020 IEEE 2nd international conference on computer science and educational informatization (CSEI), IEEE Prodgers L, Travis E, Pownall M (2023) “It’s hard to feel a part of something when you’ve never met people”: defining “learning community” in an online era. Higher Education 85(6):1219–1234 Silva NF (2001) Design computing education: developing a post-grad CAAD curriculum based on specific desing project. In: Gero JS, Chase S, Rosenman M (eds) CAADRIA2001. Key Centre of Design Computing and Cognition, University of Sydney, pp 315–318 University of New South Wales (2023) UNSW handbook. https://www.handbook.unsw.edu.au/und ergraduate/courses/2023/CODE1161?year=2023. Accessed 13 June 2023 University of Sydney (2021) Architecture, design and planning handbook 2021. https://www.syd ney.edu.au/handbooks/archive/2021/architecture/undergraduate/b_design_computing.shtml. html. Accessed 13 June 2023 Weiss RE (2003) Designing problems to promote higher-order thinking. New Drctns Teach & Learn 2003(95):25–31 Yang Y, Cornelius LF (2004) Students’ perceptions towards the quality of online education: a qualitative approach. Association for Educational Communications and Technology Yin RK (1994) Case study research: design and methods. Sage Publications

Chapter 13

The Impact of Visual Character on Perceived Stress Levels: An Intelligent Approach Applied to University Campus Design Zhixian Li , Xiaoyi Zu , Ju Hyun Lee , and Michael J. Ostwald

Abstract Various types of streetscapes have been the subject of past research, with university campus planning being identified as one example where a psychological impact has been observed. However, due to the visual complexity of campus streetscapes, little or no clear approach is available to quantitively assess their potential impacts. Furthermore, collecting empirical data about environmental stress levels in urban spaces remains a significant challenge. In response, this chapter presents an “intelligent” approach to estimating the relationship between street view imagery (SVI) properties and perceived stress in seven university campuses in China. Specifically, an automatic, semantic segmentation method is used to measure the visual properties of 6056 SVIs—the visual element proportions (VEPs) of design elements and visual features. Then, a human–machine adversarial model using a random forest is applied to predict the perceived stress scores (PSSs) of SVIs. Through this combination of a computer vision technique and machine learning, this research identifies the various impacts of visual elements on PSSs. The research also tests the significance of three visual features holistically contributing to lower stress levels in campus design. This chapter concludes with a discussion of the findings and a contribution to architectural and urban studies. Keywords Campus design · Perceived stress · Street view imagery (SVI) · Machine learning · Deep learning · Semantic segmentation

Z. Li (B) · J. H. Lee · M. J. Ostwald School of Built Environment, Faculty of Arts, Design and Architecture, The University of New South Wales, Sydney, NSW 2052, Australia e-mail: [email protected] X. Zu School of Architecture, Tsinghua University, Haidian District, Beijing 100084, China © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 J. H. Lee et al. (eds.), Multimodality in Architecture, https://doi.org/10.1007/978-3-031-49511-3_13

229

230

Z. Li et al.

13.1 Introduction Past research has identified a variety of mental health issues (e.g., depression, anxiety, and stress) that are prevalent in tertiary education systems (Bayram and Bilgel 2008; Jones et al. 2018; Pascoe et al. 2020), but the impact of campus design character on psychological stress is largely unknown. To address this knowledge gap, this chapter examines the relationship between campus streetscape character and perceived stress. Research in environmental psychology has repeatedly demonstrated that the characteristics of environments can have an impact on human health (Evans 2003; Parsons 1991). Past psychological research, however, largely relies on subjective assessments of stress, typically collected from interviews or questionnaires (Meng et al. 2020; Peacock and Wong 1990; Shields and Slavich 2017), both of which are prone to various types of memory bias. In contrast, some computational approaches are used to simulate and measure human environmental perceptions and behaviours (Lee and Ostwald 2021; Lindberg et al. 2018; Yin et al. 2020), including responses to the visual characteristics of the built environment (Lee and Ostwald 2023). Biometric data developed from skin conductance, heart rate and electroencephalography (EEG) can also be used for research on environmental psychology (Li et al. 2022a, b, c; Li and Sullivan 2016). Although these computational approaches are quite effective, they have not yet been applied to model the relationship between environmental design and perceived stress. To fill this knowledge gap, this chapter addresses a research question, “how can we computationally measure and predict the relationship between visual properties and psychological responses in university campus design?”. The visual elements of campus streetscapes are too heterogeneous to be measured using a standard computational method. However, a computer vision technique based on street view imagery (SVI) combined with deep learning algorithms can be used for this purpose (Quercia et al. 2014; Rundle et al. 2011). For example, data from SVI providers, such as Google, Tencent and Baidu, can be used by researchers to automatically generate big data from geotagged SVIs, allowing them to virtually investigate streets in many cities around the world (Li et al. 2022a, b, c). With this approach, hundreds of SVIs can be captured in a short time and then used for visual element measurement (Badland et al. 2010), quality evaluation (Hou and Biljecki 2022) and automatic quantification using deep learning (Dubey et al. 2016). Specifically, the proportions of various visual elements in SVI (e.g., tree, building, sky, and road) can be automatically and precisely measured using deep convolutional neural networks (He and Li 2021; Wu et al. 2019). There are several well-known deep learning algorithms such as FCN (fully convolutional network) (Long et al. 2015) and Resnet (residual neural network) (He et al. 2016), and this research adopts SegNet (semantic segmentation model) because it has proven to be an effective algorithm for precisely classifying the visual elements of SVIs (Badrinarayanan et al. 2017; Han et al. 2022).

13 The Impact of Visual Character on Perceived Stress Levels …

231

Notably, collecting perceived stress scores (PSSs) correlated to thousands of SVIs isn’t practical. Thus, this chapter uses a random forest (RF) machine learning algorithm, which has proven capable of effectively solving real-world classification problems, to predict human stress levels (Breiman 2001; Fernandez-Delgado et al. 2014). That is, while RF is often used as a classifier for semantic image segmentation (Badrinarayanan et al. 2017), it can be used to estimate human perception assessment using a human–machine adversarial model (Yao et al. 2019). In summary, this chapter presents an “intelligent” approach—combining automatic semantic segmentation using SegNet and human–machine adversarial scoring using RF—to measure and predict the PSSs of SVIs in seven university campuses. The following methodology section explains case selection, the development of variables (SVI collection, segmentation, visual element proportion (VEP), visual features, and PSS prediction), and the statistical methods employed. Specifically, this research examines three visual features (nature, connectivity, and enclosure) as important independent variables. It then reports visual and psychological properties in campus streetscapes and the relationship between these variables. This chapter concludes with a discussion of limitations and implications for future research.

13.2 Methodology 13.2.1 Case Selection To examine the visual and psychological variables in campus streetscapes, seven universities in the city of Shanghai were selected—A. Shanghai University of Sport, B. East China University of Science and Technology, C. Donghua University, D. Shanghai University of Finance and Economics, E. Fudan University, F. Shanghai University of Technology, and G. Tongji University. These seven provide a consistent set of sufficient size and accessibility so that the chosen methods can be tested. Furthermore, confounding factors such as climate and culture could be minimized by selecting major campuses in a single city. Chinese campus design is based on the Soviet enclave form, developing unique SVIs. Compared to integrated campuses in urban settings in Western countries, university campuses in China are closer to “gated communities” (Sun et al. 2018), defined by fences and gates. Students are also recommended to live on their university campus (Zhong et al. 2018).

13.2.2 Street View Imagery (SVI) Collection SVI, as an actual image of the city, is an important data source used for a variety of urban studies such as spatial data infrastructure, greenery, health and well-being, urban morphology, transportation and mobility, walkability, socio-economic studies,

232

Z. Li et al.

real estate, and urban perception (Biljecki and Ito 2021; Siriaraya et al. 2020). It has become a popular method in recent research into both urban environments and psychological perceptions (Li et al. 2022a, b, c). This chapter uses Baidu Maps, one of China’s largest online SVI providers (Biljecki and Ito 2021). First, OpenStreetMap’s seven universities’ road network data and campus road vector lines were extracted. Viewpoints were set at every 50 m intervals on each main road via ArcGIS. A Python Baidu Map API Library was then applied to develop four directional SVIs (0°, 90°, 180° and 270°) from a street viewpoint. To simulate human perspectives, the image size of each SVI was a maximum of 600 × 400 px; the width of the field of view (FOV) was 90°; and the field of angle (pitch) was 20°. Collectively, 6056 SVIs were developed from 1514 SVI-viewpoints (case A: 137, B: 191, C: 114, D: 96, E: 208, F: 461, G: 307).

13.2.3 Segmentation and Visual Element Proportion (VEP) Having generated the 6056 SVIs, a deep learning algorithm was employed to extract the visual elements of each. Specifically, an image segmentation algorithm, SegNet, provided a deep convolutional network architecture using pixel-wise classification. Past research has identified this approach as one of the best ways of obtaining highly accurate semantic segmentation results (Badrinarayanan et al. 2017). As such, SegNet was used to segment each SVI into visual elements (e.g., trees, buildings, and roads) and to automatically calculate their proportions—visual element proportion (VEP). The annotated images from the ADE20K semantic segmentation dataset were used to train the SegNet model. The ADE20K dataset includes 151 categories (e.g., trees, buildings, cars) (Zhou et al. 2017). Figure 13.1 illustrates the convolutional encoderdecoder of SVI segmentation and some outputs of the SegNet model.

13.2.4 Visual Features In addition to individual visual elements, several visual features have been proven to significantly correlate with people’s mental health (Dai et al. 2021). In environmental perception, a set of visual elements is holistically used to evaluate complex perceptions (Donderi 2006). In this chapter, three visual features, “nature” (or natural scenery), “enclosure”, and “connectivity”, are calculated from the SVIs to examine their impacts on PSSs. Specifically, each factor is defined as the average VEP value of four SVIs from a viewpoint. “Nature” is the average VEP of “tree”, “grass”, “plant”, “sky”, and “water”, and “enclosure” is the average VEP of “building”, “tree”, and “fence”. Notably, instead of “walkability”, which can be measured by “the ratio of the sidewalk to the overall road” (Dai et al. 2021), this research measures “connectivity” by calculating the average VEP value of both “road” and “sidewalk”. Universities

13 The Impact of Visual Character on Perceived Stress Levels …

233

Fig. 13.1 SVI segmentation by the SegNet model, adapted from Badrinarayanan et al. (2017)

commonly support pedestrian-friendly street design, maintaining inclusive, accessible streets (Asadi-Shekari et al. 2014), where both roads and sidewalks are available for people to use. Furthermore, vehicles are not allowed on campus without special approval in many universities. Vehicles must also slow down in the campus environment to enable pedestrians to pass first. Thus, “connectivity” could be an important visual feature on campus. “Nature (or natural scenery)” refers to the natural elements in the built environment. Exposure to nature, such as green spaces and natural landscapes, can have a beneficial effect on human stress levels as well as student performance (Li and Sullivan 2016). However, not all nature exposure is effective in stress reduction, and a variety of factors influences the relationship between nature and stress. Some studies have shown that mental health is also related to the type of green vegetation, with a higher density of trees being beneficial to mental health. In contrast, understory vegetation is negatively associated with mental health (Jiang et al. 2020). “Enclosure” in urban environments is critical in human neural responses and visual perception. As a fundamental physical component, enclosures indicate whether the environment is safe, sheltered, or visible and clear (Stamps and Smith 2002). In addition, environments with low levels of “enclosure” may lack the opportunity for stress recovery due to potentially restricted access to biophilic elements. In contrast, access to these elements has been linked to stress recovery (Ulrich 1993). As for “gated” campuses, this research treats “enclosure” as the combination of “building”, “tree”, and “fence” in a street view. Lastly, “connectivity” is often regarded as an important factor that affects stress levels. For example, people living in highly accessible neighbourhoods may have lower stress and anxiety levels and better health (Ma et al.

234

Z. Li et al.

Fig. 13.2 A human–machine adversarial model to predict the PSSs of SVIs

2023). The correlation between connectivity and stress can be attributed to a variety of factors, including increased physical activity and improved social interaction and cohesion (Baobeid et al. 2021; Zhu et al. 2013). In this research, the VEP combines “road” and “sidewalk”.

13.2.5 PSS Prediction by Random Forest (RF) After the automatic SVI segmentation, the PSS of each SVI was measured by an RF model (Fig. 13.2). As for the human–machine adversarial model, twenty Chinese undergraduate students (gender: 10 males and 10 females) scored a random sample of 100 SVIs using a stress perception scale from 1 to 6 (1 is the lowest level of perceived stress while 6 indicates the highest level)—the first author collected written consent forms before their experiments. This research selected participants who enrolled in Shanghai universities, with the exclusion of those who were undergoing stressrecovery medication or therapy. RF was then applied to fit the relationship between the semantic segmentation results and the PSSs of 2000 SVIs and to predict the remainder (PSSs of 4056 SVIs). After a student scored more than 50 images, a human–machine adversarial model was trained by the existing scoring dataset. Next, the recommended scores for the SVIs were fed-back to the student to correct the estimated PSSs until each reached 100 PSSs. The robust automated system finally estimated all the PSSs of given SVIs.

13.2.6 Statistics Along with descriptive statistics, a one-way analysis of variance (ANOVA) was conducted to examine the visual and psychological properties of seven universities. In the test, η2 was also calculated to estimate effect sizes and interpreted according to proposed values of small (≥ 0.01), medium (≥ 0.06) and large (≥ 0.14) (Cohen 1988). Multiple linear regressions were then used to examine the impacts of visual properties (VEPs and visual features) on PSSs.

13 The Impact of Visual Character on Perceived Stress Levels …

235

Fig. 13.3 VEPs of eight visual elements and three visual features

13.3 Results 13.3.1 Visual and Psychological Properties 13.3.1.1

Visual Element Proportion (VEP)

Whilst SegNet identified 150 visual elements from 6056 SVIs, only the top 8 visual elements (tree, building, road, sky, sidewalk, plant, grass, and wall) are reported in this research (Fig. 13.3). They accounted for 90.9% of the 150 VEPs of collected SVIs. “Tree” develops the largest VEP of university campuses (24.5%), followed by “building” (21.8%) and “road” (14.2%). The average VEP of “sky”, “sidewalk”, “plant”, “grass”, and “wall” is 9.2%, 9.1%, 5.6%, 2.6%, and 1.3%, respectively. Notably, the results of the one-way ANOVA test for VEPs identified multiple significant differences across the seven universities. For example, the VEP of “tree” in case C (M = 17.1%) was significantly lower than case A (M = 20.6%), D (M = 25.0%), E (M = 24.4%), F (M = 25.9%), and G (M = 28.1%). In contrast, the VEP of “building” in case C was significantly higher than in case A (M = 13.2%), D (M = 20.5%), E (M = 18.0%), F (M = 22.4%), and G (M = 19.8%). Interestingly, the VEPs of “road” (M = 26.2%) and “sky” (M = 19.6%) in case A were significantly higher than the remainder. These results confirm that the VEP offers a potentially useful basis for examining a streetscape’s visual character.

13.3.1.2

Visual Features

This research, furthermore, examined holistic characteristics of visual elements in a campus streetscape because they could collectively impact on perceived stress. As illustrated in Fig. 13.3, the highest average factor proportion or VEP value is “enclosure”, which accounts for 46.3%. The average VEP value of “nature”, which

236

Z. Li et al.

could be a positive factor in lowering PSS, is 35.8%, but the range of VEP values is comparatively wide. Case D developed the highest average value of “nature” (M = 40.2%), which was significantly higher than case A (M = 30.9%), B (M = 28.7%), and C (M = 25.9%). Case A developed the highest average value of “connectivity”. In particular, the “connectivity” (M = 28.5%) of case A was significantly higher than case B (M = 23.5%), D (M = 20.8%), F (M = 21.2%), and G (M = 24.0%). The average value of “connectivity” in seven cases was 23.3%. These descriptive statistics indicate that the visual features proposed in this research could describe the characteristics of a street view and function as independent variables in regression.

13.3.1.3

Perceived Stress Scores (PSS)

Figure 13.4 shows the PSSs of the seven universities on their maps. The colours at the viewpoints are darker for higher PSSs. The SVIs of case A developed the lowest average PSS (M = 2.312), which was significantly lower than case B (M = 3.486), C (M = 3.887), E (M = 3.482), F (M = 3.283), and G (M = 2.996). This finding was interesting because the campus developed the highest VEPs of “road” and “sky”. In contrast, case C developed the highest average PSS (M = 3.887), which was significantly higher than cases A, D, F, and G. This result was consistent with the fact that case C developed the lowest VEP of “tree” and the largest VEP of “building” among the seven cases. In summary, independent and dependent variables generated from the automatic semantic segmentation and human–machine adversarial model proved effective for examining and comparing the visual and physiological properties of campus streetscapes.

Fig. 13.4 The ranges of PSSs in the seven cases and their PSSs

13 The Impact of Visual Character on Perceived Stress Levels …

237

13.3.2 The Relationship Between Visual Elements and PSSs 13.3.2.1

Impacts of Visual Element Proportions on PSSs

Multiple linear regression was used to determine the relationship between VEPs and PSSs. First, it was found that the VEPs of “building” significantly predicted PSSs (β = 0.345, p = 0.000). This suggests that the perception of stress is greater when the category building is a larger proportion of the environment. In contrast, the VEPs of “tree” (β = –0.168, p = 0.017), “sky” (β = –0.353, p = 0.000), “sidewalk” (β = –0.149, p = 0.000), “plant” (β = –0.139, p = 0.000) and “grass” (β = –0.115, p = 0.000) have negative impacts on PSSs. That is, stress scores decrease as the VEPs of trees, sky, sidewalks, plants, and grass increase. It was found that the VEPs of “road” and “wall” did not significantly predict PSSs (β = 0.045, p = 0.057). In summary, except for “wall”, the dominant visual elements were significant predictors of PSSs.

13.3.2.2

Trees, Buildings, Roads, and Sky

A one-way ANOVA test compares the VEP results to investigate differences among the three PSS levels—high (4–6), medium (3–4) and low (1–2). Figure 13.5 illustrates the ANOVA results for four visual elements (“trees”, “buildings”, “roads”, and “sky”) where significant differences have been identified. The VEPs of “tree” in street views developing the low level of PSSs (M low = 0.293, SD = 0.092) were significantly higher than those producing the medium level of PSSs (M medium = 0.235, SD = 0.080), with a medium effect size of η2 = 0.113. Likewise, the VEPs of “tree” in SVIs generating the medium level of PSSs were significantly higher than those causing the high level of PSSs (M high = 0.208, SD = 0.120). That is, if the VEPs of “tree” in a campus streetscape are less than 0.208, students tend to have high stress levels. If the VEPs of “tree” in an SVI are around or larger than 0.293, it could cause a low level of perceived stress. The effect of “building” on stress levels (Mlow = 0.139, SD = 0.074; Mmedium = 0.202, SD = 0.086; Mhigh = 0.313, SD = 0.148) was also quite large, p = 0.000, η2 = 0.306. PSS could be relatively low if the VEPs of “building” in the environment were less than 0.139. Similarly, if the VEP of “building” in an SVI is around or more than 0.202, students tend to have a high stress level. The VEPs of “road” in SVIs developing the low level of PSSs (Mlow = 0.118, SD = 0.090) were also significantly lower than those linking to the medium-stress level (Mmedium = 0.153, SD = 0.096) and the high-stress level (Mhigh = 0.156, SD = 0.082). Although there was no significant difference in the VEPs of “road” between medium- and high-stress levels, the results implied that the VEPs of “road” in a campus streetscape are positively related to the PSS levels. As such, even in the “gated” campus environment, “road” is still an obstructive design element for students. However, the impact of “road” was relatively small (η2 = 0.037). In addition, the regression analysis indicated the VEP of “road” was not a significant

238

Z. Li et al.

Fig. 13.5 Box and whisker plots of the VEP results of “tree”, “building”, “road”, and “sky” across three stress levels with their ANOVA tests (ns not significant, **: p < 0.01)

predictor of PSSs. In this context, it is difficult to consider “road” as an independent variable. In contrast, the effect of “sky” on stress levels was notable, with an effect size of η2 = 0.132. The VEPs of “sky” in streetscapes developing the low level of PSSs (Mlow = 0.109, SD = 0.063) were significantly lower than those producing the high level of PSSs (Mhigh = 0.059, SD = 0.048). Likewise, the VEPs of “sky” in SVIs generating the medium level of PSSs (Mmedium = 0.109, SD = 0.067) is significantly lower than those associated with the high level of PSSs. That is, if the VEP of “sky” in a street view is around or greater than 0.109, PSS could be comparatively low.

13.3.2.3

Sidewalk, Plant, Grass, and Wall

It was also evident that the VEPs of four visual elements (“sidewalk”, “plant”, “grass”, and “wall”) across three stress levels were significant differences. First, the VEPs of “sidewalk” in campus streetscapes developing the low level of PSSs (Mlow = 0.114, SD = 0.086) were significantly greater than those producing the medium level of PSSs (Mmedium = 0.099, SD = 0.071). In addition, the VEPs of “sidewalk” in SVIs producing the medium level of PSSs are significantly higher than those linking to the medium level of PSSs (Mhigh = 0.060, SD = 0.055). The findings suggest that stress levels decreased as the visual proportion of “sidewalk” in the campus design increased. That is, the VEP of “sidewalk” in an SVI greater than 0.114 should be recommended for stress reduction. Second, the VEPs of “plant” in street views developing the low level of PSSs (Mlow = 0.067, SD = 0.059) were significantly higher than those causing both the medium-stress level (Mmedium = 0.052, SD = 0.043) and the high-stress level (Mhigh = 0.050, SD = 0.047). However, there was no significant difference in the VEPs of “plant” between medium- and high-stress levels. In contrast, there were substantial differences in the VEPs of “grass” in SVIs associated with the three PSS levels (Mlow = 0.069, SD = 0.061; Mmedium = 0.054, SD = 0.050; Mhigh = 0.042, SD = 0.054). That is, if the VEPs of “grass” in a campus streetscape are less than 0.042, students tend to have high levels of stress. The VEPs

13 The Impact of Visual Character on Perceived Stress Levels …

239

of “grass” greater than 0.069 in an SVI could cause a low level of perceived stress. The effect of “grass” was, however, relatively small (η2 = 0.037) because of its low proportion in a street view. Lastly, there was a significant difference between the VEPs of “wall” in SVIs linking to the low-stress level (Mlow = 0.007, SD = 0.026) and those developing the high-stress level (Mhigh = 0.012, SD = 0.028). Despite this, its effect was minimal (η2 = 0.007), and the VEP of “wall” in a campus street view was not a significant predictor of PSSs in the regression model.

13.3.3 The Relationship Between Visual Features and PSSs 13.3.3.1

Impacts of Visual Features on PSSs

According to the results of regression analysis, the VEPs of three visual features (“nature”, “enclosure”, and “connectivity”) are significant predictors of PSSs. First, the VEP of “nature” in the environment has an inverse correlation with the PSS (β = –0.521, p = 0.000). Expectedly, as the proportion of “nature” increases, PSS decreases. In contrast, there is a positive correlation between the VEPs of “enclosure” and PSSs (β = 0.256, p = 0.000). The greater the visual “enclosure”, the greater the students” PSS. The impact of the VEPs of “connectivity” in a campus streetscape on PSS is negative (β = –0.115, p = 0.000). As such, the higher “connectivity”, the less perceived stress. The findings imply that the proposed visual features could work effectively for stress-free campus design.

13.3.3.2

Nature, Enclosure, and Connectivity

Given the one-way ANOVA results, which revealed significant differences in VEPs between the low and high PSS levels, a paired samples t-test was used to compare the means of the VEPs of three visual features from the two PSS levels. The t-test was also intended to reveal the effect of the visual proportion of each visual feature on perceived stress. Figure 13.6 presents the VEP values of visual features for the two stress groups (low and high). One of the first differences apparent in the data is that the VEP results of “nature” in SVIs linking to the low level of PSSs (M low = 0.510, SD = 0.110), compared to those developing the high PSS level (M high = 0.391, SD = 0.139), were significantly higher, t = 17.903, p = 0.000, with a significant effect (Cohen’s d = 0.757). This result confirms that street views developing a low level of stress have more “nature” design elements. Past research suggests that “nature” should be the most important visual influence on perceived stress. In contrast, the VEP results of “enclosure” in streetscapes generating the low level of PSSs (M low = 0.435, SD = 0.079) compared to those associated with the high PSS level (M high = 0.509, SD = 0.082) demonstrated significantly lower values, t = –15.204, p = 0.000 (Cohen’s d = 0.642). In this sense, Chinese students might feel stressed in a highly enclosed environment and prefer “openness”. In this research,

240

Z. Li et al.

Fig. 13.6 Box and whisker plots of the VEP results of “nature”, “enclosure”, and “connectivity” across two stress levels with their t-test results (**: p < 0.01)

“openness” was not defined as a visual feature, but it might be presented by the VEP of “sky” being greater than 0.109 and the VEP of “building” less than 0.139 in an SVI. In addition, “connectivity” could help to support “openness” psychologically. The “connectivity” VEPs of SVIs developing the low level of PSSs (0.243, SD = 0.083) were significantly higher than those linking to the high level of PSSs (M high = 0.223, SD = 0.084), t = 4.157, p = 0.000, but the effects were minor (Cohen’s d = 0.176). The result confirmed that the third visual feature for university campus design, defined by the average VEP value of both “road” and “sidewalk”, was an acceptable variable.

13.4 Discussion This chapter addresses three significant research gaps in human environmental perception. First, research on streetscape like an SVI is only recently developing (He and Li 2021; Yao et al. 2021). Furthermore, university campuses which are separate from the city could differ from those set within a typical urban landscape (Breitung 2012). Second, past psychological research is limited to subjective stress measures (Meng et al. 2020; Shields and Slavich 2017), while it is possible to automatically simulate and measure human environmental perceptions (He and Li 2021; Wu et al. 2019). Lastly, there needs to be a holistic view of visual elements in a campus street view. Specifically, in this research, “connectivity” was different from “walkability” focusing on “sidewalk” (Dai et al. 2021). “Enclosure” was also defined by a combined VEP value of “building”, “tree”, and “fence” in an SVI, separating university campuses from urban space (Kan et al. 2017; Li et al. 2022a, b, c). In this way, this research has provided new knowledge about the relationships between domain-specific VEPs and PSSs in university landscape design.

13 The Impact of Visual Character on Perceived Stress Levels …

241

Although students’ perceived stress could be correlated with performance and daily lives (Khan et al. 2013) and other sensory dimensions (Grahn and Stigsdotter 2003), this chapter revealed that visual design elements affect their perceived stress in a campus streetscape. Furthermore, the VEP ranges of eight visual elements and three visual features could be used for measuring the visual characters of campus designs as well as for future campus design guidelines. An obvious implication is that to reduce students’ perceived stress on campus, designers should focus on the VEPs of trees, sky, sidewalks, plants and grass. The VEP results in this chapter are beneficial for this purpose. Interestingly, whilst “road” developed the largest proportion in an urban SVI (Chen et al. 2022; Han et al. 2022), “tree” was the most dominant visual element in the seven Chinese campuses and the most important one for campus stress management. The VEP of “tree” in an SVI greater than 29.3% is strongly recommended in campus design. Likewise, it is suggested that the VEPs of greater than 10.9% “sky”, 11.4% “sidewalk”, 6.7% “plant”, and 6.9% “grass” in an SVI can reduce perceived stress. In contrast, the VEPs of less than 13.9% “building”, 11.8% “road”, and 0.7% “wall” are beneficial in reducing perceived stress on campus. In this context, it is recommended that campus streetscapes have VEPs of greater than 51.0% “nature” and 24.3% “connectivity” but less than 43.5% “enclosure”. Although this chapter presents intelligent approaches to measuring and predicting the relationship between visual properties and psychological responses in university campus design, some limitations should be addressed in a future study. First, SVIs developed by the Baidu Maps API might have inconsistencies in time, season, and weather, which can be improved by additional crowdsourced databases or SVI collection methods. Second, the human–machine adversarial model was examined using the seven Chinese campuses in Shanghai, but it should be further verified or validated using cases in other cities or other countries to enhance the generalisability of the results. Third, although a sample size of 20 participants is applicable for this purpose (Han et al. 2022), the ideal number of participants in the RF model, based on their backgrounds and behavioural characteristics, needs further clarification. Lastly, the colour, seasonality, or content of a façade could have a different impact on PSS. The current methodology would not capture this type of non-geometric design element, but it should be considered in future research.

13.5 Conclusion This chapter has investigated a research question, “how can we computationally measure and predict the relationship between visual properties and psychological responses in university campus design?” Combining recent advances in machine learning and computer vision techniques, this research has presented an intelligent approach combining automatic semantic segmentation and a human–machine adversarial model, which facilitates capturing the impact of a streetscape on perceived stress in university campus design from 6056 SVIs.

242

Z. Li et al.

Responsive design to public mental health and well-being is important in realising human-centred design principles, but it has been largely dependent on subjective assessments. In contrast, this research, combining advanced machine learning methods with SVI, provides new perspectives on landscape and urban design, along with the evolving development of big data. Furthermore, the presented research framework reveals the impact of visual elements and factors on perceived stress in the university campus environment. As a long-term residential environment for students, low-stress campus design would support their well-being and effective learning. In summary, this research contributes to the reduction of students’ perceived stress through streetscape design and provides feasible, quantitative recommendations for future university campus design.

References Asadi-Shekari Z, Moeinaddini M, Shah MZ (2014) A pedestrian level of service method for evaluating and promoting walking facilities on campus streets. Land Use Policy 38:175–193. https:// doi.org/10.1016/j.landusepol.2013.11.007 Badland HM, Opit S, Witten K, Kearns RA, Mavoa S (2010) Can virtual streetscape audits reliably replace physical streetscape audits? J Urban Health 87(6):1007–1016. https://doi.org/10.1007/ s11524-010-9505-x Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495. https://doi.org/10.1109/TPAMI.2016.2644615 Baobeid A, Koç M, Al-Ghamdi SG (2021) Walkability and its relationships with health, sustainability, and livability: elements of physical environment and evaluation frameworks. Frontiers in Built Environment 7. https://doi.org/10.3389/fbuil.2021.721218 Bayram N, Bilgel N (2008) The prevalence and socio-demographic correlations of depression, anxiety and stress among a group of university students. Soc Psychiatry Psychiatr Epidemiol 43(8):667–672. https://doi.org/10.1007/s00127-008-0345-x Biljecki F, Ito K (2021) Street view imagery in urban analytics and GIS: a review. Landsc Urban Plan 215:104217. https://doi.org/10.1016/j.landurbplan.2021.104217 Breiman L (2001) Random forests. Mach Learn 45(1):5–32. https://doi.org/10.1023/A:101093340 4324 Breitung W (2012) Enclave urbanism in China: attitudes towards gated communities in Guangzhou. Urban Geogr 33(2):278–294. https://doi.org/10.2747/0272-3638.33.2.278 Chen C, Li H, Luo W, Xie J, Yao J, Wu L, Xia Y (2022) Predicting the effect of street environment on residents’ mood states in large urban areas using machine learning and street view images. Science of the Total Environment 816:151605. https://doi.org/10.1016/j.scitotenv.2021.151605 Cohen J (1988) Statistical power analysis for the behavioral sciences, 2nd ed. Routledge: New York, NY, USA Dai L, Zheng C, Dong Z, Yao Y, Wang R, Zhang X, Ren S, Zhang J, Song X, Guan Q (2021) Analyzing the correlation between visual space and residents’ psychology in Wuhan, China using street-view images and deep-learning technique. City and Environment Interactions 11:100069. https://doi.org/10.1016/j.cacint.2021.100069 Donderi DC (2006) An information theory analysis of visual complexity and dissimilarity. Perception 35:823–835. https://doi.org/10.1068/p5249

13 The Impact of Visual Character on Perceived Stress Levels …

243

Dubey A, Naik N, Parikh D, Raskar R, Hidalgo CA (2016) Deep learning the city: quantifying urban perception at a global scale. In: Leibe B, Matas J, Sebe N, Welling M (eds) Computer vision—ECCV 2016. Springer International Publishing, Cham, pp 196–212 Evans GW (2003) The built environment and mental health. J Urban Health. https://doi.org/10. 1093/jurban/jtg063 Fernandez-Delgado M, Cernadas E, Barro S, Amorim D (2014) Do we need hundreds of classifiers to solve real world classification problems? Journal of Machine Learning Research Grahn P, Stigsdotter UA (2003) Landscape planning and stress. Urban Urban Greening 2(1):1–18. https://doi.org/10.1078/1618-8667-00019 Han X, Wang L, Seo SH, He J, Jung T (2022) Measuring perceived psychological stress in urban built environments using google street view and deep learning. Front Public Health 10:891736. https://doi.org/10.3389/fpubh.2022.891736 He N, Li G (2021) Urban neighbourhood environment assessment based on street view image processing: a review of research trends. Environmental Challenges 4:100090. https://doi.org/ 10.1016/j.envc.2021.100090 He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778. http://doi.org/10. 1109/CVPR.2016.90 Hou Y, Biljecki F (2022) A comprehensive framework for evaluating the quality of street view imagery. Int J Appl Earth Obs Geoinf 115:103094. https://doi.org/10.1016/j.jag.2022.103094 Jiang X, Larsen L, Sullivan WC (2020) Connections between daily greenness exposure and health outcomes. Int J Environ Res Public Health 17(11):3965 Jones PJ, Park SY, Tyler Lefevor G (2018) Contemporary college student anxiety: the role of academic distress, financial stress, and support. J Coll Couns 21(3):252–264. https://doi.org/10. 1002/jocc.12107 Kan HY, Forsyth A, Rowe P (2017) Redesigning China’s superblock neighbourhoods: policies, opportunities and challenges. J Urban Des 6(22):757–777. https://doi.org/10.1080/13574809. 2017.1337493 Khan MJ, Altaf S, Kausar H (2013) Effect of perceived academic stress on students’ performance. FWU Journal of Social Sciences 7(2):146–151 Lee JH, Ostwald MJ (2021) Fractal dimension calculation and visual attention simulation: assessing the visual character of an architectural façade. Buildings 11:163. https://doi.org/10.3390/buildi ngs11040163 Lee JH, Ostwald MJ (2023) The “visual attractiveness” of architectural facades: measuring visual complexity and attractive strength in architecture. Architectural Science Review 66(1):42–52. https://doi.org/10.1080/00038628.2022.2137458 Li D, Sullivan WC (2016) Impact of views to school landscapes on recovery from stress and mental fatigue. Landscape Urban Plan 149–158. https://doi.org/10.1016/j.landurbplan.2015.12.015 Li W-Y, Chiou S-C, Huang B-X (2022a) The sense of safety and active leisure in gated enclaves: evidence from Fuzhou University campus. Sustainability 14(13):7784 Li Y, Peng L, Wu C, Zhang J (2022b) Street view imagery (SVI) in the built environment: a theoretical and systematic review. Buildings 12:1167. https://doi.org/10.3390/buildings12081167 Li Z, Huang X, White M (2022c) Effects of the visual character of transitional spaces on human stress recovery in a virtual reality environment. Int J Environ Res Public Health 19(20):13143. https://doi.org/10.3390/ijerph192013143 Lindberg CM, Srinivasan K, Gilligan B, Razjouyan J, Lee H, Najafi B, Canada KJ et al (2018) Effects of office workstation type on physical activity and stress. Occup Environ Med 75(10):689–695. https://doi.org/10.1136/oemed-2018-105077 Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 3431–3440 Ma L, Ye R, Ettema D, Van Lierop D (2023) Role of the neighborhood environment in psychological resilience. Landsc Urban Plan 235:104761. https://doi.org/10.1016/j.landurbplan.2023.104761

244

Z. Li et al.

Meng R, Li J, Wang Z, Zhang D, Liu B, Luo Y, Hu Y, Yu C (2020) The Chinese version of the perceived stress questionnaire: development and validation amongst medical students and workers. Health and Quality of Life Outcomes 18(1):70. https://doi.org/10.1186/s12955-02001307-1 Parsons R (1991) The potential influences of environmental perception on human health. J Environ Psychol 11(1):1–23. https://doi.org/10.1016/S0272-4944(05)80002-7 Pascoe MC, Hetrick SE, Parker AG (2020) The impact of stress on students in secondary school and higher education. Int J Adolesc Youth 25(1):104–112. https://doi.org/10.1080/02673843. 2019.1596823 Peacock EJ, Wong PTP (1990) The stress appraisal measure (SAM): a multidimensional approach to cognitive appraisal. Stress Medicine 6(3):227–236. https://doi.org/10.1002/smi.2460060308 Quercia D, O’Hare NK, Cramer H (2014) Aesthetic capital: what makes London look beautiful, quiet, and happy? In: Proceedings of the 17th ACM conference on computer supported cooperative work & social computing. Association for Computing Machinery, Baltimore, pp 945–955 Rundle AG, Bader MD, Richards CA, Neckerman KM, Teitler JO (2011) Using Google Street View to audit neighborhood environments. Am J Prev Med 40(1):94–100. https://doi.org/10.1016/j. amepre.2010.09.034 Shields GS, Slavich GM (2017) Lifetime stress exposure and health: a review of contemporary assessment methods and biological mechanisms. Soc Pers Psychol Compass 11(8):e12335. https://doi.org/10.1111/spc3.12335 Siriaraya P, Wang Y, Zhang Y, Wakamiya S, Jeszenszky P, Kawai Y, Jatowt A (2020) Beyond the shortest route: a survey on quality-aware route navigation for pedestrians. IEEE Access 8:135569–135590. https://doi.org/10.1109/ACCESS.2020.3011924 Stamps AE, Smith S (2002) Environmental enclosure in urban settings. Environ Behav 34(6):781– 794. https://doi.org/10.1177/001391602237246 Sun C, Cheng J, Lin A, Peng M (2018) Gated university campus and its implications for sociospatial inequality: evidence from students’ accessibility to local public transport. Habitat Int 80:11–27. https://doi.org/10.1016/j.habitatint.2018.08.008 Ulrich RS (1993) Biophilia, biophobia, and natural landscapes. In: Kellert SR, Wilson EO (eds) The biophilia hypothesis. Island Press, Washington DC, pp 73–137 Wu J, Feng Z, Peng Y, Liu Q, He Q (2019) Neglected green street landscapes: a re-evaluation method of green justice. Urban Forestry & Urban Greening 41:344–353. https://doi.org/10.1016/j.ufug. 2019.05.004 Yao Y, Liang Z, Yuan Z, Liu P, Bie Y, Zhang J, Wang R, Wang J, Guan Q (2019) A human-machine adversarial scoring framework for urban perception assessment using street-view images. Int J Geogr Inf Sci 33(12):2363–2384. https://doi.org/10.1080/13658816.2019.1643024 Yao W, Zhang X, Gong Q (2021) The effect of exposure to the natural environment on stress reduction: a meta-analysis. Urban Forestry & Urban Greening 57:126932. https://doi.org/10. 1016/j.ufug.2020.126932 Yin J, Yuan J, Arfaei N, Catalano PJ, Allen JG, Spengler JD (2020) Effects of biophilic indoor environment on stress and anxiety recovery: a between-subjects experiment in virtual reality. Environ Int 136:105427. https://doi.org/10.1016/j.envint.2019.105427 Zhong R, Zhao W, Zou Y, Mason RJ (2018) University campuses and housing markets: evidence from Nanjing. Prof Geogr 70(2):175–185. https://doi.org/10.1080/00330124.2017.1325750 Zhou B, Zhao H, Puig X, Fidler S, Barriuso A, Torralba A (2017) Scene parsing through ADE20K dataset. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Honolulu, pp 5122–5130. https://doi.org/10.1109/CVPR.2017.544 Zhu X, Lu Z, Yu C, Lee C, Mann G (2013) Walkable communities: impacts on residents’ physical and social health. World Health Design 6(3):68–75

Chapter 14

Design Education and Learner Experiences in a Computer-Mediated Environment Myung Eun Cho and Mi Jeong Kim

Abstract Various online class types have changed student learning activities, exchanges with classmates online, and community formation during the pandemic. This chapter investigates the experience of learners in online design education and explored alternative tools for design education in a computer-mediated environment. Two studies have been conducted sequentially. It first identifies the problems associated with the switch in learning modalities by performing a questionnaire survey. This chapter then analyses how to overcome the problems by using new web-based tools in collaboration and communication through an experiment. The mediated environment by Miro has supported the design process in terms of learners’ set-up goals, reinterpretation of problems, cognitive engagement, and cognitive synchronization. This chapter demonstrates the possibility of effective media for online classes to overcome the limitations of studio-centred design education. Keywords Learning experience · Web-based tool · Design education · Miro · Presence · Problem-solving

14.1 Introduction Over the past decades, universities have been trying to improve access to classes and the efficiency of teaching and learning based on Information and Communications Technology (ICT). While acknowledging the need for ICT-based classes to enhance quality and strengthen competitiveness, Korean universities tended not to employ them actively (Kim and Park 2020). This conventional perception changed with the spread of COVID-19. Due to the rapid transition to remote education, instructors and students faced challenges in conducting their teaching and learning M. E. Cho Construction Research Institute, Hanyang University, Seoul 04763, Republic of Korea M. J. Kim (B) School of Architecture, Hanyang University, Seoul 04763, Republic of Korea e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 J. H. Lee et al. (eds.), Multimodality in Architecture, https://doi.org/10.1007/978-3-031-49511-3_14

245

246

M. E. Cho and M. J. Kim

activities. Online class competency has become a significant issue in enhancing the quality of university-level education. This emerging landscape of online education has, consequently, garnered attention from a wider range of research communities, addressing the development of appropriate learning models and teaching methods using technology applications (Choi et al. 2022; Park and Han 2020). Design education at universities had rarely been conducted online due to the nature of studio classes. However, due to the impact of COVID-19, architectural education has also been converted to an online format. Teachers and students face new educational challenges in such online formats. Over the past few years, an online LMS (Learning Management System) environment has been employed to run online classes, and architecture departments such as Harvard University and MIT have integrated highly functional web tools such as Miro and Figma into studio classes. The integration of offline and online classes was explored, and methods for smooth convergence of augmented interactions between teachers and students were sought. Notably, the positive aspects of the online class experience were evaluated (Al Maani et al. 2021; Megahed and Hassan 2022; Milovanovi´c et al. 2020). By using computers, laptops, tablets and mobile phones, a flexible learning environment is created in which learners can access materials, share ideas, and engage in online discussions regardless of time and distance. This chapter investigated learners’ experiences in online design education and explored alternative tools for design education in a computer-mediated environment through two studies. The first study identified the problems associated with the switch in learning modalities through a questionnaire survey. The second study analysed how to overcome the problems by using new web-based tools through an experiment.

14.2 Background 14.2.1 Presence in a Computer-Mediated Environment Presence in a learning space through mediated communication is recognised as an important predictor of perceived learning outcomes and learning satisfaction (Gunawardena and Zittle 1997; Ijsselsteijn and Riva 2003). Presence in a computermediated environment supports the cognitive goals of learning through ongoing interaction, and a lack of such can lead to student frustration and low levels of emotional learning. Presence is therefore considered a critical issue in improving effective education in online learning environments (Garrison et al. 2001; Kreijns et al. 2022). The concept of presence is very broad and diverse. The initial definition included many technical aspects, such as the degree to which input and output channels match between machine and human. There has recently been a shift from technologyfocused definitions confined to remote operating systems to a broader psychological understanding (Lee 2004; Lombard and Ditton 1997). Presence encompasses

14 Design Education and Learner Experiences in a Computer-Mediated …

247

several dimensions, including telepresence, spatial presence, social presence, and co-presence (Nowak and Biocca 2003; Slater and Wilbur 1997). Spatial presence is defined as “the user’s subjective feeling of being there,” in the space displayed by the medium (Slater and Wilbur 1997). Since spatial presence is understood as the experience of a user located in a media environment, it is assumed that this can occur not only in a virtual reality environment but also in video games, movies, and books (Wirth et al. 2012). The generation of spatial presence not only makes the user feel as if they are inside the computer-mediated environment but also makes them aware of their action potential in such; users feel like they can actually participate in the action of the media presentation, rather than simply observing it (Kreijns et al. 2022). The term co-presence means that one is perceiving the other and the other is actively perceiving the self. In a more expanded sense, it means that interactors can approach each other through perception and continue a meaningful relationship through interaction (Nowak and Biocca 2003). Short et al. (1976) defined social presence as “the importance of other people in mediated communication and consequent interpersonal interactions.” From the perspective of social cognitive theory, social presence is an essential factor in facilitating learning interactions in online environments (Oh et al. 2018). According to this theory, people learn not only from their own experiences but from the observation, imitation, and modelling of others, and their interactions with others adjust their behaviour and thoughts (Ferguson 2010). Accordingly, others can become peers who help develop cognitive, emotional, and mental abilities.

14.2.2 The Community of Inquiry (CoI) Framework Garrison et al. (2000) proposed the CoI (Community of Inquiry) model to support the construction of online education and blended (face-to-face and non-face-toface) educational experiences. The CoI model supports critical thinking, inquiry, and discourse between students and teachers, and serves as a theoretical framework for designing optimal online learning environments. In the past decades, many researchers have been preoccupied with issues such as e-learning or measuring student satisfaction with technology or communication behaviour. The CoI framework has been the basis for a significant number of online education studies (Caskurlu et al. 2021; Kozan and Caskurlu 2018), and thus has had a significant impact on online education research and practice. The CoI model describes the educational experience of learners across three dimensions: social presence, cognitive presence, and teaching presence. Teaching presence means designing, facilitating, and guiding learners’ cognitive and social processes so that individual learners can realise educationally meaningful learning effects (Garrison and Arbaugh 2007). Instructional presence refers to the instructor’s role in class design, operation, support, and management. It includes the instructor’s coordination ability and sense of instructional presence, which is subdivided into the

248

M. E. Cho and M. J. Kim

categories of instructional management, building understanding, and direct instruction. It refers to the learner’s perception of the aspects that instructional designers plan, design, and organize, in terms of the structure, interaction, and evaluation of learning contents or activities. Garrison et al. (2001) argued that learners should be directly guided to understand learning contents, carry out learning activities, and use various information meaningfully to create new knowledge. Social presence is the ability of learners to integrate themselves into a community of inquiry, to communicate intentionally, and to develop interpersonal relationships in ways that project their personalities (Garrison and Arbaugh 2007). It is subdivided into the categories of affective expression, open communication, and group cohesion. Collaborative activities among learners provide greater opportunities to increase social presence and enhance online community (Richardson and Swan 2019). Among the three elements of the CoI framework, social presence has been studied the most extensively, and it is well-established that various activities that promote social presence improve learners’ satisfaction (Arbaugh and Benbunan-Fich 2006). Cognitive presence refers to the degree to which learners can construct and confirm meaning and knowledge based on continuous reflection and dialogue (Garrison et al. 2001). Learners enhance their cognitive presence and critical thinking skills through interaction with each other. Cognitive presence is divided into the categories of the triggering event, exploration, integration, and resolution (Duphorne and Gunawardena 2005). Architecture education has traditionally followed a studio method that induces learners to solve creative problems. With the help of an expert instructor, students acquire knowledge and skills through internal thinking and problem-solving (Savery and Duffy 1995). In contrast to structured problem situations, in which solutions to individual problems exist objectively, active learning is achieved in the process of solving unstructured problem situations, presented to learners on their own or in teams to improve design expertise (Gallagher and Stepien 1996). Problem-based learning requires both self-directed and cooperative learning at the team level. A team member must actively participate in the team’s decision as a member of the team to which he or she belongs and proceed with self-directed and active tasks and activities. In design education dealing with unstructured problem situations, the power of the group can increase the learning effect. Active dialogue and exchange of opinions through communities of practice promote understanding and resolution of unstructured problem situations. The online CoI originates from the constructivist approach. To facilitate successful educational experiences in online learning, supporting collaborative learning and discourse in the learner community is essential, which is also significantly related to improving students’ perceived learning abilities (Garrison and Arbaugh 2007). The CoI framework provides insights and methodologies for online learning and is a useful tool for in-depth analysis of the complexity of online learning. The online education experience of students was analysed using the CoI framework in this context.

14 Design Education and Learner Experiences in a Computer-Mediated …

249

14.2.3 Creative Design Problem-Solving Process In design, “creative” is usually referred to as an evaluation of the final product. However, cognitive psychology focuses on design activities, associating them with specific processes that have the potential to produce creative outcomes (Gero and McNeill 1998). Creative design is a kind of non-routine design process in which important events or unexpected results occur (Cross et al. 1996). Designers discover hidden features in proposed representations and perceive key concepts with sudden insights essential to creative design (Suwa et al. 1998). Creative thinking is closely related to problem exploration, which provides more opportunities for insight. For a designer to find a satisfactory solution by being away from the ambiguity of the initial design problem, it is essential to explore a problem that defines new constraints during the design process (Oxman 2008). The process of co-evolution is critical to this problem-finding behaviour. Cross et al. (1996) explain that creative design can be created in the process of co-evolution of both problem space and solution space. The term co-evolution was originally used to describe the process in which two or more species evolve and develop, depend on each other very closely, and interact with each other. In design, design, problems, and solutions do not develop independently but approach problem-solving by closely influencing each other (Maher and Tang 2003). Co-evolution is an iterative process of constant analysis, measurement, and evaluation between the problem area and the solution area. According to the concept of co-evolution, the designer’s perception of the problem situation continues to change, and the designer constantly reconstructs the problem until a satisfactory goal is reached. In the course of problem–solution iterations, the designer suddenly arrives at the solution without realising how the solution was found (Cross et al. 1996). The interaction between the problem domain and the solution domain in the design process is a characteristic of the creative problem-solving process. In the process of problem exploration and co-evolution, cooperation at the team level is much more advantageous than individual cooperation because collaboration can be a driving force to enable efficient design. Cognitive synchronization is noted as a cognitive activity to understand the designer’s spatial awareness in collaborative design. Cognitive synchronization referred to as “clarification,” “analyses,” and “problem clarifying” (d’Astous et al. 2004) is part of the process of negotiating through arguments and discussions among peers; designers spend much time in actual work on cognitive synchronization (Shum and Hammond 1994). In the process of cognitive synchronization, a designer proposes an idea, and the proposed idea is either adopted or not adopted into the design according to certain constraints after repeated discussion. Cognitive synchronization is essential for collaborative design (Stempfle and Badke-Schaub 2002).

250

M. E. Cho and M. J. Kim

14.3 Methodology This chapter begins by investigating the experience of online classes from the perspective of learners and, based on the challenges identified, then explores a process for finding effective ways to use media that can increase the effectiveness of design classes. The first study analysed the changes in learning experienced by students in the Department of Architecture and identified the problems associated with all classes at universities being suddenly replaced with online lectures as part of the response to COVID-19 (Cho et al. 2023). During the pandemic, domestic universities supported educational activities through LMS learning platforms. LMS is a learning management system for learning activities and management of learners in an online learning environment. It provides a learning environment such as the formation of online classes, creation and sharing of lecture contents, management of student attendance and grades, bulletin board, and Q&A. The instructors in the LMS system can track the learning process and manage the learning history. In addition, real-time video conferencing platforms such as Zoom, Cisco, and Webex were introduced to enable mutual communication in online lectures. Social networks were also used to construct an interaction and sharing system between teachers and students, and between students and students. During the data collection, classes were conducted in the form of recorded lectures and real-time video lectures, but the studio, a practical class, was conducted using a mix of face-to-face and online sessions in line with the nature of the lecture. This chapter investigated learners’ online learning experience using the CoI framework. The second study analysed case studies of third and fourth-year students in the Department of Architecture at H University in Korea. Using Miro, a web-based collaboration tool, the participants were divided into two groups of three, and the design assignments were carried out separately in three separate offices. First, the functions of the screen and various icons in Miro were explained. Students used each function to become familiar with the Miro environment and had a Q&A session. In the next step, a design task was performed by the group in Miro which took a total of 50 min. The specific details of the design assignments are shown in Table 14.1. A survey was conducted to investigate their sense of presence and the design problem-solving process; their experiences in Miro were additionally acquired through interview. An evaluation tool was developed to measure students’ experiences when performing design tasks in a computer-mediated environment. The evaluation criteria are largely divided into spatial presence, co-presence, social presence, and the design problem-solving process (see Table 14.2). Spatial presence consists of self-location, possible action, attention to environment, and interface awareness. Through spatial presence, users feel as if they are physically in the media environment and recognise their action potential in the mediated environment. A basic precondition for experiencing spatial presence is interest in the medium. Users who pay attention to the mediated environment can experience a high sense of presence (Slater and Wilbur 1997). The user’s interest in the stimulus

14 Design Education and Learner Experiences in a Computer-Mediated …

251

Table 14.1 Design task Design Task You are required to design the H University Library. The design should be a state-of-the-art library in which students can actively communicate, leading to the activation of communities within the university. As a team of three, you need to solve the given design problem while freely exchanging opinions with each other Design Specification and Submission Sheet – The site is located within H University, and the location and size are freely determined – The library should provide not only a space for reading, but also an integrated reading service that combines analogue and digital materials – We need a space for students to communicate and build a community – Please complete the design concept and floor plan (diagram) – Submission: 3–4 Miro boards and captured images

Table 14.2 Evaluation framework for learner experience Item

Contents

Spatial presence

Self-location Possible action Attention to environment Interface awareness

Co-presence

Perceived other’s presence Attentional allocation

Social presence

(Perceived) emotional interdependence (Perceived) behaviour interdependence Mutual understanding Mutual assistance

Design problem-solving

Set-up goal Unexpected discoveries Reinterpretation/revisit Engagement Cognitive synchronization

of the media has a positive effect on spatial presence. When interacting with the environment, presence can be enhanced if the interface is executed naturally. If the control mode is artificial, particularly in the environment, presence may decrease (Wirth et al. 2012). Co-presence consists of perceived others and attentional allocation. It is the degree to which one believes one is not alone or isolated and is entirely psychologically connected to others (Nowak and Biocca 2003). When individuals experience copresence, they pay attention to the other person and to their surroundings. Social

252

M. E. Cho and M. J. Kim

presence consists of perceived emotional interdependence, behavioural interdependence, and mutual understanding. Social presence ranges across interaction with others, psychological engagement, and joint behavioural engagement. High-level social presence is the degree of psychological engagement, access, and emotional state of others. It is characterised by empathy or sense and response, and emotional interdependence (Tu 2000). Design problem-solving consists of set-up goals, unexpected discoveries, reinterpretation, cognitive engagement, and cognitive synchronization. It includes the co-evolutionary process of recognising design problems and revising ideas through the setting of new goals related to learners’ creative problem-finding behaviours, unexpected discoveries, and behaviours of revisiting previously mentioned information or past activities. Reinterpretation of a problem reflects the designer’s change in perception of the problem situation and is a perceptual action that pays attention to suggestive spatial–temporal features (Kim and Maher 2008; Suwa et al. 1998). Cognitive synchronization is related to collective activity: proposing opinions, arguing, asking, and answering questions, and negotiating each other’s opinions on design proposals. Cognitive synchronization refers to the contentious process of collaborative design that includes solution proposals and various types of arguments (Maher and Tang 2003).

14.4 Findings 14.4.1 Learners’ Online Experience by Col Framework Students at the Department of Architecture who took online classes from the first semester of 2020 to the first semester of 2021 were targeted. Using the CoI framework consisting of teaching presence, social presence, and cognitive presence (for 27 items), synchronous and asynchronous online classes were quantitatively evaluated (see Table 14.3). The questionnaire items consisted of a five-point Likert-type scale ranging from 1 (I do not agree at all) to 5 (I fully agree). Figure 14.1 shows a comparison of the average ranks of the three presences. The average score was in the range of 3 points (3 points: average), implying a mediocre overall experience. Among these, students’ social presence was the lowest (M = 2.98, SD = 0.84), followed by cognitive presence (M = 3.64, SD = 0.78), while teaching presence had the highest evaluation score (M = 3.74, SD = 0.77). According to the teaching presence analysis, teacher support related to learning activities (e.g., subject goals, topics, learning activity participation methods, important deadlines, or schedules) was high, but this was insufficient to create an environment for cooperative learning. In addition, students had negative views about overly teacher-driven and inauthentic collaboration and discussion. They complained that the community was not running successfully. According to the social presence analysis, students thought that online communication was an excellent medium for social

14 Design Education and Learner Experiences in a Computer-Mediated …

253

Table 14.3 CoI results CoI (Community of Inquiry)

Total (n = 104)

Teaching presence

M

The instructor clearly communicated important course topics

3.88 0.82

The instructor provided clear instructions on how to participate in course learning activities

4.00 0.90

The instructor clearly communicated important due dates/time frames for learning activities

4.33 0.80

SD

The instructor was helpful in guiding the class toward understanding course topics 4.08 0.85 in a way that helped me clarify my thinking The instructor helped to keep course participants engaged and participating in productive dialogue

3.88 0.99

The instructor helped keep the course participants on task in a way that helped me to learn

3.38 1.15

Instructor actions reinforced the development of a sense of community among course participants

3.04 1.18

The instructor helped to focus discussion on relevant issues in a way that helped me to learn

3.38 1.15

The instructor provided feedback in a timely fashion

3.69 1.08

Social presence

M

Getting to know other course participants gave me a sense of belonging in the course

2.56 1.29

I was able to form distinct impressions of some course participants

3.23 1.09

Online or web-based communication is an excellent medium for social interaction

3.85 0.97

I felt comfortable conversing through the online medium

3.23 1.15

I felt comfortable participating in the course discussions

2.85 1.12

I felt comfortable interacting with other course participants

2.65 1.18

SD

I felt comfortable disagreeing with other course participants while still maintaining 2.17 1.16 a sense of trust I felt that my point of view was acknowledged by other course participants

3.31 0.99

Online discussions helped me to develop a sense of collaboration

2.96 1.11

Cognitive presence

M

Problems posed increased my interest in course issues

3.42 0.99

Course activities piqued my curiosity

3.46 1.03

I felt motivated to explore content-related questions

3.58 0.99

SD

I utilized a variety of information sources to explore problems posed in this course 3.94 1.05 Online discussions were valuable in helping me appreciate different perspectives

3.46 0.95

Learning activities helped me construct explanations/solutions

3.67 0.99

Reflection on course content and discussions helped me understand fundamental concepts in this class

3.71 0.88 (continued)

254

M. E. Cho and M. J. Kim

Table 14.3 (continued) CoI (Community of Inquiry)

Total (n = 104)

Teaching presence

M

I have developed solutions to course problems that can be applied in practice

3.77 0.89

I can apply the knowledge created in this course to my work or other non-class-related activities

3.75 1.00

SD

Fig. 14.1 Boxplot of CoI results

interaction, but that there was little social interaction in online classes. It was difficult to express themselves in actual online classes, so they were silent. Students who failed to build cohesive communities had an increased sense of isolation and disconnection. Students expected meaningful feedback from peers as well as personal guidance and feedback from teachers. They wanted to share their views with friends, seek their opinions, encourage each other, exchange information, and reflect on their learning and develop through this. Students were not used to communicating in online classes. According to the results of the survey, students valued non-verbal communication highly. In the case of face-to-face classes, non-verbal communication via a teacher’s gestures or actions is possible, increasing students’ feeling of interest in the class. However, in the case of online classes, it was difficult to feel this. Some students found it burdensome to turn on the camera; in some classes, there were many cases where only the speakers were turned on. Students did not want to collaborate online because collaboration or

14 Design Education and Learner Experiences in a Computer-Mediated …

255

discussion did not work properly in such environments. These results illustrate how important social exchange and interaction are for students participating in online learning. Interaction through the formation of a community of students can promote a positive online learning experience.

14.4.2 Exploring New Educational Strategies in Design Problem-Solving Using the evaluation tool, the degree of presence felt by the students in Miro and their experience in the design problem-solving process were analysed. Presence included a total of 23 questions, including 8 questions on spatial presence, 6 questions on copresence, and 9 questions on social presence. The design problem-solving process consisted of a total of 11 questions on a 7-point Likert scale (7 points: very much, 6 points: much, 5 points: a little, 4 points: normal, 3 points: not a little, 2 points: not much, 1 point: not at all). Figure 14.2 shows the results for each of spatial presence, co-presence, social presence, and the design problem-solving process by summing the scores for each item in each area. Spatial presence was the lowest at 5.85 points, and co-presence and social presence both recorded 6.30 points. The design problem-solving process scored 6.15 points. Students who performed design tasks in Miro had a high sense of co-presence and social presence. Regarding social presence, students influenced by other team members showed a high level of behavioural interdependence, understood each

Fig. 14.2 Average values of survey results by three presence levels and design problem-solving

256

M. E. Cho and M. J. Kim

other’s opinions and thoughts, and helped each other. Social presence has supported various behaviours related to design solving-problems. When performing design assignments, students’ experiences in the Miro environment were found to be very different from those in conventional face-to-face classes. First, what students mentioned as the biggest advantage in the Miro environment was parallel work. To perform assignments, students used Miro’s electronic board as a common workspace. While presenting and voicing their opinions, they drew pictures on the electronic board and wrote down words or simple key words that were mentioned in the discussion. Students said that simultaneous work was the biggest difference from the collaboration tools they had experienced before. While drawing freely with an electronic pen, they present their opinions, and were able to expand their design thinking by adding and modifying what other team members had drawn. The practice of drawing while speaking provides a form of flexibility that is important for early design activities. In addition, the touch-sensitivity of the electronic pen induces tactile sense and has a positive effect on spatial perception (Klemmer et al. 2008). Second, there was a fluid transition between the various electronic tools. It was found that students frequently used electronic tools (such as Google Search, PowerPoint, Illustrator) in the current teamwork. However, students noted as disadvantages of these tools that they are difficult to share among team members during the brainstorming process and that they are inconvenient to use in the design process because they are separated from class-related tools. On the other hand, the Miro environment provides some of the functions necessary for the initial concept stage of design and can be easily embedded and used so that the various flows are connected without interruption while students are performing work. For example, drawing tools often used by illustrators are also provided in Miro, so students could draw on the electronic board with familiarity. To find a reference useful for the assignment and share it with team members, they locate a website in the embedded Google and copy the URL information to the electronic board; the image can move immediately, and it is possible to annotate it as it is. It was found that the Miro environment smoothly supports the design process by enabling an uninterrupted user flow state for design thinking or problem-solving. Third, students cited easy access and exploration of design history as strengths. In the Miro environment, all work processes are documented from the earliest stages of design. Even after all the work is complete, it is possible to visually explore according to the timeline, and to modify the work contents. Thus, students could iteratively go back to previous records to check, correct, or reset goals. This facilitates input and output in the design solution process and enables comprehensive thinking by broadening the overall view in recognising problems or searching for solutions. Fourth, flexible interaction of remote teamwork was noted. While mind maps among team members were very useful in the initial design problem-solving process, it was found that students were not able to use mind maps in their usual teamwork. In the Miro environment, it was not difficult for team members to solve design tasks through mind maps. For example, it was possible to jointly sketch by exchanging opinions and erasing, drawing, and moving several people together on an electronic

14 Design Education and Learner Experiences in a Computer-Mediated …

257

board; through this work, it was possible to develop design work while continuing to share ideas. Whenever an annotation was added to the electronic board, the name of the team member was displayed, so it was easy to know who was working. Students sent emoticons or left notes in addition to verbal communication. Non-verbal communication was also conducted. Team members benefited from using the Miro board as a common workspace, allowing multiple people to simultaneously view, discuss, and modify designs. The work contents of the Miro board enabled focusing on detail through the zoom in and out functions.

14.5 Discussion The students’ online learning experiences can be characterised as follows: Various online class types: According to the results of previous research, students prefer synchronous video-communication, which allows them to recognise and respond easily to others (Borup et al. 2012; Cox et al. 2004). However, the first study shows that students preferred text-based recorded lectures in some cases. In the case of theoretical subjects and technical subjects, students preferred recorded lectures that allowed them to selectively listen to parts, thus they were able to effectively study lessons. Students noted that real-time video lectures are appropriate when the teacher’s immediate feedback, active participation and, interaction through collaboration or discussion are needed. When receiving guidance from a teacher in an online class, the shared drawings with other students were evaluated as a positive aspect. Many teachers were accustomed to offline classes, and there was significant system maladaptation; even in online classes, there were many cases where the lecture method and lecture notes were conducted like offline sessions. In this case, the quality of the class decreased, as did the satisfaction level. To conduct online classes, it is necessary to develop class contents and teaching methods differently. Changed students’ learning activities: In online classes, students adopted a different digital learning method. When taking classes, students open a new window and take notes in digital documents or save important parts as images while listening. They used the replay and capture functions, which are the biggest features of online classes. For example, for computer programming or building equipment, they were able to study efficiently while repeatedly listening to difficult parts through recorded lectures. As working on computers increased, the way of perceiving space also changed. Contrary to concerns, critique, which was shared on the screen during online classes, received positive reviews compared to other subjects. The teacher’s critiques through video-communication presentations and feedback were useful because students could see other students’ critiques. To maximize the learning effect, it is necessary to establish a technical learning management system based on the changed learning activities of students. Interacting with classmates online and building a community: Building a community of learners can be a powerful motivator to expand learning. Students valued the sense of community in the class. The activation of interaction in the

258

M. E. Cho and M. J. Kim

Fig. 14.3 Learner experience in the computer-mediated environment

LMS environment is mainly in the teacher–student connection, and the student– student connection was found to be lacking. This also affected learning activities. It was difficult for students to communicate in online classes. In the state of being connected via a camera and speaker, they could sense the presence of other students, but could not interact with them, or study together and ask each other questions about class content. In particular, the first and second-year students who started with the COVID-19 pandemic often had no group of friends with whom to listen to classes, share class contents, study together, or do assignments, so it was difficult to form a learner community. For students who had never experienced campus life, college life without peers was atrophied, and they became passive in class. Learners’ experiences in a computer-mediated environment, Miro, can be characterised as follows based on the result of the second study (Fig. 14.3): Flexible interactions, exploration of design history, parallel activities, and fluidly transition.

14.6 Conclusion This chapter described the students’ online learning experiences and explored the possibility of a new dedicated environment for design problem-solving. Using the CoI framework, this research found that the low sense of community caused by the weak social and spatial presence discouraged students’ social exchanges in online classes. A strategy for physically and virtually building a community of learners should be provided for the cooperative learning environment. Current online learning environments might not support the problem-solving process efficiently in design education.

14 Design Education and Learner Experiences in a Computer-Mediated …

259

Thus, this chapter explored how students solve collaboration and communicationcentred design problems using new web-based tools as a study on the technologycentred educational environment. The usability in design classes of a collaboration tool called Miro was verified as a computer-mediated environment with case studies. Miro is characterised by providing a sense of presence as a space rather than a mere learning tool and by enabling space-based collaboration as it allows users to exchange opinions without space and time constraints, such as leaving comments through the post-it function in addition to real-time collaboration. It is predicted to be advantageous for remote collaboration by forming a learning community among team members who do not know each other through communication and social presence. The Miro environment enables students to engage in problem-oriented creative design thinking in the early stages of concept design. In the future, it is necessary to conduct further studies including strategic approaches targeting more diverse tools and extensive cognitive analysis of learners with detailed design tasks. Acknowledgements This research was supported by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the of Education (2022R1I1A1A01065263). This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. NRF-2023R1A2C2004992).

References Al Maani D, Alnusairat S, Al-Jokhadar A (2021) Transforming learning for architecture: online design studio as the new norm for crises adaptation under COVID-19. Open House International 46(3):348–358. https://doi.org/10.1108/OHI-01-2021-0016 Arbaugh JB, Benbunan-Fich R (2006) An investigation of epistemological and social dimensions of teaching in online learning environments. Academy of Management Learning & Education 5(4):435–447. https://doi.org/10.5465/AMLE.2006.23473204 Borup J, West RE, Graham CR (2012) Improving online social presence through asynchronous video. The Internet and Higher Education 15(3):195–203. https://doi.org/10.1016/j.iheduc.2011. 11.001 Caskurlu S, Richardson JC, Maeda Y, Kozan K (2021) The qualitative evidence behind the factors impacting online learning experiences as informed by the community of inquiry framework: a thematic synthesis. Comput Educ 165:104111. https://doi.org/10.1016/j.compedu.2020.104111 Cho ME, Lee JH, Kim MJ (2023) Identifying online learning experience of architecture students for a smart education environment. Journal of Asian Architecture and Building Engineering 22(4):1903–1914. https://doi.org/10.1080/13467581.2022.2145216 Choi Y, Park N, Kim C (2022) The relationship between learning behaviour and academic achievements of pre-service teachers in online learning using LMS. The Korean Association of Information Education 26(2):85–95 Cox G, Carr T, Hall M (2004) Evaluating the use of synchronous communication in two blended courses. Computer Assisted Learning 20(3):183–193. https://doi.org/10.1111/j.13652729.2004.00084.x Cross N, Christiaans H, Dorst K (1996) Analysing design activity. Wiley d’Astous P, Détienne F, Visser W, Robillard P (2004) Changing our view on design evaluation meetings methodology: a study of software technical review meetings 25(6):625–655

260

M. E. Cho and M. J. Kim

Duphorne PL, Gunawardena CN (2005) The effect of three computer conferencing designs on critical thinking skills of nursing students. American Journal of Distance Education 19(1):37–50 Ferguson R (2010) Peer interaction: the experience of distance students at university level. Computer Assisted Learning 26(6):574–584. https://doi.org/10.1111/j.1365-2729.2010.00386.x Gallagher SA, Stepien WJ (1996) Content acquisition in problem-based learning. The Education of the Gifted 19(3):257–275. https://doi.org/10.1177/016235329601900302 Garrison DR, Anderson T, Archer W (2000) Critical inquiry in a text-based environment: computer conferencing in higher education. Internet and Higher Education 2(2):87–105 Garrison DR, Anderson T, Archer W (2001) Critical thinking and computer conferencing: a model and tool to assess cognitive presence. American Journal of Distance Education 15(1):7–23 Garrison DR, Arbaugh JB (2007) Researching the community of inquiry framework: review, issues, and future directions. The Internet and Higher Education 10(3):157–172. https://doi.org/10. 1016/j.iheduc.2007.04.001 Gero JS, McNeill T (1998) An approach to the analysis of design protocols. Des Stud 19(1):21–61 Gunawardena C, Zittle F (1997) Social presence as a predictor of satisfaction within a computer mediated conferencing environment. American Journal of Distance Education 11(3):8–26 Ijsselsteijn W, Riva G (2003) Being there: the experience of presence in mediated environments. In: Riva FDG, IJsselsteijn WA (eds) Being there: concepts, effects and measurements of user presence in synthetic environments. Studies in new technologies and practices in communication. IOS Press, Amsterdam, pp 3–16 Kim J, Park J (2020) Text-mining analysis for university instructors experiences in the transition to online teaching. Educ Technol 36(3):839–871 Kim MJ, Maher ML (2008) The impact of tangible user interfaces on spatial cognition during collaborative design. Des Stud 29(3):222–253. https://doi.org/10.1016/j.destud.2007.12.006 Klemmer SR, Everitt KM, Landay JA (2008) Integrating physical and digital interactions on walls for fluid design collaboration. Human-Computer Interaction 23:138–213 Kozan K, Caskurlu S (2018) On the Nth presence for the community of inquiry framework. Comput Educ 122:104–118. https://doi.org/10.1016/j.compedu.2018.03.010 Kreijns K, Xu K, Weidlich J (2022) Social presence: conceptualization and measurement. Educ Psychol Rev 34(1):139–170. https://doi.org/10.1007/s10648-021-09623-8 Lee KM (2004) Presence, explicated. Commun Theory 14(27):27–50. https://doi.org/10.1111/j. 1468-2885.2004.tb00302.x Lombard M, Ditton T (1997) At the heart of it all: the concept of presence. Computer-Mediated Communication 3(2):JCMC321. https://doi.org/10.1111/j.1083-6101.1997.tb00072.x Maher M, Tang H-H (2003) Co-evolution as a computational and cognitive model of design. Res Eng Design 14(1):47–64. https://doi.org/10.1007/s00163-002-0016-y Megahed N, Hassan A (2022) A blended learning strategy: reimagining the post-Covid-19 architectural education. Archnet-IJAR: International Journal of Architectural Research 16(1):184–202. https://doi.org/10.1108/ARCH-04-2021-0081 - c A, Peši´c M, Bugarski J, Todorovi´c D, Sokolovi´c Milovanovi´c A, Kosti´c M, Zori´c A, Ðordevi´ N, Josifovski A (2020) Transferring COVID-19 challenges into learning potentials: online workshops in architectural education. Sustainability 12(17):7024. https://doi.org/10.3390/su1 2177024 Nowak KL, Biocca F (2003) The effect of the agency and anthropomorphism on users’ sense of telepresence, copresence, and social presence in virtual environments. Presence 12(5):481–494. https://doi.org/10.1162/105474603322761289 Oh CS, Bailenson JN, Welch GF (2018) A systematic review of social presence: definition, antecedents, and implications. Frontiers in Robotics and AI 5. https://doi.org/10.3389/frobt. 2018.00114 Oxman R (2008) Digital architecture as a challenge for design pedagogy: theory, knowledge, models and medium. Des Stud 29(2):99–120. https://doi.org/10.1016/j.destud.2007.12.003

14 Design Education and Learner Experiences in a Computer-Mediated …

261

Park S, Han S (2020) Differences of learning motivation among university students according to their previous online learning experience under the COVID-19 situation. Leaner-Centered Curriculum and Instruction 20(21):39–48 Richardson JC, Swan K (2019) Examining social presence in online courses in relation to students’ perceived learning and satisfaction. Asynchronous Learning Networks 7(1):68–88. https://doi. org/10.24059/olj.v7i1.1864 Savery JR, Duffy TM (1995) Problem based learning: an instructional model and its constructivist framework. Educ Technol 35(5):31–38 Short J, Williams E, Christie B (1976) The social psychology of telecommunications. Wiley, London Shum SB, Hammond N (1994) Argumentation-based design rationale: what use at what cost? Int J Hum Comput Stud 40(4):603–652. https://doi.org/10.1006/ijhc.1994.1029 Slater M, Wilbur S (1997) A framework for immersive virtual environments five: speculations on the role of presence in virtual environments. Presence: Teleoperators and Virtual Environments 6(6):603–616. https://doi.org/10.1162/pres.1997.6.6.603 Stempfle J, Badke-Schaub P (2002) Thinking in design teams—an analysis of team communication. Des Stud 23(5):473–496. https://doi.org/10.1016/S0142-694X(02)00004-2 Suwa M, Purcell T, Gero J (1998) Macroscopic analysis of design processes based on a scheme for coding designers’ cognitive actions. Des Stud 19(4):455–483. https://doi.org/10.1016/S0142694X(98)00016-7 Tu CH (2000) On-line learning migration: from social learning theory to social presence theory in a CMC environment. Network and Computer Application 23(1):27–37. https://doi.org/10.1006/ jnca.1999.0099 Wirth W, Hofer M, Schramm H (2012) The role of emotional involvement and trait absorption in the formation of spatial presence. Media Psychol 15(1):19–43. https://doi.org/10.1080/15213269. 2011.648536

Chapter 15

Pattern Languages: Concepts and Applications in Design for Ageing and Design Education Michael J. Dawes

Abstract Christopher Alexander’s A Pattern Language codifies design problems and their associated solutions that occur frequently throughout the built environment with the intent of empowering people to shape the spaces they inhabit. However, despite the broad appeal of pattern languages, Alexander’s theory has had relatively little impact on architectural practice and education, and is also the subject of extensive criticism. Yet despite this, Alexander’s patterns remain an intriguing concept for architectural scholars who continue to develop new pattern languages because the pattern format remains a convenient and compelling means for recording and sharing socio-spatial information. This chapter undertakes a qualitative and non-exhaustive literature review of “new” architectural pattern languages which results in the identification of several strengths, weaknesses, and potential lessons for developing new languages. The primary contributions of this chapter are providing a concise review of the principles and concepts of pattern languages, and a discussion of the potential for developing new architectural patterns in design education and residential aged care applications. Keywords Christopher Alexander · A pattern language · Patterns for ageing · Patterns for design education

15.1 Introduction Throughout much of the history of architecture, numerous buildings were created using local natural materials and unspecialised human labour. Christopher Alexander argues that this traditional method of production is typically iterative, taking place over a long period of time during which the design undergoes numerous, almost constant, revisions, which enable the users to ensure their building slowly adapts M. J. Dawes (B) School of Built Environment, Faculty of Arts, Design and Architecture, The University of New South Wales, Sydney, NSW 2052, Australia e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024 J. H. Lee et al. (eds.), Multimodality in Architecture, https://doi.org/10.1007/978-3-031-49511-3_15

263

264

M. J. Dawes

to their changing circumstances. This process allegedly leads to a unique but difficult to define property that Alexander somewhat interchangeably refers to as “life”, “beauty”, or “the quality without a name”. Alexander also alleges that this property is largely absent from architecture created within the last one hundred or so years. The absence of this “life” in contemporary design inspired Alexander to devote his career to the study of architectural aesthetics and to produce three unique but closely related theories of architecture (Grabow 1983). The focus of this chapter is Alexander’s second theory of architecture, primarily published in three texts, The Timeless Way of Building (Alexander 1979), A Pattern Language (Alexander et al. 1977), and The Oregon Experiment (Alexander et al. 1975). This chapter aims to examine a sample of “new” architectural pattern languages to identify strengths, weaknesses, and potential lessons that can support the development of new pattern languages for use in design education and accommodation for the elderly. In the context of this chapter, “new” architectural pattern languages are those developed following the publication of Alexander’s canonical language. The premise of A Pattern Language is that certain design challenges occur repeatedly throughout the built environment and that it is possible to find an ideal, but generic, solution which can be utilised any time a specific instance of those problems occurs. Alexander proposed to achieve this by developing a novel format for recording design situations that incorporates philosophical, social, and architectural principles into a brief but compelling description of the problem and its solution. This consolidated format contributed to the immense popularity of A Pattern Language, which is believed to be the most widely read architectural treatise ever published (Saunders 2002b). However, despite this popularity, A Pattern Language has had relatively little impact on the architectural profession, the canonical texts are widely criticised, and Alexander himself rejected the theory to pursue his third theory of architecture (Marshal 2012). Despite this, the fundamental premise of A Pattern Language retains significant value, and both scholars of architecture, and other fields, continue to develop new pattern languages. However, to fully understand A Pattern Language and its potential future applications, it is necessary to understand Alexander’s first theory of architecture, as many of the concepts in this theory are present in A Pattern Language, albeit having evolved in subtle but important ways. The next portion of this chapter reviews the origins, principles, and concepts of pattern languages. Then, the chapter undertakes a non-exhaustive literature review and qualitative analysis of new architectural pattern languages to identify several lessons for developing new architectural pattern languages. Finally, these lessons are discussed in the context of design education and accommodation for older people.

15.2 Background Alexander’s doctoral thesis, later published as Notes on the Synthesis of Form (Alexander 1964), can be viewed as his first theory of architecture, an attempt to develop a process through which a designer could create new buildings that capture

15 Pattern Languages: Concepts and Applications in Design for Ageing …

265

the unique character of traditional architecture. Many of the fundamental concepts that underpin this text are also found in a modified form in Alexander’s later theories. Therefore, an understanding of Alexander’s early work provides a robust foundation for understanding some of the more ambiguous and amorphous aspects of A Pattern Language. At this point in his career, Alexander believed that the dissatisfying nature of contemporary environments could be attributed to the presence of irritating and contradictory design variables that he refers to as “misfits”. A misfit is a design variable that fails to achieve harmony with the remainder of the design and the context in which it is located, thus becoming an irritant to the user. A design variable could relate to any aspect of the design and may include but are not limited to metric dimensions, colour or material choices, particular forms, or more philosophical concepts such as a human need to share a meal among friends. A single misfit could undermine the entire ensemble of the design and its context, thereby preventing the environment from achieving the same quality that he admired in traditional design. Alexander argues that the quality he favours in traditional design is the result of two factors, the first being that a design user will undertake direct and immediate action to rectify any misfit. Direct and immediate action is facilitated through the availability and obtainability of locally sourced materials and the use of materials with limited durability, requiring users to regularly undertake remedial actions. The second is that the actions undertaken to correct misfits are guided by strong sociocultural traditions, which serve to limit remedial actions to those approved by society and thereby prevent the emergence of radical actions. These two factors allegedly produce a society in which design is a slow but continual process of evolution which is relatively homeostatic, self-organising, and quick to achieve a state of equilibrium. A society that Alexander describes as being “unself-conscious”. In contrast, Alexander argues that it is impossible for users to take direct remedial action in contemporary society, and that the guiding traditions have broken down or are no longer respected. Three major factors allegedly prevent users from taking direct and immediate remedial actions; the first is that contemporary designs are typically constructed using specialised labour, and many users lack the skill to take action on their own. The second is that contemporary design makes extensive use of durable materials, meaning that once built, it becomes difficult for users to take remedial action, and the third is that certain legal structures of ownership prevent users from acting without the approval of third parties. Meanwhile, Alexander also argues that contemporary designers have become dedicated, self-conscious form makers who are no longer bound by traditions. Rather, contemporary designers intervene to alter design variables guided by academic theories, conceptual models, and the need for publishable imagery, regardless of whether the design variables they are altering exhibit misfit, or good fit. Subsequently, contemporary designers risk creating more new misfits than they rectify. The contemporary designer must also address a much larger number of design variables compared to traditional designers, and this makes contemporary design a much more complex undertaking. The combination of these variables makes it almost impossible for contemporary design to purposefully reproduce the qualities of traditional design.

266

M. J. Dawes

Alexander’s solution to misfits arising from contemporary societies focuses upon the application of mathematics, specifically, set theory and graph theory, and combining computational analysis with the intuition of the designer. Set theory, developed by Cantor (1874), involves the study of classification and a set can contain any number of diverse objects, including other subsets of objects. Graph theory, pioneered by Euler (1736), involves the study of relationships between objects and can be represented graphically, whereby an object is a node, and a relationship is a link. By combining these two branches of mathematics, Alexander seeks to break a design problem into its component parts. This process identifies and represents every design variable in a project and every relationship between these design variables. This allows Alexander to represent the underlying structure of a design problem as a series of nodes and connections (Fig. 15.1). The next stage in Alexander’s process involves using computational analysis to organise the design variables into a treelike hierarchy to guide the designer on which misfits to address first. Here, the designer would turn to their intuition to create three diagrams for each design variable. The first diagram represents physical properties, such as the geometry of the design variable, and the second diagram illustrates the functional requirements of the design variable, such as a need to control access to a space. The final diagram combines elements of the first two and is effectively the misfit eliminating solution for the design variable, which Alexander calls a “constructive diagram”. When the designer has created constructive diagrams for all design variables, these are synthesised into larger constructive diagrams following the hierarchical tree structure. Alexander alleges that it would then be a simple matter to translate the synthesised diagram into a completed building. Alexander illustrates this conceptualisation of the structure of design problems with a description of a series of interconnected lights, that effectively transplants William Ashby’s Design for a Brian: The Origins of Adaptive Behaviour (Ashby 1960) into an architectural context. In this description, a misfit is represented by an

Fig. 15.1 Design variables envisioned as a series of nodes and connections

15 Pattern Languages: Concepts and Applications in Design for Ageing …

267

illuminated light, and design variables exhibiting “good fit” are defined by dimmed lights. Alexander’s objective is to eliminate all misfits and therefore, to cause all lights to become dimmed. However, eliminating misfits is not a simple process. Due to the connections between lights, eliminating a misfit in one design variable can change the design in ways that cause new misfits in connected design variables, or in terms of Alexander’s lights, dimming one light has the potential to cause another already dimmed light to illuminate. Because of this: [n]o complex system will succeed in adapting in a reasonable amount of time unless the adaptation can proceed subsystem by subsystem, each subsystem relatively independent of the others (Alexander 1964, p. 41)

Therefore, Alexander’s proposal is to identify and then analyse the structure of the design problems to identify highly connected design variables that form sub-clusters within the larger structure and to treat these as sub-problems to be solved. Due to there being relatively few connections between these highly connected clusters of design variables, the designer could focus on solving sub-problems with little risk of creating new misfits in other clusters, effectively simplifying the design problem into manageable components. Alexander also uses this model of connected design variables to illustrate his understanding of traditional and contemporary design differences. The members of a traditional society will identify the illuminated lights that represent misfits and undertake the minimum of action required to correct the misfit and dim that light. Once all lights are dimmed, the traditional society will undertake no further action until a new misfit is detected (Fig. 15.2). When considered as Alexander’s network of connected lights, contemporary design alters design variables regardless of whether their associated light is illuminated or not. This leaves some misfits unaddressed and potentially creates new misfits, causing previously dimmed lights to illuminate. This effectively prevents contemporary society from ever intentionally reproducing the quality found in traditional design (Fig. 15.3). Alexander’s methodology analyses the problem to identify its structure, analyses the structure to identify a hierarchy of design variables, and then address each design variable in turn, thereby eliminating all misfits (Fig. 15.4). While Alexander’s innovative conceptualisation of design problems and means of solving them is initially celebrated, later evaluations identify multiple logical and textual contradictions, ambiguities and unresolved issues that would make a practical implementation of this approach deeply problematic (Kuhn 1998; Lendaris 1979; Salsutri 2010; Studer 1965). One example of these issues is that the misfit is a central concept in his theories, yet Alexander fails to provide a rigorous definition for the term, instead providing “a number of characteristics scattered over several passages” (Salsutri 2010, p. 104:2). It must also be noted that Alexander’s models of traditional and contemporary societies are largely anecdotal, and he does not provide rigorous evidence to support these arguments. Finally, even Alexander’s own efforts to utilise his process of design synthesis are unsuccessful (Grabow 1983) and he renounces his theory in the preface of a later printing of Notes on the synthesis of Form (Alexander 1971). A significant challenge in implementing this method was the enormous cost of computational analysis at the time, and the fact that every design

268

M. J. Dawes

Fig. 15.2 Traditional societies only take action (grey shaded area) to eliminate misfits represented by illuminated (white) nodes

Fig. 15.3 Contemporary design (represented by a grey rectangle) alters design variables based on ideologies or theories regardless of fit (black node) or misfit (white node) status

problem requires such analysis. However, Alexander also realised that the data from his analyses included repeating sets of connected design variables that represent generic arrangements of architectural form, and he believed the resulting constructive diagrams “had immense power”. These generic architectural forms evolved to become the “patterns” which form the basis of his second theory of architecture.

15 Pattern Languages: Concepts and Applications in Design for Ageing …

269

Fig. 15.4 Alexander proposed to analyse a design, convert it to a hierarchical tree in which some weaker connections are temporarily ignored (dashed lines), and only take action (shaded grey) to resolve misfits represented by illuminated (white) nodes

15.2.1 A Pattern Language Alexander’s second theory of architecture retains the belief that traditional design is superior to contemporary design in both production and aesthetics and his desire to provide designers with the means to reproduce traditional design qualities in contemporary projects. To pursue the development of patterns, Alexander secured funding to establish a research institute called the Centre for Environmental Studies (CES) with Himself as director and a small number of non-professional designers and students as staff. Each pattern identifies an architectural problem that occurs regularly throughout the environment and provides a generic solution to that problem. The intent of this approach is to enable any individual undertaking an architectural or urban design task to identify the patterns that are relevant to their project and then adapt the generic solutions to the individual’s specific circumstances. In effect, Alexander identifies groups of highly connected design variables, undertakes the research and analysis required to eliminate misfits and publishes the results, thereby freeing the designer from undertaking expensive and time-consuming analyses themselves (Fig. 15.5). Each of the 253 patterns in Alexander’s language follows an identical format that commences with a name, a reference number, confidence rating, and in most cases a photograph of one well realised example of the pattern. The bulk of a pattern consists of a problem context and description, such as where the problem is likely to arise, factors that contribute to the emergence of the problem, and the failings of common yet flawed solutions. This section of the pattern typically ranges from one to three pages in length. It is followed by Alexander’s solution, typically defined in one brief paragraph, and accompanied by a diagram similar to the constructed diagrams of his first theory. The remaining portions of the pattern constitute two sets of references to other patterns. The first set refers to larger patterns that provide the context in which the current pattern is often located, such as CIRCULATION REALMS providing

270

M. J. Dawes

Fig. 15.5 Highly connected clusters of design variables merge to become patterns (shaded grey) and the generic solutions of the patterns have eliminated misfits

the context for the MAIN ENTRANCE. The second set of references are to smaller patterns that will help to complete the current pattern, such as an ENTRANCE ROOM completing the MAIN ENTRANCE. It is these references that serve to unite the individual patterns into a coherent language. Alexander’s proposed process for using A Pattern Language is for the language user of the design to select approximately one dozen patterns of various scales that best describe the project. The language user will then follow the references to additional patterns that provide context or completion for the selected patterns and evaluate whether these additional patterns are applicable to the project. Through this process, the language user will accumulate a series of interconnected patterns that they can then adapt to the specific circumstances of their site and synthesise into a completed design. While Alexander believes that every pattern provides at least the kernel of the correct solution and should, therefore, eliminate misfits, the process of adapting the generic solutions to a specific site leaves room for the language user to introduce misfits and undermine the quality of the completed work. The fact that A Pattern Language can generate designs that fail to achieve the quality of traditional design (Kohn 2002; Seamon 2006) and the fact that it is possible to achieve this quality without the use of patterns (Broadbent 1980; Kalb 2014; Protzen 1980; Walker 2003), are two among many criticisms of Alexander’s second theory of architecture. Other criticisms relating to the user’s implementation of A Pattern Language include claims that some patterns are contradictory and incompatible (Protzen 1980; Saunders 2002a). Meanwhile, other patterns allegedly prescribe flawed design solutions (Broadbent 1980; Gelernter 2000; Salingaros 2000) which are unsuitable for modern societies (Elsheshtawy 2001; Gelernter 1983; Mehaffy 2004, 2008). Additionally, it is also demonstrated that many of the patterns which Alexander assigned low confidence values occupy critical positions within the structure of the language (Dawes and Ostwald 2018). One of the more frequent objections

15 Pattern Languages: Concepts and Applications in Design for Ageing …

271

to using patterns is that they are overly restrictive of the designer and disallow radical solutions (Bhatt 2010; Davis 1983; Dovey 1990; Messina 2003; Protzen 1980). However, it is unlikely that Alexander would view this as a criticism because his conceptualisation of the failings of contemporary design includes designers rejecting traditions in preference of radical solutions. One factor in patterns failing to achieve the quality of traditional design and providing flawed solutions may be that they are poorly explained and the evidence they are based on is faulty, and Alexander’s patterns are criticised for being driven by ideology and un-rigorous reasoning (Broadbent 1980; Dovey 1990; Kohn 2002; Montgomery 1970; Protzen 1980). The appearance of unscholarly practices in Alexander’s writing also includes allegations that he fails to provide explicit definitions for critical elements of his theory, such as patterns, pattern synthesis and the “forces” that shape misfits (Lea 1994; Silva and Paraizo 2008). Alexander is also said to use deliberately provocative statements that undermine his credibility (King 1993; Kohn 2002) and which may contribute to his work having relatively little impact among architectural practitioners. Universal and dogmatic statements and the physical appearance of his texts resembling bibles may also discourage criticism and engagement with his ideas (Bhatt 2010; Broadbent 1980; Dovey 1990; Messina 2003; Protzen 1980; Walker 2003). The origins of many of these criticisms may be found in suggestions that Alexander confuses subjective and objective phenomena (Bhatt 2010; Saunders 2002a, 2003) and refuses to acknowledge alternative experiences and social, political, and economic realities that conflict with his world view (Broadbent 1980; Davis 1983; Elsheshtawy 2001; Kalb 2014; Kohn 2002; Montgomery 1970; Walker 2003). Therefore, it is argued that many of the criticisms of A Pattern Language originate in Alexander’s idiosyncratic ontology and epistemology (Dawes and Ostwald 2017). Ultimately, even Alexander rejects A Pattern Language for having too little generative power and too little focus on geometry, choosing instead to focus on a third theory of architecture based around fifteen geometric properties such as “strong centres”, “good shape”, and “the void”. However, Alexander’s third theory abandons the pattern format, which so effectively packages spatial, social, and cultural knowledge in a concise and compelling way. Despite Alexander moving on from pattern languages, the underlying concept of A Pattern Language as a series of interconnected generic solutions to common problems retains significant value. Scholars continue to develop and publish new patterns and entire new pattern languages.

15.3 Methodology The development of new pattern languages is both anticipated and encouraged by Alexander, as is emphasised by the careful naming of his canonical text: We have called it ‘A Pattern Language’ with the emphasis on the ‘A’ […] we imagine this pattern language might be related to the countless thousands of other languages we hope people will make for themselves in the future (Alexander et al. 1977), p xvi.

272

M. J. Dawes

The current chapter consists of an initial exploration of potential avenues for new applications of architectural pattern languages. It utilises a qualitative and nonexhaustive literature review of pattern languages developed following the publication of Alexander’s A Pattern Language to identify some of their strengths and weaknesses and possible lessons that might guide the development of future research. The focus of this review is the development and use of patterns in built environment disciplines, and new pattern languages in other disciplines are beyond the scope of this chapter.

15.4 Findings New pattern languages have found considerable success and influence in various fields such as computer programming and software design, yet in contrast, there are relatively few extensions of A Pattern Language in built environment disciplines. The new pattern languages with an architectural focus can be understood as belonging to three somewhat overlapping groups. The first group includes aspirational design patterns similar to those published in A Pattern Language, the second group identifies and uses patterns as a type of environmental assessment tool, and the third group focuses on using the pattern format to document the existing conditions of specific locations. An excellent example of aspirational patterns is found in A New Pattern Language for Growing Regions (Mehaffy et al. 2020). This recently published text contains 80 new patterns which, like Alexander’s language, range in scale from regional planning through to small details such as door handles, but also include a diverse range of building types, including schools, shops, hospitals, and industry, and topics including taxation, finance, technology, and governance. One of the significant strengths of this text is that in addition to listing the connections between their new patterns, the authors also include references to A Pattern Language, thereby linking the two texts and extending the work of the original. Another strength of this new pattern language is that the patterns include references to additional publications, including government documents and peer-reviewed research. The referencing included in these design patterns serves to defend these new patterns against some of the criticisms of Alexander’s language, such as the patterns being based on biased or anecdotal evidence. This approach of referencing the research that underpins patterns is shared with the Cooling the Commons Pattern Deck (Cooling the Commons 2023), which includes 40 patterns dedicated to reducing urban heat effects in Western Sydney. These patterns include topics such as social services, transportation, tree planting, and the use of outdoor space. While the Pattern Deck diverges from the traditional pattern format with subsections such as enablers and constraints, it retains the inter-pattern connections of a language. Another approach to new pattern languages focuses on using them as environmental assessment tools, particularly in school buildings. In this context, researchers analyse the designs of a series of “best practice’ and award-winning schools and

15 Pattern Languages: Concepts and Applications in Design for Ageing …

273

scholarly literature to identify architectural and pedagogical features and document these as a list of patterns representing high quality school environments (Ogden et al. 2010; Tanner 2000). Researchers may then visit case study schools to evaluate the extent to which each of these patterns is present to quantify the architectural quality of educational spaces (Andersen 1999). Significant correlations exist between these environmental assessments and metrics of student achievement, such as standardised testing (Hughes 2005; Tanner 2008, 2009). While these studies demonstrate the adaptability of pattern languages by using them as assessment tools rather than design guides, much of this style of research shares a common trait of only publishing a selection of the patterns that are used in the analysis (Moore and Lackney 1993). This makes it challenging for other researchers to utilise or learn from the patterns and limits the ability of designers to incorporate those patterns into new schools. The third use of new pattern languages is to document the existing socio-cultural patterns of a specific location (Hamid et al. 2022). This process commences with documenting the selected environment, extracting patterns from this documentation with the assistance of focus groups, and then evaluating the patterns among different participant groups. Similar studies that aim to document the patterns of specific locations and aspirational design patterns also follow collaborative development models that incorporate significant input from local interviews and focus groups (Eglin 2020). One potential pitfall of documenting existing conditions as patterns is the risk of codifying examples of low-quality environments or creating patterns that Alexander might describe as dead or containing misfits. In this sense, it is essential to distinguish between documentation patterns and design aspiration patterns, and to apply critical evaluations to the latter. From a research standpoint, several common strengths and weaknesses emerge from this non-exhaustive literature review from which it is possible to draw lessons for future efforts at developing pattern languages. Perhaps the greatest strengths of patterns are their recognisable and repeatable format and unlimited range of scales and subjects. These aspects of patterns combine to enable anyone with knowledge of patterns to quickly identify the information they seek and how that information relates to other topics. However, this is only possible if patterns are published and accessible. Patterns are also only useful if their intended audience is receptive to using them. One common criticism of Alexander’s language is that it is too restrictive (Dawes and Ostwald 2017), and recent research on design approaches reveals that some built environment professionals retain this belief (Casakin 2018). However, a strength of patterns is that they are intended to be revised by language users and this has the potential to allow patterns to become more flexible, and for patterns that publish the research they are based upon this adaptability enables patterns to remain rigorous by reflecting the latest research insights. For researchers intending to investigate new pattern languages several lessons can be drawn from the current discussion and the strengths and weaknesses identified in Table 15.1. These lessons include: new patterns typically require collaborative and iterative development, patterns are most useful when they are published in full and are freely accessible, careful selection of pattern content can increase audience size or impact, and useful patterns are those which are likely to be utilised and implemented.

274

M. J. Dawes

Table 15.1 Strengths and weaknesses of new pattern languages Strengths

Weaknesses

Recognisable and repeatable format

Unpublished pattern languages

Unlimited scale and subject matter

May be rejected by professionals

Potential widespread impacts

Require large audiences or widespread impacts

Can be rigorous, revised, and improved

May conflict with building codes or legislations

Patterns can be compelling

May document problems rather than solutions

These lessons should be considered in the context of selecting new avenues for the research into the development of future pattern languages.

15.4.1 Architectural Patterns for Design Education One potential application for new pattern languages is in architectural education. While there are numerous examples of researchers and educators developing pedagogical pattern languages to guide teaching activities (Bergin 2001; Bergin et al. 2012; Goodyear and Yang 2009), there are only a small number of examples of educators attempting to use Alexander’s patterns, or new pattern languages, as educational tools for architecture students (Pontikis 2010). However, the compact pattern format, describing architectural forms paired with their associated social, cultural, economic, and technical origins has significant potential to enable students to gain a holistic understanding of the discipline. When used in educational design studios pattern languages provide several benefits including providing a means of framing design problems, defining design requirements, and organising the design process (Davis 1983; Karlgren and Ramberg 2012). When used as part of collaborative design studio task, pattern languages not only promote discussion but also focus discussions because students can concentrate on the content of one pattern and be confident that any additional issues will be discussed when the relevant pattern is discussed. In this way, patterns promote critical thinking (Davis 1983). The common criticism of patterns being too restrictive of a designer’s creativity also arises when using patterns in educational design studios, with students being prevented from following their preferences (Davis 1983; Pontikis 2010). One irony with this criticism is that many architects are consciously or un-consciously using their own pattern languages in their daily employment (Broadbent 1980), without formally encoding this knowledge in pattern format. For example, if tasked with designing a house, an architect is likely to produce bedrooms that follow a similar design to the bedrooms in all the other houses that the architect has designed. The architect is effectively working from an informal pattern, whether they realise this or not. However, the restrictive nature of patterns need not be a disadvantage in a design studio. Any flaws or restrictions in patterns can represent the starting point of educational activities that will see the students tasked with improving Alexander’s

15 Pattern Languages: Concepts and Applications in Design for Ageing …

275

patterns or creating new ones, thereby encouraging deep critical thinking about both architectural form and broader issues (Pontikis 2010). As a final expansion of using architectural patterns in design education, it should be noted that Alexander himself led a collaborative affordable housing project in which families received funding to design and construct their own homes (Alexander et al. 1985). While this type of activity falls outside most architectural degree programs, it does suggest that pattern languages can provide design education for those who are unable to attend universities.

15.4.2 Architectural Patterns for an Ageing Population A second potential avenue for new pattern languages in an Australian context is developing patterns for the elderly. The Australian Bureau of Statistics records population data through regular censuses and has revealed that Australia has an ageing population, meaning that the median age of Australians is increasing, and the percentage of Australians aged over 65 years is also increasing (ABS 2020). There are 4.2 million Australians over the age of 65, and the Australian Government is annually providing $30 billion of funding for aged care, this figure is estimated to rise to $46 billion annually by 2027 (Hermant 2023). As the population ages, their physical prowess and mobility typically decrease, and this often results in requiring changes to their existing environment, relocation to aged care accommodation, assistance to achieve tasks they could formerly complete alone, or a combination of these outcomes. While national standards for the quality of aged care exist, including provisions in the National Construction Code, many of these relate to medical care, nutrition, or building egress, rather than broader quality of life considerations. Some important findings of the recently completed Royal Commission Into Aged Care Quality and Safety recommends the development of design guidelines for aged care accommodation: Designing age-friendly communities that support people to stay in their own homes into later life, age-friendly city and town design (Pagone and Briggs 2021, p. 80) the Australian Government should develop and publish a comprehensive set of national aged care design principles and guidelines on accessible and dementia-friendly design for residential aged care. The National Aged Care Design Principles and Guidelines should permit flexibility in their application in different circumstances. (Pagone and Briggs 2021, p. 106) People’s accommodation should, where possible, cater to their changing needs (Pagone and Briggs 2021, p. 107)

These recommendations share many similarities to architectural patterns. The first quote, like A Pattern Language, refers to a broad range of built environment scales including the home, town, and city. The second quote calls for a comprehensive set of national guidelines and that those guidelines should permit flexibility in their application. Patterns are intended to represent generic solutions to frequently occurring

276

M. J. Dawes

problems in the built environment and closely align with the call for flexible guidelines. Furthermore, the unifying connections between patterns enable the linking of multiple generic solutions to provide both a comprehensive and cohesive set of guidelines. The final quote refers to architecture as not being static, a view which aligns with Alexander’s view of traditional design as slowly evolving to adapt to the changing needs of the population, and also to A Pattern Language being intended as a guide to empower laypeople to enact those changes. At a superficial level, design guidelines and principles for an ageing population appear to be a topic well suited for the development of new patterns. However, patterns for ageing should also be considered in the context of lessons learned from the development of other new pattern languages.

15.5 Discussion The first lesson derived from the review of recently developed patterns is the need for collaborative and iterative development. The Australian Government Department of Health has already commenced developing the recommended accommodation design guidelines and has undertaken this task through multiple open online surveys, webinars, and engagements with university researchers (DoH 2022). This process has identified at least 15 principles for aged care design, including topics such as “A meaningful experience of ‘home’”, “gardens and outdoor spaces”, “a sense of community”, and “consultation and co-design” all of which align with the aspirational content of design patterns. There is an opportunity for researchers to extend this collaborative effort and develop this content into accommodation patterns for ageing people. There are unlikely to be significant barriers to publishing this content online in an open access format, along with the research the patterns are based on, and to include future updates and revisions to the patterns. This is particularly true if an Australian Government department hosts the patterns, as this will lend additional credibility to the pattern content. Thus, accommodation patterns for ageing people also align well with the second lesson garnered from a non-exhaustive review of new pattern languages. Developing new patterns for design education presents different opportunities in terms of collaboration and publication. It is much less likely that architecture students will be afforded opportunities to collaborate with a diverse group of the public. However, this limitation may be somewhat countered if collaborative tasks or groupwork assignments are embedded into their study curricula. Furthermore, the inability to collaborate with the public may actually be beneficial. Roleplaying as citizens, developers, or approval authorities while developing new pattern languages can provide students with new insights that they might not otherwise achieve without stepping out of the role of architecture student. In terms of publication, it is unlikely that patterns developed by students would be widely published. However, there is potential for students to develop their own pattern language as an ongoing educational activity in which the students in subsequent years would refine, revise, and extend

15 Pattern Languages: Concepts and Applications in Design for Ageing …

277

the language, effectively developing a new language specific to their educational institution and immediate peers. The final two closely related lessons for new pattern languages are that they (i) have the potential to provide the widespread impact useful in justifying research efforts to funding organisations and (ii) provide the opportunity to focus on patterns that will achieve this through large-scale user implementation. One potential challenge with guidelines prepared by governments or large industry bodies is the potential for these documents to take the form of sanitised lists of requirements. This format is excellent for industry professionals seeking concise and precise information but may also alienate the general public, thereby limiting the potential impact of the guidelines. Such guidelines are equivalent to the solution subsection of patterns and are only a portion of a pattern’s content. In addition to a solution, a pattern provides a problem context and description, and this may contribute to the compelling nature of patterns. These additional portions of patterns describe where the problem is likely to occur, why the problem occurs and why other design options fail to solve the problem. These descriptions often use engaging language that encourages readers to recall or imagine themselves experiencing the hardship being described and has the potential to lead the reader to become emotionally invested in learning the solution to their imagined hardship. This emotional investment may cause the pattern’s solution to appear more insightful or profound compared to similar information published as a sanitised list of requirements. Furthermore, “[e]motions can influence the generation of an action in two ways: the tendency and readiness to act, and the decision to act” (Zhu and Thagard 2002, p. 27), therefore emotional investment in pattern solutions may contribute to the reader taking action, whereas the same may not be true for sanitised lists of requirements. Direct action on the part of users aligns with Alexander’s intention for pattern languages and the final lesson from the review of new patterns. Smaller-scale patterns within the home are likely to be implemented relatively easily as the actions undertaken require fewer resources and external partners than larger-scale patterns, which may require demolition, renovation, or new construction. Focusing on small, easily implemented patterns that may have a widespread impact, such as ‘patterns for ageing at home’ to accompany broader Australian Government guidelines, appears to be a potentially fruitful avenue for researching new architectural patterns. In the context of design education, there is less potential for widespread impact and less potential for patterns to be implemented simply because a student cohort at an individual educational institution is much smaller than the population of ageing Australians. However, it is possible that students may be able to focus on developing a pattern language which their institution will use to guide future development. This concept is the subject of The Oregon Experiment (Alexander et al. 1975), the third of the three major texts documenting Alexander’s second theory of architecture. If an educational institution was prepared to follow a similar model of development, there is some potential for student-developed patterns to not only serve as an educational tool but to also be implemented and to have a significant impact on the future of their institution.

278

M. J. Dawes

15.5.1 Limitations and Future Research The most significant limitation to the research and findings of the current chapter is that it represents the early stages of research exploration and that the review of new patterns developed after Alexander published his pattern language is non-exhaustive. There are likely to be additional architectural pattern languages that are not included in the current discussion, the analysis of which may highlight new strengths, weaknesses, and lessons that are not identified in this chapter. A further limitation of the current discussion is the focus on architectural patterns. Pattern languages have received significantly greater attention in non-built environment disciplines, and despite these fields being beyond the scope of the current research, they may provide additional important insights into developing new patterns within built environment research areas. A final limitation of the current chapter is the focus on accommodation for ageing and older adults and for use in design education. As the review of new architectural patterns highlights, there is significant potential to develop pattern languages for widespread institutional buildings such as schools and medical care facilities. Therefore, three immediate directions for future research exist. The first is to expand the literature review to identify architectural pattern languages omitted from the current chapter. The second is to investigate whether using and developing pattern languages in other disciplines can improve built environment pattern research. The third avenue for immediate further research is to investigate whether other institutional building types, which also have the potential for significant and widespread research impact, could benefit from applications of pattern languages. Additional avenues for further investigation include using machine learning to identify and document new pattern languages.

15.6 Conclusion Despite having relatively little impact on the architectural profession, Christopher Alexander’s concept of pattern languages continues to be developed and expanded. The current chapter contributes to a discussion of potential future applications of this concept by undertaking a non-exhaustive review of new architectural pattern languages to identify their strengths, weaknesses, and lessons that might guide the development of future pattern languages. This chapter then discusses these lessons in the context of developing new architectural patterns for use in design education and the design of accommodation for elderly and ageing people. In doing so, this chapter reviews Australia’s ageing population and the recent Royal Commission Into Aged Care Quality and Safety and finds that the recommended accommodation guidelines closely align with pattern languages in scope, content, and the need for flexibility in application. As a result, this chapter concludes that there is significant potential for these guidelines to be documented in pattern language format and for new, smaller

15 Pattern Languages: Concepts and Applications in Design for Ageing …

279

scale, patterns focusing on ageing in place to be developed in support and extension of these guidelines. This chapter also concludes that using architectural patterns in design education presents intriguing opportunities for future research, including overseeing students developing, revising, refining, and extending a pattern language as a multi-year, ongoing, and collaborative teaching activity. However, aged care accommodation and design education are not the only possible areas where developing new patterns may be valuable research efforts. Other institutional buildings, such as schools and medical care facilities, may also present opportunities for developing architectural patterns that can have a widespread impact on society.

References ABS (2020) Twenty years of population change. https://www.abs.gov.au/articles/twenty-years-pop ulation-change. Accessed 12 Aug 2023 Alexander C (1964) Notes on the synthesis of form. Harvard University Press Alexander C (1971) Notes on the synthesis of form. Harvard University Press, Cambridge Alexander C (1979) The timeless way of building. Oxford University Press, New York Alexander C, Silverstien M, Angel S, Ishikawa S, Abrams D (1975) The Oregon experiment. Oxford University Press, New York Alexander C, Ishikawa S, Silverstien M (1977) A pattern language: towns, buildings, construction. Oxford University Press, New York Alexander C, Davis H, Martinez J, Corner D (1985) The production of houses. Oxford Universtiy Press, New York Andersen S (1999) The relationship between school design variables and scores on the Iowa test of basic skills. University of Georgia, Georgia Ashby WR (1960) Design for a brain; the origin of adaptive behavior, 2nd edn. Aufl. Wiley, New York Bergin J (2001) A pattern language for initial course design. ACM SIGCSE Bulletin 33:282–286. https://doi.org/10.1145/364447.364602 Bergin J, Eckstein J, Völter M, Sipos M, Wallingford E, Marquardt K, Chandler J, Sharp H, Manns ML (2012) Pedagogical patterns: advice for educators: createspace independent publishing platform Bhatt R (2010) Christopher Alexander’s pattern language: an alternative exploration of spacemaking practices. J Archit 15(6):711–729 Broadbent G (1980) Pattern language. Des Stud 1(4):252–253 Cantor G (1874) Ueber eine Eigenschaft des Inbegriffes aller reellen algebraischen Zahlen. Journal Für Die Reine Und Angewandte Mathematik 77:258–262 Casakin H (2018) The use of patterns as an urban design approach. Urban Science 2(4). https://doi. org/10.3390/urbansci2040101 Cooling the Commons (2023) The pattern deck. Accessed 11 Aug 2023 Davis H (1983) Individual houses in groups: the pattern language in a teaching studio. Journal of Architectural Education 36(3):14–19 Dawes MJ, Ostwald MJ (2017) Christopher Alexander’s a pattern language: analysing, mapping and classifying the critical response. City Territory Architecture 4(17). https://doi.org/10.1186/ s40410-017-0073-1 Dawes MJ, Ostwald MJ (2018) The mathematical structure of Alexander’s a pattern language: an analysis of the role of invariant patterns. Environment and Planning b: Urban Analytics and City Science 47(1):7–24. https://doi.org/10.1177/2399808318761396

280

M. J. Dawes

DoH (2022) Aged care design standards consultation outcomes. Australian Government Department of Health and Aged Care Dovey K (1990) The pattern language and its enemies. Des Stud 11(1):3–9 Eglin R (2020) Capturing a community’s vision as a pattern language: case study of the Mooiplaas community in Great Kei Municipality, South Africa. https://doi.org/10.18820/2415-0495/trp 76i1.3 Elsheshtawy Y (2001) Searching for theory in Christopher Alexander’s intellectual roots. Archit Sci Rev 44(4):395–403 Euler L (1736) Solutio problematis ad geometriam situs pertinentis. Commentarii Academiae Scientiarum Imperialis Petropolitanae 8(7) Gelernter M (1983) Christopher Alexander and the pattern language. Architectural Journal 1:17–21 Gelernter M (2000) Sun-filled window seats. Archit Res Q 4(2):190–192 Goodyear P, Yang DF (2009) Patterns and pattern languages in educational design. In: McBeath C, Atkinson R, Jonas-Dwyer D, Phillips R (eds) Beyond the comfort zone: proceedings of the 21st ASCILITE conference. Perth, pp 339–347 Grabow S (1983) Christopher Alexander: the search for a new paradigm in architecture. Oriel Press, Stocksfield Hamid M, Hanks L, Wang Q (2022) Extracting a local pattern language: a case study of Khartoum’s City Centre. Urban Design International. https://doi.org/10.1057/s41289-022-00205-w Hermant N (2023) Federal budget projects a shift in aged care, which promises to save Australia billions. https://www.abc.net.au/news/2023-05-11/budget-aged-care-measures-and-home-care/ 102328544. Accessed 12 Aug 2023 Hughes SM (2005) The relationship between school design variables and scores on the iowa test of basic skills. Baylor University Kalb J (2014) Life in design: Christopher Alexander and the nature of order. International Journal of Architectural Research 8(2):94–98 Karlgren K, Ramberg R (2012) The use of design patterns in overcoming misunderstandings in collaborative interaction design. CoDesign 8(4):231–246 King IF (1993) Christopher Alexander and contemporary architecuture, Special Issue. Architecture and Urbanism, Tokyo Kohn W (2002) The lost prophet of architecture. Wilson Quarterly Summer:26–34 Kuhn C (1998) Diagrams are forever Christopher Alexander’s “pattern langugage” and notes on the synthesis of form. Daidalos 69/70(1998/99):136–145 Lea D (1994) Christopher Alexander: an introduction for object-oriented designers. Software Engineering Notes 19(1):39–46 Lendaris GG (1979) On the human aspects in structural modeling. Technol Forecast Soc Chang 14(4):329–351 Marshal S (2012) Science, pseudo-science and urban design. Urban Design International 17(4):257– 271 Mehaffy MW (2004) A conversation with Andrés Duany. Katarxis 3 Mehaffy MW (2008) Generative methods in urban design: a progress assessment. J Urban 1(1):57– 75 Mehaffy MW, Kryazheva Y, Rudd A, Salingaros NA (2020) A new pattern language for growing regions. Sustasis Foundation Messina J (2003) Architecture: some musings on the state of the art. Journal of the Southwest 45(1/ 2):1–7 Montgomery R (1970) Pattern language. Architectural Forum 132(1):52–59 Moore GT, Lackney JA (1993) Design patterns for American schools: responding to the reform movement. Paper presented at the Prairie school national conference on architecture and education, Racine, Wisconsin Ogden H, Upitis R, Brook J, Peterson A, Davis J, Troop M (2010) Entering school: how entryways and foyers foster social interaction. Child Youth Environ 20(2):150–174 Pagone T, Briggs L (2021) Royal commission into aged care quality and safety

15 Pattern Languages: Concepts and Applications in Design for Ageing …

281

Pontikis K (2010) Teaching Christopher Alexander’s theoretical framework in a capstone interior design studio. Design Principles and Practices: an International Journal 4(3):407–425 Protzen J-P (1980) The poverty of the pattern language. Des Stud 1(5):291–295 Salingaros NA (2000) The structure of pattern languages. Archit Res Q 4(2):149–161 Salsutri FA (2010) Misfits, balance, requirements and systems: thoughts on Alexander’s notes on the synthesis of form. Paper presented at the proceedings of 2010 conference of the design research society, Montreal Saunders W (2002a) Commentary: ever more popular, ever more dogmatic: the sad sequel to Christopher Alexander’s work. Archit Rec 190(5):93–96 Saunders W (2002b) A pattern languge by Christopher Alexander. Harvard Design Magazine. Winter/Spring(16):74–78 Saunders W (2003) The nature of order II: William Saunders responds to Christopher Alexander’s letter. In: Architectural record. http://www.architecturalrecord.com/articles/12371-the-natureof-order-ii? Accessed 14 June 2017 Seamon D (2006) Clarifying and evaluating Alexander’s theory of wholeness by interpreting his approach as a tetrad of activity. Paper presented at the annual meetings of the international association for environmental philosophy, Philadelphia, PA Silva EM, Paraizo C (2008) Urban projects database based on Alexander’s pattern language: the case of Favela-Bairro. Int J Archit Comput 6(3):261–277 Studer RG (1965) Books: notes on the synthesis of form. Journal of Architectural Education 19(4):P62–P64 Tanner K (2000) The influence of school architecture on academic achievement. Journal of Educational Administration 38(4):309–330 Tanner K (2008) Explaining relationships among student outcomes and the school’s physical environment. J Adv Acad 19(3):444–471. https://doi.org/10.4219/jaa-2008-812 Tanner K (2009) Effects of school design on student outcomes. Journal of Educational Administration 47(3):381–399 Walker B (2003) Another kind of science: Christopher Alexander on democratic theory and the built environment. Can J Polit Sci 36(5):1053–1072 Zhu J, Thagard P (2002) Emotion and action. Philos Psychol 15(1):19–36. https://doi.org/10.1080/ 09515080120109397