Creativity in the Age of Digital Reproduction: xArch Symposium (Lecture Notes in Civil Engineering, 343) 9819706203, 9789819706204

Citation preview

Lecture Notes in Civil Engineering

Giancarlo Di Marco Davide Lombardi Mia Tedjosaputro   Editors

Creativity in the Age of Digital Reproduction xArch Symposium

Lecture Notes in Civil Engineering

343

Series Editors Marco di Prisco, Politecnico di Milano, Milano, Italy Sheng-Hong Chen, School of Water Resources and Hydropower Engineering, Wuhan University, Wuhan, China Ioannis Vayas, Institute of Steel Structures, National Technical University of Athens, Athens, Greece Sanjay Kumar Shukla, School of Engineering, Edith Cowan University, Joondalup, WA, Australia Anuj Sharma, Iowa State University, Ames, IA, USA Nagesh Kumar, Department of Civil Engineering, Indian Institute of Science Bangalore, Bengaluru, Karnataka, India Chien Ming Wang, School of Civil Engineering, The University of Queensland, Brisbane, QLD, Australia Zhen-Dong Cui, China University of Mining and Technology, Xuzhou, China

Lecture Notes in Civil Engineering (LNCE) publishes the latest developments in Civil Engineering—quickly, informally and in top quality. Though original research reported in proceedings and post-proceedings represents the core of LNCE, edited volumes of exceptionally high quality and interest may also be considered for publication. Volumes published in LNCE embrace all aspects and subfields of, as well as new challenges in, Civil Engineering. Topics in the series include: • • • • • • • • • • • • • • •

Construction and Structural Mechanics Building Materials Concrete, Steel and Timber Structures Geotechnical Engineering Earthquake Engineering Coastal Engineering Ocean and Offshore Engineering; Ships and Floating Structures Hydraulics, Hydrology and Water Resources Engineering Environmental Engineering and Sustainability Structural Health and Monitoring Surveying and Geographical Information Systems Indoor Environments Transportation and Traffic Risk Analysis Safety and Security

To submit a proposal or request further information, please contact the appropriate Springer Editor: – Pierpaolo Riva at [email protected] (Europe and Americas); – Swati Meherishi at [email protected] (Asia—except China, Australia, and New Zealand); – Wayne Hu at [email protected] (China). All books in the series now indexed by Scopus and EI Compendex database!

Giancarlo Di Marco · Davide Lombardi · Mia Tedjosaputro Editors

Creativity in the Age of Digital Reproduction xArch Symposium

Editors Giancarlo Di Marco Xi’an Jiaotong-Liverpool University Suzhou, China

Davide Lombardi Xi’an Jiaotong-Liverpool University Suzhou, China

Mia Tedjosaputro Xi’an Jiaotong-Liverpool University Suzhou, China

ISSN 2366-2557 ISSN 2366-2565 (electronic) Lecture Notes in Civil Engineering ISBN 978-981-97-0620-4 ISBN 978-981-97-0621-1 (eBook) https://doi.org/10.1007/978-981-97-0621-1 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Singapore Pte Ltd. The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721, Singapore Paper in this product is recyclable.

Contents

Creativity in the Age of Digital Reproduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Giancarlo Di Marco, Mia Tedjosaputro, and Davide Lombardi

1

Computational Design Session Architectural Design Under Pandemic of Today’s AI: From Human-Computer Interaction (HCI) to Human-Machine Conversation (HMC) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lok Hang Cheung, Juan Carlos Dall’Asta, Giancarlo Di Marco, and Asterios Agkathidis

9

Creativity at the Edge of Digital Reproduction — from Coop Himmelb(l)au to Deep Himmelb(l)au . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Lei Feng

18

Computing Analogue Interactive Installations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Michail Georgiou, Odysseas Georgiou, and Eva Korae

25

UAV-Based Geometry Data Acquisition for Building Energy Modelling . . . . . . . Mengfan Jin and Marco Cimillo

34

Cellular Automata as Design Tools for Artificial Ecologies . . . . . . . . . . . . . . . . . . Yiming Liu and Christiane M. Herr

42

Deployable Origami Wall with Patterned Knit Panels . . . . . . . . . . . . . . . . . . . . . . . Virginia Ellyn Melnyk

50

Critical Social Computing for Digital Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Muhammad Talha Muftee

61

Data-Responsive Architecture in Urban Open Space: Sensing Social and Environmental Data and Regulating Spatial Configuration in Real-Time . . . Hyunjae Nam

69

Design of the Intelligent Bridge Drainage Monitoring and Control System . . . . . Danni Zheng, Yiheng Feng, and Li Li

77

SAUCE – SpAcevehicle-bUilding Connectivity Evaluation . . . . . . . . . . . . . . . . . . Zhelun Zhu

85

vi

Contents

Digital Experience Session Challenges and Opportunities in Using Digital Pedagogy for Game-Based Architecture Education: A Case in China . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Silvia Albano, Wan Meng, Wenruo Xu, and Na Li

95

Digital Creativity in Urban Interventions: Using Technology as an Engagement and Idea Inducing Tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 Daria Belkouri and Theodoros Dounas Digital Hybridities: Theorising the ‘Social’ and the ‘Local’ of Fabrication Technologies in Craft Practice . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Matthew Holmes and Alejandro Veliz Reyes Exploring an Evolving Architectural Pedagogy in the Age of Digital Creativity and Artificial Intelligence: Examining the Challenges to Critical Thinking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 John Latto Exploring 3D Concrete Printing of Lattice Structures on Robotically-Shaped Sand Formwork for Circular Futures . . . . . . . . . . . . . . . . . 128 Cristina Nan and Alessio Vigorito Geometric Variability and Viability in Designing and Fabricating Concrete Façade Components–A Systematic Review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 Deyan Quan, Christiane M. Herr, and Davide Lombardi Augmented LEGO™ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 Yang Song and Wei Zhao Virtual Reality and EEG in Creativity Research: Investigating the Impact of Designed Environments on Creative Performance . . . . . . . . . . . . . . . . . . . . . . . . 152 Fatemeh Taherysayah and Claudia Westermann Analysis of Differences in Street Visual Walkability Between Human and Machine Perception: A Case Study of an Anonymous University Campus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160 Yuchen Xie, Yunqin Li, Lingshan Huang, and Jiaxin Zhang AI and Environmental Session Exploration of Conceptual Design Generation Based on the Deep Learning Model – Discussing the Application of AI Generator to the Preliminary Architectural Design Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171 Yuxin Bao and Changying Xiang

Contents

vii

The Taste of Textures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179 José Antonio Carrillo Andrada, José de la Rosa Morón, and José Luis Oliver Ramírez AI Machine Learning in Creative Architectural Design Processes . . . . . . . . . . . . . 190 Juan Carlos Dall’Asta and Giancarlo Di Marco An Assessment of Thermal Comfort in Urban Quality of Life in Architecture Using Fuzzy Logic in Decision Making: A Case Study of Iran . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201 Alireza Gogani, Faezeh Choobkar, and Asli Cekmis A Language Prompt Model for Architectural Aesthetics . . . . . . . . . . . . . . . . . . . . . 209 Graham Brenton McKay Employing AI-Based Tools to Support Exhibition Design: A Science and Technology Museum Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217 Lulu Pei, Chenxiao Li, Jun Xiao, Yu Zhang, and Christiane M. Herr An AI-Mediated VR Sound Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 225 Giovanni Santini and Zhonghao Chen Space Narrative: Generating Images and 3D Scenes of Chinese Garden from Text Using Deep Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236 Jiaxi Shi and Hao Hua Strategies of Interconnecting Deep Learning Models in AI-Driven Design Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244 Shermeen Yousif and Daniel Bolojan Meta-morphing Architectural Domains: The Role of Humans and AI in Post-human Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253 Asif Hasan Zeshan and Susannah Dickinson Optimization and Design of Building-Integrated Photovoltaic Systems for a High-Rise Building in Shenzhen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262 Yuqi Zhang, Christiane M. Herr, and Yongcong Guo Structuring Gamified Participatory Public Space Design Developing a Design Quality Evaluation System to Support Digital Co-creation Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270 Shutong Zhu, Provides Ng, and Jeroen van Ameijde Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283

Creativity in the Age of Digital Reproduction Giancarlo Di Marco(B)

, Mia Tedjosaputro , and Davide Lombardi

Xi’an Jiaotong-Liverpool University, Suzhou, China [email protected]

When a wise man points at the moon the imbecile examines the finger Confucius

1 Introduction Inspired by Walter Benjamin’s famous work “The Work of Art in the Age of Mechanical Reproduction” and conceived as the motto for the first edition of the xArch symposium, Creativity in the Age of Digital Reproduction is a provocation but also, presumptuously, the only productive and meaningful approach to the study of AI, as further explained in this chapter. AI is a significant breakthrough and a disruptive technology, making Industry 5.0 real and, more importantly, giving a different meaning to the concept of Humanity 2.0 [1] while opening the door to Humanity 3.0. Strangely enough, but understandably in an economy- and finance-centred society, the debate around AI focuses on the practical consequences like the impact on the workforce or the extinction of the human race. It’s mainly under this light that we split into two factions: passionate about AI and catastrophists, lovers and haters – the two sides of an irrational coin. Another approach is to conduct separate reflections and put AI back to where it probably belongs – the toolbox. With the emergence of artificial intelligence (AI) and machine learning, the design field is poised for another significant shift. The concept of identical repetition, which Carpo attributes to the invention of the printing press [2], allowed for the mass production of exact copies. This revolutionised design and manufacturing processes, making it easier to reproduce designs on a large scale. However, as technology advanced, the notion of identical repetition shifted towards mass customisation. Digital technologies and computer-aided design (CAD) enabled designers to create customisable products by adapting existing templates to fit individual preferences. AI has shown the potential to generate endless variations of creative work, starting from pre-existing datasets and going beyond mass customisation. By analysing vast amounts of data, AI algorithms can learn patterns, identify trends, and generate novel designs based on existing templates or concepts. It exponentially expanded the power of reproducing images and designs, leveraging, in some cases, on verbal input, i.e. overcoming the necessity of holding precise skills to achieve meaningful results. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 1–5, 2024. https://doi.org/10.1007/978-981-97-0621-1_1

2

G. Di Marco et al.

AI algorithms can analyse user preferences, market trends, and historical data to create unique designs that cater to individual needs. This level of customisation allows designers to harness AI’s power to generate designs tailored to specific contexts, user preferences, and even environmental conditions. Moreover, AI assists designers in exploring new design possibilities by suggesting variations based on existing concepts. AI algorithms can learn from existing designs, identify patterns, and propose alternative iterations that designers may not have previously considered. While the shift towards endless variations of pre-existing material through AI and machine learning holds great potential, it also poses ethical challenges. The issue of intellectual property and copyright becomes more complex when AI generates designs based on existing works; likewise, a change in the concept of design value in relation to its cost [3] is inevitable, given the intrinsic nature of AI tools as tools that aims at reaching the wider possible audience. Striking a balance between the benefits of AIgenerated design and the protection of originality will be an ongoing challenge that designers, policymakers, and society must address. Furthermore, the re-introduction of scripting, in the simplified form of prompts, leads to a U-turn in the evolution of the use of digital tools, that after being pushed towards a verbal-to-visual [4], now moves to a form of dialogue between the machine and the operator, lifting the asperities and unintelligibility of traditional coding language. AI and machine learning have the potential to revolutionise the field of design by shifting from identical repetition and mass customisation to endless variations of pre-existing materials. This shift opens up possibilities for customised, context-specific designs that can cater to individual needs.

2 Reflection 2.1 Technology Technology is one of the ways in which Humanity has manifested its intelligence. We have unleashed our creativity and explored new possibilities to improve the world or the surrounding space by relegating tedious processes and repetitive or heavy tasks. Technology has evolved through the millennia, experiencing two critical breakthroughs: Automation and Digitalisation. Still perceived as a natural evolution of classic technology, Automation was made possible by inventions such as steam engines and electric motors. We can refer to traditional and automated technology as analog or analogue. Digitalisation, on the other hand, represents a disruptive moment in the evolution of technology. With limited requirements for preparation and skills, analogue technologies led to democratic and standardised processes, where the human factor consisted of two specific roles: the technical, relatively easy to undertake, and the creative, requiring particular skills and a different mindset. Technology shifted massively from the physical to the digital world in the second half of the 20th century and has been evolving at a very fast pace. However, until recent years, digital technology has only been perceived as a very sophisticated tool requiring a complex set of soft and hard technical skills.

Creativity in the Age of Digital Reproduction

3

Creativity has also been affected by the digital shift: it was easier to imagine new processes when technology was easy to understand and control, but a more advanced and complex technology also required additional creative thinking skills. We are now on the verge of a third breakthrough represented by the most radical evolution of digital technology: AI. Undeniably, for the first time in our history, we are giving up our most distinctive feature: creativity. 2.2 Intelligence To what extent is it possible to talk about artificial intelligence? As of today, no matter the field of applications or how advanced the specific AI is, the way an AI works is straightforward: take a vast amount of data, detect patterns, observe an event, and classify the event according to the library of patterns detected. Is this process enough to define an intelligent system? If, as human beings, we could accumulate, have access to, and use a vast amount of data, how many patterns would we recognise? In other words, is the cognitive ability to identify patterns and apply them when conducting other tasks the essence of being intelligent? Isn’t it instead a low-level type of intelligence? As pointed out in an interesting work on emotional intelligence [5], the need for cognitive-emotional integration in the information-processing architecture of autonomous agents is essential [6]. In the absence of this emotional part, an AI can still detect emotions as a mix of visual and audio signs (facial expressions and voice alterations, pixels and wavelengths), classify the emotional status of a subject, and trigger some known response patterns: but can the AI understand such emotional status? Again, is the capability of triggering a statistically-determined response alone a symptom of intelligence? The hype surrounding AI is due mainly to the fascination with the immediate potential and results that please our ego and generate the enthusiastic feeling of finally transcending our limitations regarding skills and time. But if we think carefully about what this tool does in most applications, we find out that this intelligence is actually quite stupid. Finally, it is crucial to point out how, throughout the entire history of mankind, many of the most important inventions and innovations have been made in the almost total absence of information and data, sometimes in a counterintuitive way, thanks to intuition and genius. One day, an AI will be activated and, in a few seconds, will spontaneously demonstrate interest in a topic, study it, and have an intuition to make the unimaginable possible – that will be the proof of intelligence. 2.3 Creativity Co-creation has become one of the favourite words of those who talk enthusiastically about AI as an assistant capable of enhancing human creative processes. This should force everyone to question the very meaning of creativity. If, for example, we consider any of the current AI-based image generators, the way they work is straightforward: a genetic algorithm takes an input and generates a set of random solutions based on a vast amount of data, then the user determines if a particular

4

G. Di Marco et al.

solution fits the desired outcome and can ask the AI to run further genetic iterations on that specific option, thus refining the result in an iterative process. Despite the considerable amount of data used to train the AI, and independently from how much the database will grow in the future, AI-based solutions will always consist, by definition, of new solutions that fit in the same pattern. In fact, there are already tools capable of detecting if an AI has created a work. The real added value of AI compared to human creativity is the possibility of generating, testing, and validating an incredible number of solutions in a reasonable amount of time and quickly helping the user get the best result. Those who believe that the human factor will always be an essential part of the creative process can see this as a confirmation of their belief. But the threshold between AI-enhanced and AI-driven is dangerously close, and the danger doesn’t come from the tool itself but rather from us as Humanity. 2.4 Humanity Let’s assume for a moment that there is a real danger of AI being so powerful that it will overcome mankind. The real question is: What should mankind do to maintain its superiority against AI? The problem can be reduced to the fundamental difference between master and user, or slave, a matter that has become relevant since digitalisation happened, and it is even more critical in the age of computation. Research conducted a few years ago on the introduction of digital technology in architecture undergraduate programs [7] pointed out how the myth of self-education based on the availability of knowledge on the web is, in fact, a myth. In order for the vast amount of information available online to become useful knowledge for the user, it is fundamental to have an expert guide capable of connecting such information in a meaningful way. In other words, a student or someone willing to undergo a self-learning program should already possess enough knowledge and experience in the specific field to separate the good content from the bad or irrelevant one. As a tool capable of producing content, an AI must also be controlled in the creative process, meaning that the user must be skilled and knowledgeable. Thinking one generation ahead, the user of AI will be an alpha-generation person, someone that we define, in the hype of this technological disruption, as a digital native, but who, in fact, will most likely be a digital slave. This person will probably have received an education largely based on simplified content presented via innovative learning and teaching methods (gamification, virtual reality, metaverse, etc.), meaning he/she will most likely have a generic understanding of complex concepts and theories, maybe knowing some of their effects but not the fundamental principles behind them. In this sense, the real danger does not come from a powerful AI but rather from a culturally weak humanity.

3 Conclusions The idea behind the xArch symposium and its motto, Creativity in the Age of Digital Reproduction (see the disclaimer at the end of this paragraph), was precisely this: to spark a critical reflection on AI and digital technologies in general and see where Humanity

Creativity in the Age of Digital Reproduction

5

stands in this new panorama, trying to avoid enthusiasm and catastrophism and focusing on the human factor, possibly thinking about an antidote not against AI per se, but against the relentless cultural flattening that this tool can entail or at least accelerate. From the work undertaken by the researchers contributing to this symposium, it is clear that in the field of digital design, at least, AI is opening up new possibilities for co-creativity in exciting and novel ways not seen before. However, the human designer has to possess the ability and the skills to filter the amount of outputs automatically generated by AI. Ability and skills that can only come from human experience and previous case studies, in a word, from culture. In education, we are duty-bound to ensure that our students embrace this rapidly developing technology ethically and responsibly while being aware of the potential risks and dangers. It is imperative that students be given the intellectual tools to employ AI resources sagely and ethically, and that these considerations do not constrain their creativity, but rather encourage it to flourish. These are dynamic and exciting times in the industry, and it is of particular importance to exploit the new tools in our toolkit to the fullest while limiting any potentially pernicious effects they may have on students’ intellectual capabilities and ethical compasses. Disclaimer. When we wrote the motto Creativity in the Age of Digital Reproduction, we were unaware that this sentence was first – and for the only time – used in 2015 [8], and by the way without particular emphasis nor additional comments. We didn’t know this work and we feel obliged to give credit to its first author.

References 1. Fuller, S.: Humanity 2.0, Palgrave Macmillan London (2011). https://doi.org/10.1057/978023 0316720 2. Carpo, M.: The Digital Turn in Architecture 1992–2012 (2012). https://doi.org/10.1002/978 1118795811 3. Dounas, T., Lombardi, D., Jabi, W.: Collective Digital Factories for Buildings: Stigmergic Collaboration Through Cryptoeconomics. https://doi.org/10.1007/978-981-19-3759-0_11 4. Carpo, M.: The Second Digital Turn in Architecture (2017). 9976.001.0001 5. Kambur, E.: Emotional Intelligence or Artificial Intelligence?: Emotional Artificial Intelligence, Florya Chronicles of Political Economy, 7 (2021). https://doi.org/10.17932/IAU.FCPE. 2015.010/fcpe_v07i20044 6. Pessoa, L.: Do Intelligent Robots Need Emotion?. Trends in Cognitive Sciences 21 (2017). https://doi.org/10.1016/j.tics.2017.06.010 7. Di Marco, G.: A reasoned approach to the integration of design and fabrication technologies in architecture education, KnE Social Sciences (2019). https://doi.org/10.18502/kss.v3i27.5553 8. Marczewska, K.: The Iterative turn, Durham theses. Durham University (2015). Available at Durham E-Theses Online: http://etheses.dur.ac.uk/11034/

Computational Design Session

Architectural Design Under Pandemic of Today’s AI: From Human-Computer Interaction (HCI) to Human-Machine Conversation (HMC) Lok Hang Cheung1(B)

, Juan Carlos Dall’Asta1 , Giancarlo Di Marco1 and Asterios Agkathidis2

,

1 Xi’an Jiaotong-Liverpool University, Suzhou 215123, China

[email protected], {Juancarlos.dallasta, Giancarlo.DiMarco}@xjtlu.edu.cn 2 University of Liverpool, Liverpool, UK [email protected]

Abstract. Human-computer interaction (HCI) has been explored in the architecture discipline since the 1960s. It stated that humans and architecture (as machines) are both designers. However, with the rapid growth of AI in recent years, misusing AI has resulted in social problems, as named the “pandemics of today’s AI” by Paul Pangaro. This paper aims to find insights into the causes and potential ways of taming the situation, followed by proposing a paradigm for developing HCI-based architecture in the future. Key HCI-based projects and recent developments will be analysed through major academia and research bodies. This research identifies three findings, including blurred definitions of interactivity, loss of physicality and lack of real-world application. Hence, we propose focusing on HumanMachine Conversation (HMC) as a more directed approach to developing HCI in architecture. This focus emphasises the “machine” as a focus on the physical built environment and “conversation” as the most critical interaction for future architecture. Keywords: Human-computer Interaction · Artificial Intelligence · Architectural Design · Human-machine Conversation

1 Introduction and Background 1.1 Ubiquitous “Pandemics of Today’s AI” As originally proposed by Marc Weiser, the concept of ubiquitous computing describes a future where technology is integrated into every aspect of the built environment [1]. In addition, Moore’s Law predicts that the number of transistors on a microchip doubles approximately every two years, resulting in an exponential growth in computing power [2]. While our lives are becoming increasingly immersed in technology, concerns have been raised from the perspective of the architecture discipline. Ranulph Glanville © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 9–17, 2024. https://doi.org/10.1007/978-981-97-0621-1_2

10

L. H. Cheung et al.

expressed his “worst computing nightmare” in 1995, where designers are unaware of being overtaken by computers because they misunderstand computers as pure tools [3]. More worryingly, with the rapid growth of AI in recent years, Paul Pangaro has warned of the pandemics of “today’s AI.” Misusing AI has resulted in social problems and gradually diminished human purposes [4]. 1.2 Research Aim and Questions Given that this inevitable phenomenon of technology integration is already occurring, it is necessary to revisit the Human-Computer Interaction (HCI) discipline within architectural design and propose a paradigm to tame this situation. To achieve the aim, three questions have to be asked. 1. How the interactions between humans and computers were originally envisioned and designed in the architecture discipline? 2. How has HCI-based architecture developed in recent years that differs from the original intention? 3. What is the future direction of research and development in HCI-based architecture? 1.3 Research Methodology To answer the first question, we would take a step back to understand how humans and computers were originally envisioned to interact. In addition, how the ideas were experimented with within the design and architecture discipline. This study stemmed from Gordon Pask, a British HCI pioneer who greatly influenced many architects such as Cedric Price and Peter Cook and major research groups such as the Architectural Association and MIT Media Lab [5, 6]. Three of his key HCI projects between the 1950s and 1970s will be studied. Then, reviews of recent developments from key research groups and individuals worldwide are conducted to respond to the second question. Not only using common databases such as SAGE Journals and Web of Science, innovative methods such as applying Connected Papers to find interconnected researchers influenced by HCI pioneers such as Gordon Pask, Paul Pangaro Ranulph Glanville, etc. A wide range of keywords might link to HCI-based architecture, including cybernetics, human-computer integration, human-building interaction, interactive built environment, etc. The main aim is to study practice-based research, including prototyping, exhibition, experiment and teaching. To answer the third question, we combined the results from the first two sections with theory-based literature discussing HCI-based architecture. After that, a paradigm is proposed (Fig. 1).

Architectural Design Under Pandemic of Today’s AI

11

Fig. 1. Research Structure

2 HCI in Design and Architecture Discipline Human-Computer Interaction (HCI) originated from cybernetics, which was introduced by Norbert Wiener, known as the “Father of Cybernetics,” as the study of “control and communication between animals and machines” [7]. The relationship between cybernetics and design has been extensively explored [8, 9]. Since the 1950s, Gordon Pask developed a series of physical prototypes that integrated HCI into the design discipline and extended it to the architecture discipline. 2.1 Project 1: Musicolour, as a Design Partner In 1953, Gordon Pask developed Musicolour, an instrument capable of listening to music and responding with a sequence of coloured lights [10]. When a musician played the piano, the computer would listen and respond with a mixture of frequency-based colours of lights. Additionally, the system had a memory mechanism that caused Musicolour to become “bored” when the musician kept playing the same range of musical notes. As the degree of “boredom” increased, the lighting’s responsiveness decreased, causing the lighting colour to become more “dull”. When the musician saw the less intriguing projections, it inspired the musician to improvise their performance [10]. Musicians using Musicolour have reported quickly feeling like they are performing “in a flow” [10]. Therefore, the computer is a design partner that makes suggestions to the musician rather than predicts or responds linearly. In other words, the computer is meant to be a design partner that collaborates and converses with humans. Also, it is important to note that one of Pask’s projects led to the establishing of the Conversation Theory [11]. 2.2 Project 2: Generator, as a Suggestive Architecture Since the 1960s, Gordon Pask brought this idea into the architecture discipline. Two different approaches were explored during this period. Firstly, HCI has explored between users and the built environment. In 1976, Cedric Price designed Generator as a retreat centre with consultancy from Julia and John Frazer [12, 13]. According to John Frazer, the Generator Project is considered the “First Intelligent Building”. The idea of creating a dynamic environment that responds to users’ needs has been introduced previously.

12

L. H. Cheung et al.

However, similar Musicolour concepts have been implemented in the built environment. In Generator, users could rearrange the space according to their desired activities with the help of a crane. However, if the configuration remained unchanged for an extended period, the building would become “bored” and suggest a new layout. This was intended to stimulate users’ imagination for new activities [12, 13]. Hence, the suggestive properties of computers from Musicolour have transferred into the built environment. 2.3 Project 3: URBAN 5, as an Architectural Design Partner On the other hand, HCI was explored in the context of the architectural design process. In this case, the role of the computer is not embedded in the built environment but within a computer design program as a design partner. The Architecture Machine Group (AMG), which later transformed into MIT Media Lab, developed URBAN5. Compared to the pre-entered design requirements, this computer drawing program conversed with designers on-screen in natural language based on the drawings that architects produced [14]. Despite there having been many obstacles during the actual practices [15], the vision is more than a “fast draftsman who doesn’t eat” [16]. In conclusion, after the 1970s, HCI has been developed in two paths, focusing on user-architecture relationships and architect-design process relationships. Meanwhile, Conversation Theory (CT) was established by Gordon Pask based on a series of experiments [11]. In the next section, we will study the recent developments from these three different perspectives (Fig. 2).

Fig. 2. Diagram of Musiclour by Paul Pangaro, Model of Generator by Cedric Price and Photo of URBAN 5 by Architecture Machine Group

3 Recent Developments and Observations 3.1 User-Architecture Driven Research In 2015, MIT Media Lab developed experiments in real-world scenarios. They developed CityOffice and CityHome, an office and apartment designed to integrate with responsive modules, including flexible partition screens and sensor-integrated apparatus such as

Architectural Design Under Pandemic of Today’s AI

13

chargers and furniture [17, 18]. In 2010, the Living Architecture Systems Group (LASG) at the University of Waterloo designed the Hylozoic Ground, which focused on provoking interactions between humans and the built environment, while the project is expressed as an art exhibition rather than a real-world scenario [19]. After years of iterations, LASG published modularised components as “Design Kits” for designing interactive architecture in 2022 [20, 21]. Instead of scenario or interaction-based, a new discipline named Human-Building Interaction (HBI) was explored in 2018 [22]. It developed small devices and frameworks to evaluate how inhabitants behave in the built environment [23]. In conclusion, there were different approaches in explorations of how the user interacts with the computer-integrated built environment, including scenario-based, interaction-provoking-based and interaction-evaluation-based. 3.2 Design Process-Driven Research Regarding the architectural design process-driven research, there are mainly two approaches to the design of possible interfaces. First, Daniel Davis developed a “flexible programming” plug-in in Rhinoceros, a standard architecture modelling software which focuses on real-time display while coding [24]. Second, the Tangible Media group of MIT Media Lab focuses on using physical objects in the real world to interact with computer programs and data [25]. Similarly, Dietmar Köring developed the Conscious City Laboratory, which focuses on a similar interface that allows people to collaboratively participate in urban planning through a touch-screen table [26]. 3.3 Theory-Driven Research Between 2016 and 2019, Defina Fantini, a PhD student of Ranulph Glanville, conducted research that critically analysed currently reactive “smart” technology from the cybernetics perspective, applying clear definitions of interactions between humans and computers by developing a smart fridge system [27]. Dietmar Köring, another cybernetician in the architecture discipline, made a significant effort to translate the idea of cybernetics into a physical installation called the Animatronic Artefact [28]. Except for projects from independent researchers, there were HCI-driven research groups in the academia. For example, the Theodore Spyropoulos Studio of the Architectural Association Design Research Laboratory focused on Behavior-Based Design and Augmented Environments. It explored the potential of computers and AI in the context of the built environment. The projects were usually presented as prototypes with computational simulations or artefacts and installations being experimented in open spaces [29]. For the Bartlett School of Architecture, the Master in Design for Performance and Interaction course explored interactions through multi-media performances [30]. Therefore, it is observed that for the research focusing on interaction and physical representation of theories, their research outcomes are either art exhibition, proof-of-concept by prototyping or simulation, or physical experiments in open space. 3.4 Findings There are three main observations that research outcomes are losing interactivity, realworld application and physicality. For example, the CityOffice and CityHome projects

14

L. H. Cheung et al.

were reactive and automatic rather than interactive. Ruairi Glynn stated that there are frequently encountered instances of confusion between reactive, automatic, and interactive systems [31]. In other cases, research outcomes mostly are either artefacts or software interfaces. Even when the physical environment was included in the research context, like HBI, computers were used to analyse the physical environment rather than being integrated into the environment to interact with humans.

4 Discussion and Proposal In 1969, Pask explicitly listed his view in ‘the architectural relevance of Cybernetics’ that ‘… a building cannot be viewed simply in isolation. It is only meaningful as a human environment. It perpetually interacts with its inhabitants, on the one hand serving them and on the other hand controlling their behaviour.’ [32]. 4.1 Conversation as the Key Type of Interaction Researchers in the architecture discipline have recognised that conversation is the key type of interaction in architecture [31, 33, 34]. What distinguishes conversation from other types of interactions is its unique feature involving a learning process, where humans and computers attempt to make agreements through iterative dialogues [35]. This is similar to how Musicolour interacts with musicians through constant understandings and explorations. Moreover, conversation has been recognised and emphasised as an architectural design approach [31, 33, 34]. 4.2 Human-Machine Conversation (HMC) as an Architecture Paradigm To tackle the first observation, researchers in architecture should shift their focus from different types of interactions to conversation. To address the second observation regarding the lack of real-world applications, it is proposed that future implications can be broadly categorised into two categories. These include HMC-based architecture, which focuses on the interactions between inhabitants and the built environment, and HMCbased architectural design processes, which focus on the interactions between designers and design mediums (e.g., computer software). Regarding the third observation concerning the loss of physicality, using the word “machine” instead of computer emphasises physicality. In the first scenario (HMC-based architecture), physicality refers to the built environment, while in the second scenario (design process), physicality can also be integrated into the inherent design environment. This is similar to how the computer is expressed as the lighting environment in the Musicolour installation. In conclusion, the idea of Human-Machine Conversation (HMC) as an Architecture Paradigm offers a potential shift in how we approach architecture and the architectural design process.

Architectural Design Under Pandemic of Today’s AI

15

5 Future Research Opportunities It is necessary to explore the proposed paradigm of HMC-based architecture in practice within the context of the architecture discipline. Two directions of research are undergoing. The first direction involves exploring conversational (AI-integrated) built environments, for example, in the context of a design studio or a meeting room, which requires the users’ creativity. Suggestive verbal and non-verbal feedback can enhance user activities, as observed in the Musicolour project. Challenges in this area include determining which elements of the built environment should serve as the medium for conversation (e.g., spatial qualities such as lighting/temperature or physical elements such as walls/furniture). Evaluating and selecting scenarios for implication is also challenging, as predictive and precise feedback is more important than suggestive on a construction site. The second direction entails investigating conversational (AI-collaborated) architectural design processes. Two layers of conversations are under exploration, including a framework of conversational design process using different AI tools such as AI art generation tools [36] and large language models. On the other hand, exploring the potential integration of AI in the existing architecture design applications to provide verbal feedback like URBAN5 as a design companion is proceeding. In both layers, we are trying to bridge the gaps between AI applications within the computational environment and physical design processes such as sketches and physical modelling. We also emphasise the necessity of human reflection as an approach to the HMC-based architectural design process. Table 1 summarises the two undergoing research stemming from the proposed HMC framework. Table 1. Two research directions under the HMC framework. Human (users)

Machine (physicality)

Conversation (medium)

Methodology

General user

AI-integrated built-environment

Spatial qualities (e.g. lighting, temperature, etc.)

Experiment through prototyping

Architectural designers

AI-integrated software and physical design objects

Reflections during the design process

Design activities

Acknowledgments. This research is supported by the XJTLU Postgraduate Research Scholarship (PGRSB211206) offered by Xi’an Jiaotong-Liverpool University.

References 1. Weiser, M.: The Computer for the 21st Century. Sci. Am. 265, 94–105 (1991)

16

L. H. Cheung et al.

2. Moore, G.: Cramming More Components onto Integrated Circuits (1965) (2021). https://doi. org/10.7551/mitpress/12274.003.0027 3. Glanville, R.: CAD Abusing Computing. In: CAAD Instruction: The New Teaching of an Architect? [eCAADe Conference Proceedings] Barcelona (Spain) 12–14 November 1992, pp. 213–224. CUMINCAD (1992) 4. Pangaro, P.: Cybernetics, AI, and Ethical Conversations. AiTech Agora Series, TU Delft, https://www.youtube.com/watch?v=VvJpkqKlv9Q, last accessed 08 November 11 2023 5. Price, C.: Gordon Pask. Kybernetes 30, 819–820 (2001). https://doi.org/10.1108/036849201 10392084 6. Cook, P.: The extraordinary Gordon Pask. Kybernetes 30, 571–572 (2001). https://doi.org/ 10.1108/03684920110391797 7. Wiener, N.: Cybernetics or control and communication in the animal and the machine. The MIT Press (2019). https://doi.org/10.7551/mitpress/11810.001.0001 8. Glanville, R.: Try again. Fail again. Fail better: the cybernetics in design and the design in cybernetics. Kybernetes 36, 1173–1206 (2007). https://doi.org/10.1108/03684920710827238 9. Sweeting, B.: Why Design Cybernetics? Presented at the July 31 (2019). https://doi.org/10. 1007/978-3-030-18557-2_10 10. Pask, G.: A Comment, A Case History, and a Plan. In: Cybernetic Serendipity, Reichardt, J. (ed.) Rapp and Carroll, 1970. Reprinted in Cybernetics, Art and Ideas, pp. 76–99. Studio Vista, London (1971) 11. Pask, G.: Developments in conversation theory—Part 1. Int. J. Man Mach. Stud. 13, 357–411 (1980). https://doi.org/10.1016/S0020-7373(80)80002-2 12. Sweeting, B.: The Generator project as a paradigm for systemic design: Relating Systems Thinking and Design 8. In: Proceedings of Relating Systems Thinking and Design (RSD8) 2019 Symposium (2019) 13. Furtado, C.L.G.M.: Cedric Price’s Generator and the Frazers’ systems research. Technoetic Arts 6, 55–72 (2008). https://doi.org/10.1386/tear.6.1.55_1 14. Negroponte, N.P.: Urban 5—an on-Line Urban Design Partner. Ekistics. 24, 289–291 (1967) 15. URBAN5: A Postmortem, https://mitp-arch.mitpress.mit.edu/pub/e6qooph1/release/1, last accessed 08 December 2023 16. Negroponte, N.: Soft Architecture Machines. The MIT Press, Cambridge (1976) 17. MIT Media Lab CityOffice: What if your space was as dynamic as you are?, https://www. youtube.com/watch?v=3Z8rGrjILX0, last accessed 08 December 2023 18. #71 CityHome, https://www.media.mit.edu/videos/labcast-71-cityhome/, last accessed 08 December 2023 19. Beesley, P., Isaacs, H., Gorbet, R.B., Ohrstedt, P.: Hylozoic Ground: Liminal Responsive Architecture - Part One (2010) 20. Living Architecture Systems Group - Living Architecture Electronics Kits, https://liv ingarchitecturesystems.com/publication/living-architecture-electronics-kit/, last accessed 08 December 2023 21. Living Architecture Systems Group - Living Architecture Exploration Kits, https://liv ingarchitecturesystems.com/publication/living-architecture-electronics-kit/, last accessed 08 December 2023 22. Alavi, H.S., et al.: Introduction to Human-Building Interaction (HBI): Interfacing HCI with Architecture and Urban Design. Acm T Comput-Hum Int. 26 (2019). https://doi.org/10.1145/ 3309714 23. Alavi, H.S., Verma, H., Mlynar, J., Lalanne, D.: On the Temporality of Adaptive Built Environments. In: Schnädelbach, H., Kirk, D. (eds.) People, Personal Data and the Built Environment, pp. 13–40. Springer International Publishing, Cham (2019). https://doi.org/10.1007/978-3319-70875-1_2

Architectural Design Under Pandemic of Today’s AI

17

24. Modelled on software engineering: flexible parametric models in the practice of architecture RMIT University, https://researchrepository.rmit.edu.au/esploro/outputs/doctoral/Modelledon-software-engineering-flexible-parametric-models-in-the-practice-of-architecture/992186 4145501341, last accessed 08 December 2023 25. Tangible Media Group, https://tangible.media.mit.edu/, last accessed 08 December 2023 26. Koering, D.: Conscious City Laboratory - explorations in the history of computation, cybernetics, and architecture; foresight for artificial intelligence and human participation within cities (2019). https://doi.org/10.14279/depositonce-8466 27. Ditmar, D.F.V.: IdIOT : second-order cybernetics in the “smart” home, https://researchonline. rca.ac.uk/2697/ (2016) 28. Koering, D.: The Technobody: an animatronic artefact as manifestation of second-order cybernetics. Proceedings of AISB Annual Convention 2018, Liverpool (2018) 29. Spyropoulos Studio, https://drl.aaschool.ac.uk/spyropoulos-studio, last accessed 08 December 2023 30. Design for Performance & Interaction MArch - UCL, https://www.ucl.ac.uk/bartlett/architect ure/programmes/postgraduate/march-design-for-performance-and-interaction, last accessed 08 December 2023 31. Glynn, R.: Conversational Environments Revisited, https://doi.org/10.1108/k.2008.06737a ab.006, last accessed 01 March 2023 32. Frazer, J.H.: The architectural relevance of cybernetics. Syst. Res. 10, 43–48 (1993). https:// doi.org/10.1002/sres.3850100307 33. Sweeting, B.: Conversation, fun, and boredom: Cybernetic approaches to intelligent environments in the work of Gordon Pask (2021) 34. van Stralen, M.: The machine for living in the conversational age. Kybernetes 44, 1388–1396 (2015). https://doi.org/10.1108/K-11-2014-0241 35. Pangaro, P., Dubberly, H.: What is Conversation? How Can We Design for Effective Conversation? Presented at the December 31 (2014). https://doi.org/10.1515/978303821284 3.144 36. Cheung, L.H., Dall’Asta, J.C.: Exploring a Collaborative and Intuitive Framework for Combined Application of AI Art Generation Tools in Architectural Design Process. Proceedings of AISB Convention 2023, Swansea (2023)

Creativity at the Edge of Digital Reproduction — from Coop Himmelb(l)au to Deep Himmelb(l)au Lei Feng(B) Xi’an Jiaotong-Liverpool University, Suzhou, China [email protected]

[…] When we speak of eagles, others think of birds; we, however, are talking about the span of their wings. When we speak of black panthers, others think of predatory animals; we, however, think of the untamed danger of architecture. […] —Wolf D. Prix 1989 (Kandeler-Fritsch, Kramer 2005: 58).

Abstract. New architectural technologies have been developed, and established throughout history, and this continues to be the case today. Looking at the Austrianbased architecture studio Coop Himmelb(l)au’s complex set of avant-garde design processes, this paper reflects upon the implications of recent and continuing advances in technologies within the fields of digital design, robotic fabrication, and artificial intelligence (AI) in the Age of Digital Reproduction, and within that purview, asks how such creativity can remain at the Edge of Digital Reproduction. Accordingly, I look back at such pivotal projects as Rooftop Remodelling Falkestrasse, Groninger Museum – the East Pavilion, BMW Welt, and Pavilion 21 Mini Opera Space; and forward at robotic fabrication processes applied in the Shenzhen Museum of Contemporary Art & Urban Planning Exhibition and the Musée des Confluences, as well as virtual and augmented reality applications and machine learning (ML) tools that are a part of the ongoing Deep Himmelb(l)au research, which we expect to transform certain techniques of the architecture industry, affect aesthetic invention itself and perhaps even bring about an extraordinary change in our very definition of architecture. Keywords: Deep Himmelb(l)au · Artificial Intelligence · Architectural Intelligence · Mind the Machine · Mind as Machine

1 Retrospect on the “Formfindung” (Kwinter, 2020: 91) In recent years, more and more academics have promoted the idea that humankind has entered the “Fourth Industrial Revolution.” Whether we agree or not, architectural technologies have developed within the framework of a genealogy of industrial-technological © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 18–24, 2024. https://doi.org/10.1007/978-981-97-0621-1_3

Creativity at the Edge of Digital Reproduction

19

advancement. New technologies, in type and function, have been established, that are very different from those of the past and even the recent present. Among those playing critical roles are artificial intelligence and digital reproduction. Coop Himmelb(l)au is a Vienna-based architecture studio. Its philosophy focuses on challenging convention, embracing new technologies, and creating buildings that respond to the dynamic needs of the modern world. This atelier has always been keen on developing and executing geometrically complex, radical and futuristic architectural projects. Its avant-garde design methodology and approach toward architecture are often characterized by a bold design language, a sculptural geometry, and an emphasis on fluidity and movement. Through looking into Coop Himmelb(l)au’s complex set of design processes over their past half century, I want to reflect on the transition from Coop Himmelb(l)au’s early experimental design methods to their more recent and continuing advances in technology within three fields – digital design, robotic fabrication, and artificial intelligence – in order to conceptualize the evolution of change from Coop Himmelb(l)au to Deep Himmelb(l)au. I will start by looking at the early experimental methods in their start-up projects, their utilization of digital design tools and 3D simulation in later classic projects, some more advanced parametric design processes applied in small-sized projects conducted after the millennium, and finally, robotic fabrication in the recent large-scale cultural projects. Coop Himmelb(l)au projects have always been conjoined with technological overtures and innovations. It is thanks to the ideas and methods they have created, the adaptability and experimentality they have attained, together with their complex set of design thinking and processes, that those milestone projects were accomplished. Experimenting in the 1980s with their specific “early methods of drawings” (Formfindung), Coop Himmelb(l)au’s point of departure projects, Rooftop Remodelling Falkestrasse (1983– 1988) and Open House (1983–1988) took provocative design language and challenged the rigidities of modernist architectural dogma. In the case of Open House, Coop Himmelb(l)au began with a spontaneous unconscious sketch as a plan, drawn with eyes closed in intense concentration; this became “an imprint of an emotion of a future space, […] a psychograph, transferred into the third dimension by reading the plan as section and elevation.” (Prix, 2020: 15) ( Fig. 1).

Fig. 1. Rooftop Remodeling Falkestrasse Model © Maskus Pillhofer, Sketch © Wolf dPrix

20

L. Feng

A later classic project is Groninger Museum – the East Pavilion (1993–1994), for which the so-called “space arm” was first applied. It was used to digitize a 3D model that was then translated into plans and drawings, the data then being transferred via modem to a shipyard. The project was based on the idea of finding and optimizing solutions for not yet smoothly defined design issues and allowing “an until then unknown latitude in designing fantastic forms, spaces, and structures” (Prix, 2020: 79). Based on studies of natural and artificial light, the design team first developed the spatial sequence, “then superimposed upon the drawing of the liquid space” (Prix, 2020: 15). Back then in 1993, the superimposing process was done on a copier, nowadays it would be done on a computer, of course. BMW Welt (2001–2007), an iconic Coop Himmelb(l)au project which marks the studio’s maturity, involves a convergence of low-tech and high-tech approaches in the design and construction process. At that time, techniques had attained the adaptability and precision of digital design. As Wolf. D. Prix, co-founder of Coop Himmelb(l)au, put it “[W]hat [is] still radical is wrestling these images from a onedimensional illustration and pushing them through – realizing them – in three dimensions” (Prix, 2020: 139). Utilization of the digital design tools went beyond conceptual drawing and moved into the further planning phases, “[a] 3D simulation of thermal currents and air streams was conducted to investigate the spread of exhaust fumes from cars driven on the [p]remiere level” (Prix, 2020: 140). Pavilion 21 Mini Opera Space (2008–2010), Martin Luther Church (2008–2010), and PANEUM – House of Bread (2014–2017) are three small-sized projects conducted after the millennium, each applying specific design methods. Among those experimental projects, Pavilion 21 Mini Opera Space, was a temporary performance space with lightweight construction, that can be dis- and re-assembled quickly; nonetheless, it can achieve marvelous acoustics and fulfils the sophisticated requirements of a concert hall. Drawing support from parametric “scripting,” it created a music-generated space and completed the transition from abstract and formless sound (a remix of Jimi Hendrix’s Purple Haze and Mozart’s Don Giovanni) to figurative and tangible soundscapes. Frequencies from both soundtracks are translated into pyramidal pop-outs, defining the computer-generated 3D model and finally becoming the “spike constructions” of the façade. Rigid disciplinary boundaries dissolved, with music becoming spatial. Combining architecture with music is not a new invention; the term “soundscape” originated in the 1940s’ designating a method of spatial composition. When Iannis Xenakis and Le Corbusier thought about the three-dimensional implementation of musical compositions, they jointly engaged their music and architecture, as applied in Le Corbusier’s Philips Pavilion and La Tourette’s windows partition (Fig. 2).

Creativity at the Edge of Digital Reproduction

21

Fig. 2. Pavilion 21 Mini Opera Space Design Process © Coop Himmelb(l)au

As for Coop Himmelb(l)au’s recent large-scale cultural projects, robotic fabrication processes were applied in the Shenzhen Museum of Contemporary Art & Urban Planning Exhibition (2007/2013–2016) and the Musée des Confluences Lyon (2001/2010– 2014). Both projects enabled human-machine cooperation, created a full-scale digital construction environment, and provided flexible and extendable on-site configuration. Shenzhen Museum of Contemporary Art & Urban Planning Exhibition is one museum with a synergetic combination of two cultural institutions. Digitalization in this project was done a twofold process that covered both design and fabrication. Digital Project is an architectural visual interface developed by Gehry Technologies. It was applied in Shenzhen to examine what parameters might influence each other during the planning and construction phases. The design team programmed the primary structural elements, the steel structure, the facade, and skin while using the BIM system to detect potential collisions on the construction site. For such extremely complex projects with such irregular geometries, a crucial aspect in achieving a high level of precision is the exceptionally detailed 1:1 scale models. In a short video, Coop Himmelb(l)au demonstrated that through human-machine cooperation, utilizing robotic fabrication can assist human workers to build faster and more economically while offering architects the maximal freedom to explore a “new aesthetic.” (Fig. 3).

22

L. Feng

Fig. 3. Human-Machine Cooperation in Musée des Confluences Lyon Model Making © Maskus Pillhofer

2 Mind the Machine vs. “Mind as Machine” (Boden: 2006) In recent years, among multitudinous technologies, the growth of artificial intelligence and its integration into architectural design has brought a new era of astounding innovation accompanied by efficiency like never before. Artificial intelligence is inevitably transforming and (re)defining today’s architecture industry. While the world celebrates the game-changing potential of AI, the other side of the AI coin is one of a trembling anxiety. In the world of architecture, this becomes manifest in the possibility that AI might replace architects in actual practice and become a threat to the creative process. The question is: Mind the Machine or Mind as Machine? While a considerable number of professionals share their concerns over the possibility of AI replacing actual architectural practice, Deep Himmelb(l)au, an ongoing deep machine learning research process, is seeking an alternative by training AI to mimic human intelligence and becoming a creative force that can be used in a variety of design fields. Deep Himmelb(l)au assists the design team with ideation and inspiration. It is an option for continuing advances in human-machine cooperation. Doubtless, Deep Himmelb(l)au is future-oriented, but at the same time, it relates profoundly to Coop Himmelb(l)au’s past. “The Past (with a capital letter),” as Jerome S. Bruner put it, “is a construction: how one constructs it depends on your perspective towards the past, and equally on the kind of future you are trying to legitimize” (Bruner, 1997: 279).

Creativity at the Edge of Digital Reproduction

23

The transition from Coop Himmelb(l)au to Deep Himmelb(l)au can be seen as a natural evolution in this studio’s practice and history. During the past. The best architectural ateliers always embrace technological overtures and innovations; thus, using Deep Himmelb(l)au as a deep learning tool, Coop Himmelb(l)au draws on experience and creativity to generate designs, sometimes combining virtual and augmented reality applications. As in other creative industries, significant technological innovations will transform the entire technique of architecture design, affecting architectural aesthetic invention itself, perhaps even transforming our very notion and definition of architecture.

3 Artificial Intelligence vs. “Architectural Intelligence” Much like feathers were developed by nature without dinosaurs ever thinking about flying, AI —which I like to call “Architectural Intelligence” (Prix, 2021; Steenson, 2022; Yuan, 2023)—is a tool that will one day allow us, architects, to fly. —Wolf D. Prix. Artificial intelligence (AI), including its subsets machine learning (ML) and generative design, can emulate human cognitive processes. By utilizing extensive amounts of data with intelligent algorithms, AI tools can swiftly generate optimal design results. Deep learning tools can study existing data patterns and features to produce new design content. Past experiences serve as a framework for interpreting and reconstructing the present and future, helping us understand and create fresh aesthetics models. After setting particular creativity criteria, can AI generate content without guidance? To what extent can AI construct interdisciplinary representations that cross among creative domains? In Architectural Intelligence: How Designers and Architects Created the Digital Landscape, Molly W. Steenson uses the term “Architectural Intelligence” to describe those architects “who engaged with cybernetics, artificial intelligence, and other technologies poured the foundation for digital interactivity” (Anonymous, 2022). Coincidentally, in 2021, during an online lecture at Tsinghua University, China, Wolf D. Prix accidentally used the term to criticize the current tendency of overusing parametric and AI tools. (Whether there is a necessary connection between the two “Architectural Intelligences” is impossible to verify). Steenson systemically investigates how certain architects “pushed the boundaries of architecture—and how their technological experiments pushed the boundaries of technology. What did computational, cybernetic, and artificial intelligence researchers have to gain by engaging with architects and architectural problems?” she asks (Anonymous, 2022); Prix expresses the concern that the architectural creative process is under threat because of the uncritical, passive use of parametric design and artificial intelligence tools in todays’ totally digitized environment, and that architecture will lose its autonomy in the age of digital reproduction.

4 From Coop Himmelb(l)au to Deep Himmelb(l)au Visual interpretation or design generation through Deep Himmelb(l)au is only part of Coop Himmelb(l)au’s design processes. Within the architectural design field, creativity and complexity require a perfect balance between aesthetics and functionality, which

24

L. Feng

further requires an exquisite equilibrium between human skills and technological tools. As a profession requiring multiple skills, the architecture disciplines converge upon structural and environmental engineering and social and material sciences. Managing to ideate and get the creativity juices flowing within a pool of data, Deep Himmelb(l)au contributes to enhancing its design team’s comprehensive knowledge concerning functionality, and enriching their instincts and sensitivities regarding “Formfindung” and aesthetics, freeing the team from routine and repetitive tasks, and so optimizing and pushing the boundaries of design. For Coop Himmelb(l)au, it is essential to consider Deep Himmelb(l)au as an experimental research project. With the result of their cumulative research efforts, it operates at the confluence between AI and deep learning, architectural design and construction, without however completely turning over control of the design process to artificial intelligence.

5 Conclusion Creativity in the Age of Digital Reproduction vs. On the Edge of Digital Reproduction In conclusion, I would like to borrow a notion from Walter Benjamin’s famous essay The Work of Art in the Age of Mechanical Reproduction, and look ahead into the rich realm of (re)defining the field of architecture. “We must expect great innovations to transform the entire technique” [of the architecture industry], “thereby affecting architectural invention itself and perhaps even bringing about an amazing change in our very notion of” (Benjamin, 1935) architecture. Always embracing technological innovations, Open House represents a mindset directed towards an open architecture. In the Age of Digital Reproduction, architectural creativity shall stay open and stay on the Edge of Digital Reproduction.

References Anonymous. Architectural Intelligence: How Designers and Architects Created the Digital Landscape. https://doi.org/10.7551/mitpress/10971.001.0001. Accessed 13 August 2023 Benjamin, W.: The work of art in the age of mechanical reproduction. In: Illuminations, edited by Hannah Arendt, translated by Harry Zohn, from the 1935 essay. Schocken Books, New York (1969) Boden, A.M.: Mind as Machine: A History of Cognitive Science. Volume 1, p. xxxvii. Clarendon Press, Oxford (2006) Kwinter, S.: Dynamic forces: architecture and combustition. Urban Environment Design 126, 88–93 (2020) Prix, D.W.: Finishing the Tower of Babel: Exclusive Interview with Wolf D. Prix. In: Urban Environment Design 126, 12–17 (2020) Prix, D.W.: On the Edge. In: Martina, K.-F., Kramer, T. (eds.) Get Off of My Cloud: Wolf D. Prix COOPHIMMELB(L)AU Texts 1968 – 2005, p. 58. Hatje Cantz Verlag, Ostfildern-Ruit (2005) Steenson, M.W.: Architectural Intelligence: How Designers and Architects Created the Digital Landscape. The MIT Press (2022). https://direct.mit.edu/books/book/3643/Architectural-Int elligenceHow-Designers-and, https://doi.org/10.7551/mitpress/10971.001.0001. Accessed 13 August 2023

Computing Analogue Interactive Installations Michail Georgiou1(B) , Odysseas Georgiou2 , and Eva Korae3 1 University of Nicosia, Nicosia, Cyprus

[email protected]

2 SEAMLEXITY LLC, Limassol, Cyprus

[email protected]

3 Cyprus University of Technology, Limassol, Cyprus

[email protected]

Abstract. This paper documents the development and application of a set of computational tools and fabrication methods to support and facilitate the design, simulation and realization of 3D Moiré Animation installations. Setting-out from the technique of traditional 2D Moiré Animations, the authors developed tools to examine a novel approach which combines the depth of field and motion of the spectator to achieve large-scale, analogue animations in three dimensions. Furthermore, the authors suggest that large scale outcomes can enhance the way people interact with outdoor spaces. For that hypothesis the particular paper illustrates the application of the tools for the realization of two large-scale interactive analogue motion graphic installations; a memorial and a temporary centrepiece for a dance festival in Cyprus. The tools operate as a free plugin for Grasshopper 3D and can be downloaded. Keywords: 3D Moiré Animation · 2D Moiré Animation · Computational Design · Digital Fabrication · Interactive Installations

1 Introduction This study presents current developments and outcomes of ongoing research, aiming towards supporting the design, simulation and realization of 2D and 3D Moiré Interactive Installations. A set of computational tools developed within the context of the research, and the application of digital fabrication techniques aims at automating the design and production of physical large scale analogue animation apparatuses created by the interaction of two superimposed layers. While for 2d Moiré Animations one needs to create a “grating” (a transparent screen with plain, evenly spaced strips or slits) and a “composite” image formed by parts of the frames in the animation, this technique is based on creating the circumstances to mask all but one frames of the animation at a time, using the grating layer. This successive registration of frames generates an “apparent motion” effect, an illusory phenomenon of movement that occurs when “two or more adjacent stimuli are briefly presented, one after the other [1]. This paper documents a novel adaptation of the technique for large scale applications within the built environment. The traditional 2d Moiré Animation method [2], is © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 25–33, 2024. https://doi.org/10.1007/978-981-97-0621-1_4

26

M. Georgiou et al.

therefore modified to meet new parameters and constraints imposed by the suggested 3d Moiré Animation technique [3]. It is to be noted that Moiré abstract patterns and effects, produced by superimposing two “gratings”, are considered by the authors a separate study set and a different category from Moiré Animations, which produce legible movement of specific images. A flock of flying birds and a performer’s walking figure are examined for the purpose of this study. Since 2D Moiré Animation is based on a limited amount of frames (usually 4– 6), improved technical design, careful selection and preparation of suitable subjects are required to ensure legibility and flow of the animation [4]. Such process requires constant revisiting and visual evaluation of the outcome involving laborious and time consuming workflow through which the designer needs to slice and recompose each frame of the animation. Additionally, variations of the width of the grid can affect parameters like speed and legibility of the animation [5]. 3D Moiré Animations on the other hand, are guided by the properties of perspective projection [6] since it is the spectator who moves rather than the two surfaces. The setup becomes even more complex as new parameters affecting the results are added, such as: • The grating material thickness (e.g. a 10 mm thickness offers a different reading than 3 mm) • The spacing of the two layers (e.g. 50 mm offers a different reading than 40 mm) • The spectators speed and • The spectators viewpoint To address the aforementioned space parameter, the design team opted for developing a set of computational tools to assist the design workflow. The tools under the name ZEBRA enabled the exploration of a large number of design options and the visual optimization of the selected results through numerous legibility assessment iterations. The above tools were directly associated with digital fabrication techniques which became an integral part of the workflow. Within the above framework, the research expands upon the findings of two realized case studies and the computational tools developed as part of their design workflow. The juxtaposition of the two case studies presents an evolution in animation complexity and verifies the validity of the tools in facilitating design, simulation and construction of large scale prototypes. The research and projects presented depart from existing work in the field [2, 4, 7] as they present applications of the technique in fundamentally different context and viewing conditions than any of their predecessors. It is further proposed that 3D Moiré Animation could potentially have applications in other fields, such as architecture [6], depending on what the commissioning authority intends to achieve. Drawing from examples like the “Inogon Naval Lights” [8], it is suggested that the outcomes may exhibit applied uses such as signage for wayfinding, traffic calming methods or even destination art in public spaces. The technique could therefore be useful in applications requiring low maintenance or affordable and sustainable alternatives enabling large scale animation effects.

Computing Analogue Interactive Installations

27

2 Methodology and ZEBRA Computational Toolset This paper expands upon the definition of 3D Moiré Animation as analogue and low maintenance large-scale legible animations on physical structures which are composed by spacing the grating and the composite image on a static apparatus [3]. The effect is therefore generated by the movement of the spectator, while the structure remains static as opposed to 2D Moiré where all the layers are required to move in different directions. ZEBRA was developed to assist the above workflow by automating the steps associated with the procedure. The following chapter presents the stages of the process along with the computational tools developed during each phase. 2.1 2D Moiré Animation The Moiré effect is a well-known phenomenon which occurs when repetitive structures (such as screens, grids or gratings) are superposed or viewed against each other. It consists of a new pattern of alternating dark and bright areas which is clearly observed at the superposition although it does not appear in any of the original structures [9]. In the case of 2D Moiré Animation, the goal is to produce movement/animation for specific images when a layer of specific grid lines or gratings interacts with a previously sliced image, joining those patterns into legible movement. The width of those lines depends on the desired resolution of the animation, the number of frames and even the viewing speed which is generated by the user. (see Fig. 1).

Fig. 1. 2D Moiré Animation by Rufus Batler Seder.

The ZEBRA tool is developed as a user cluster in Grasshopper 3D and consists of a number of components that automatically produce the two constituent parts of a 2D Moiré Animation (a grating and an interlaced image). Additional utilities enable the visualization of the animation. ZEBRA operates by reducing each frame of an original animation into vertical strips, and later combining them into one composite image. The grating is then produced based on the number of animation frames and the user desired resolution/speed. The toolset accepts Closed Curves as input and any number of frames is permitted [3]. In contrast to the traditional black-on-white-background layout (see Fig. 1.) of the technique, ZEBRA also offers the option of inverting the composition colours. The operation improves legibility at large distances especially when the background is artificially lit. (see Fig. 2).

28

M. Georgiou et al.

Fig. 2. Inverted Transparent Mask layer (Grating), Composite image and Result

Finally the user is able to visually examine the legibility and flow of the animation using the 2D Animator component which translates the grating horizontally over the interlaced image, revealing one frame at a time, and therefore enabling the animation effect. 2.2 3D Moiré Animation 3D Moiré capabilities were incorporated into ZEBRA by the formation of a set of components which synthesize the grating and the interlaced image into a 3D Apparatus based on two parameters defined by the user; material thickness and layers’ spacing (see Fig. 3). Visually examining the results of the animation at this stage pre-supposes additional user input to define the spectator’s path (distance to apparatus) and speed.

Fig. 3. Exploded view of a 3DMoiré Apparatus

To assess legibility of 3D Moiré Animations, the impact of the combination of all 2D and 3D Moiré parameters had to be investigated. The difficulties of assessing all possible iterations with realistic visual outputs lead to establishing a parametric raytracing framework able to produce analytic results. The framework was formulated

Computing Analogue Interactive Installations

29

using Grasshopper 3D plugin in Rhino 3D with iterative loops using the Anemone plugin and Microsoft Excel integration using GHowl plugin. Custom coded C# components were also used in conjunction to the above. Data recording and graph plotting would yield useful findings on the connection between frame visibility and frame transition overlap. Comparing graphs and metrics enabled the research team to assign a legibility rating to each design option, assuming constant viewer speed and distance from the grating layer (see Fig. 4). Despite the fact that the raytracing analysis was conducted for the purposes of the case studies, such functionality is not yet part of ZEBRA toolset. Future work aims at incorporating a legibility rating component.

Fig. 4. Comparison of raytracing data graph to determine the best version in terms of animation legibility. Graph Above: 35 m distance/10 mm slits. Graph Bottom: 10m distance/10 mm slits

2.3 Digital Fabrication Mainstream Digital Fabrication techniques (Laser-Cutting) were used in both projects to fabricate the grating and the composite image. Especially the complexity of the latter would render it extremely difficult to be realized using other means or methods. Due to the slenderness and height of the slits, the grating layer had to be structurally enhanced with horizontal connections to avoid weakening and swaying of the surface and therefore distortion of the animated image. A random horizontal connection pattern which would not interfere with the animation was computationally calculated and produced. Furthermore, the same algorithm enabled random reduction of slits to save on fabrication time while producing visually interesting and non-repetitive grating panels. This functionality is not yet included in ZEBRA toolset. Future work aims at incorporating a fabrication utility component.

3 Prototypes This chapter expands on two realized case studies the juxtaposition of which illustrates an evolution in animation complexity and verifies the validity of the tools in facilitating design, simulation and construction of large scale prototypes.

30

M. Georgiou et al.

3.1 Mari Memorial The memorial is a private commission by the community of Mari in Larnaca, Cyprus to commemorate the 13 people fallen in a military accident, an explosion which left a large crater on the ground at the nearby Naval Base on 11th July 2011. Construction work was completed in 2017. The project is defined by two interrelated elements with symbolic references; a path/wall and a cavity/crater. The path is framing the cavity forming the base of a weathered steel wall. The wall runs along the exterior edge of the pathway. This vertical steel element comprises 13 incrementing distinct steles (see Fig. 5). The steles act as individual 3D Moiré Animation apparatuses which are then perceived as an ever flying flock of birds. The effect is designed to be visible from the road while driving a vehicle.

Fig. 5. Mari Memorial Steles (Photographer: C. Solomou)

3.2 Ripples “Ripples”, was a temporary installation commissioned by the 15th Nea Kinisi Dance Festival, organized by the Cyprus New Movement Dance Groups, Dancers and Choreographers. The interactive installation was designed with the purpose of attracting and engaging passers-by through animating a dancing figure. It comprises 12 individual panels forming a seamless horizontal surface, supported on a metal structure, which hosts the 3D Moiré Animation effect (see Fig. 6). The animation is visible from close proximity and was designed to engage pedestrians, skaters, cyclists etc. The project was on public display for two weeks during the Limassol Dance and Documentary Festivals in July 2018. 3.3 Comparison of Case Studies The goal when designing Ripples was to achieve a continuous animation as opposed to individual animations on each panel of the Mari Memorial. Departing from distinct animations which is the predominant paradigm of traditional 2D Moiré animations became

Computing Analogue Interactive Installations

31

Fig. 6. Ripples Installation (Photographer: P.Vrionides)

a challenge for the design team. The task was directly related to the arrangement and number of frames involved. Mari Memorial was composed using only 4 overlaid frames which created the movement of each individual bird in a repetitive loop. Ripples on the other hand, used 6 frames per loop organized in 11 groups resulting in a composition of 66 distinct frames forming a procession (see Fig. 7). In contrast to Mari Memorial, each animated loop was not constrained to a single panel, but certain frames would be shared between adjacent panels. While in Mari Memorial each frame was designed, in the case of Ripples the requirement was to produce the 12m animation by recording the actual movement of a real life performer. The recording process defined the scale of the installation. As a result, two sessions of sequential photos of the performer were taken from the same spot resulting in a long animation strip. At the end of the first half of the distance the performer “jumped” out of the frame and “landed” in the next section to conceal discontinuities in the animation and to avoid distortion of the images. The photo stills were taken with the use of a green screen and stripped from their background to produce the figures of the animation frames. It was decided to select 66 key frames to achieve a smooth animation flow.

VOID

A

B

C

D

E

F

G

H

I

J

K.AT

Fig. 7. Ripples Installation, key frames

Regarding the spectator, Ripples was designed to engage people with arbitrary paths rather than moving motor vehicles with predictable route and speed. Adaptation to the viewer path resulted in different layouts for the two projects. Mari Memorial is arranged along a curved path parallel to the road, whereas for Ripples the choice was a linear arrangement of panels parallel to the boundaries of the location and based on observations and assumptions for pedestrian movement across the site.

32

M. Georgiou et al.

Legibility of the animation effect relied, among other factors, on a high contrast between the grating and the recomposed image. In Mari Memorial this was achieved using a uniform natural light-coloured background (white gravel), as opposed to the darker tone of the steles, which is artificially lit during the night to achieve the requested contrast. In the case of the Ripples installation and due to the absence of background uniformity, a semi-transparent layer (white polyester sheet) was stretched at the back of the installation surface and artificially lit during the night to enable the effect. The transparency of the background allowed visibility of the animation during daytime.

4 Conclusion This paper expands upon the definition of 3D Moiré Animations [3] and illustrates through two realised case-studies that the process enables built-environment applications offering low maintenance and affordable alternatives to current technologies enabling large-scale animation effects. Furthermore, the juxtaposition of the two consecutive case studies illustrates an evolution in animation complexity and verifies the validity of the tools in assessing legibility of the animation effect while facilitating design, simulation and construction of large-scale prototypes. In parallel, ZEBRA computational toolset is described. The plugin aims to facilitate the design, simulation and evaluation of 2D and 3D Moiré Animations for the built environment. The framework is assembled in Grasshopper 3d with the use of custom scripts written in C# programming language and most of its clusters are currently available as parts of ZEBRA. The plugin was set-up to interactively generate the constituent parts of 2D and 3D Moiré Animations while enabling the visualisation of the animation results. Future work aims at incorporating the 3D Moiré Animation legibility rating component and digital fabrication utility components in ZEBRA toolset. Acknowledgement. Mari Memorial was funded by the Community Council of Mari, Larnaca, Cyprus and supported by the Committee of the Relatives of the victims of the Mari explosion. We thank Theresa Kwok, Stavros Voskaris, Christos Xenofondos and Maria Ioannou for their assistance during the design and construction phases of the project. Ripples was funded by the 15th Nea Kinisi Dance Festival, and supported by SEAMLEXITY, the Multimedia and Graphic Arts Department of the Cyprus University of Technology and the Architecture Department of the University of Nicosia. We thank the performer Eleana Alexandrou and the volunteers Christina Christoforou, Natalie Moiseenkova, Rasha Zeneddin, Panagiotis Hadjioannou, Valentinos Charalambides, Stephani Milikouri, Costantina Yiannapi, Periklis Georgiou, Emilio Moraris and Maria Efthimiou for their assistance during the design and construction phases of the project.

References 1. Sperling, G.: Comparisons of real and apparent motion, in Journal of the Optical Society of America, AMER INST PHYSICS CIRCULATION FULFILLMENT DIV, 500 SUNNYSIDE BLVD, WOODBURY, pp. 1442 (1966)

Computing Analogue Interactive Installations

33

2. Phillips, D.: Barrier-Grid (or Picket-Fence) Animation | Illusion demos with animations# |Optical Illusions, Oct. 30, 2012. https://www.opticalillusion.net/optical-illusions/animated-moireor-scanimation/ (accessed Jul. 27, 2023) 3. Georgiou, O., Georgiou, M.: ZEBRA | COMPUTING MOIRE ANIMATIONS (2018) 4. Herbert, S. (n.d.): The Optically Animated Artwork of Rufus Butler Seder, http://www.rufusl ifetiles.com/TheOpticallyAnimatedArtworkofRufusButlerSeder.pdf (accessed Jul. 27, 2023) 5. Shan, X., Chung, J.-H.: Research on the production method of three-dimensional image scanimation. J. Digit. Converg., 14(12) (2016) 6. Brzezicki, M.: Designer’s controlled and randomly generated moiré patterns in architecture, Proceedings of GA2011 – 14th Generative Art Conference, Domus Argenia, Italy, pp. 288–301 (2011) 7. Chiou, C.: Investigating the “Blurry” Territory of Graphic Design, a Look at the Simultane-ous Realities of Illusions within the Moiré Effect, Master Thesis, York University, Ontario, Toronto (2016) 8. Bergkvist, L. A, Forsen, I.: Leading Mark Indicator, Sweden, 4629325 (1986) 9. Amidror, I.: Introduction. in: The Theory of the Moiré Phenomenon, Computational Imaging and Vision, vol 15. Springer, Dordrecht (2000)

UAV-Based Geometry Data Acquisition for Building Energy Modelling Mengfan Jin(B) and Marco Cimillo Xi’an Jiaotong-Liverpool University, Suzhou, China [email protected], [email protected]

Abstract. Building Energy Modelling (BEM) is a critical tool for various building energy-related applications, such as energy efficiency diagnosis, certification, and retrofit design. Accurate building geometric, non-geometric, and weather data are crucial for effective BEM. Conventional onsite measurement methods can be laborious and time-consuming. Furthermore, after the onsite work, creating the energy simulation model has disproportionately outweighed the attention given to highvalue engineering and energy analysis within the retrofit workflow. Unmanned Aerial Vehicles (UAVs) enable swift data collection, bypassing lengthy manual processes and offering potential solutions. This study used aerial imagery captured by a UAV to create a geometry model of a residential building in Suzhou, China, aiding energy assessment. Specialised software can convert photos into a detailed 3D model while lacking semantic information limits its utility beyond visualisation. Therefore, post-processing was conducted to generate a complete geometric model. The outcomes were compared to conventional measurement methods against accuracy and processing time. The study demonstrates that the application of UAV-based photogrammetry can semi-automatically reconstruct the building envelope with high precision, providing valuable geometry input for building energy assessment. Keywords: UAV · Building energy modelling · Data acquisition

1 Introduction 1.1 Building Energy Modelling (BEM) In 2021, building operations constituted around 30% of global final energy consumption [1]. The Paris Agreement - COP2 prompted many nations to improve building energy efficiency and prolong building lifespans. BEM is essential to evaluate current and future building performance and guide retrofit strategies. BEM employs physical models to analyse heat and mass transfer within buildings, predicting energy usage. This process involves five main stages: (1) Acquiring input data, (2) Generating geometry models, (3) Generating energy models, (4) Simulating, and (5) Validating. This study focuses on optimising the initial steps of input data acquisition and geometry model generation for more efficient energy simulation model creation. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 34–41, 2024. https://doi.org/10.1007/978-981-97-0621-1_5

UAV-Based Geometry Data Acquisition

35

Input data for BEM includes geometric, non-geometric, and weather parameters. Geometric data includes building footprints, height, Window Wall Ratio (WWR), and floor area. Non-geometric data covers envelope thermal parameters, HVAC systems, occupancy schedules and weather data. Once acquired, these data will be integrated into an energy model and simulated via a simulation engine. When acquiring geometric data for BEM, the most direct approach involves generating a geometric model from construction drawings. However, this method can be time-consuming and inaccurate if plans are incomplete due to retrofits. Open-source databases offer an alternative for existing buildings if drawings are not available. Some regions provide building details like footprint, height, stories, construction year, and type via Geographic Information System (GIS) databases. Specific cities, e.g., Berlin, Rotterdam, and New York, offer public access to 3D city models in the CityGML format. However, these data are not accessible in China due to governmental control and privacy regulations. Onsite surveying is an alternative geometry data source. However, manual tools like measuring tape and laser range finders may increase errors and are impractical for large-scale applications. Advanced methods like total stations and laser scanners enhance accuracy but reduce efficiency. Therefore, achieving a balance between accuracy and efficiency holds significance. Currently, retrieving data from online maps like OpenStreetMap and Baidu Map is the main practical alternative for large-scale applications in China, and therefore, this method will be used for comparison when evaluating the workflow proposed in this paper. Such sources offer building footprints and, in some cases, 3D massing reconstructions. When unavailable, building height and WWR can be assumed according to construction standards. Height is often computed by multiplying stories with floor height (typically 2.8 m to 3 m) per regulations. 1.2 Unmanned Aerial Vehicles (UAVs) and Oblique Photogrammetry UAVs, commonly known as drones, are autonomous or remotely operated vehicles equipped with onboard sensors. Initially developed for military use, UAVs have gradually transitioned towards commercial applications in recent decades. Nowadays, UAVs are widely used in the architecture, engineering, and construction industries. In contrast to traditional surveying, UAVs offer advantages in speed, efficiency, and cost-effectiveness. UAVs facilitate rapid data acquisition and analysis by eliminating manual measurements and streamlining procedures. One common technique in UAVbased 3D modelling is the Structure-from-Motion (SfM) algorithm [2]. In the SfM process, key features are identified in multiple images, and the relative orientation of the cameras is computed based on these matches. The output of the SfM algorithm includes precise camera positions and viewing directions, and a 3D point cloud representation of the building. This point cloud can be further processed and transformed into a closed polygon mesh to represent the building’s envelope [3]. While prior research [4] has explored UAV-based data acquisition workflow, the postprocessing aspect, particularly geometry model generation crucial for building energy assessment, received limited attention. This study aims to address this gap by investigating UAV-based building geometric data acquisition. The goal is to establish a streamlined and replicable workflow for future energy modellers. By optimising UAV-based

36

M. Jin and M. Cimillo

data acquisition, the research aims to enhance accuracy and efficiency in building energy assessments through reliable geometric data.

2 Methods The field trial was conducted at Sanxiang Xincun community in Suzhou, China. The case building is a representative example of a six-floor Danyuan building constructed in 1990. This building type represents a key target for renovating old residential communities in China. However, the original construction drawings remain inaccessible. Figure 1 illustrates the workflow to develop the geometric model to assist BEM. Task definition

Sanxiang Xincun, Gusu district, Suzhou Site observation: UAV obstacles

Pre-flight preparation

Pre-flight checklist Flight path pattern, altitude, distance from surface

In-flight data acquisition Post-flight processing: Photogrammetric 3D reconstruction

Automatic / manual UAV-based acquisition of data RGB images with real camera position and orientation, timestamp Structure from Motion and 3D reconstruction

Post-flight processing: Geometry model generation

Aerial triangulation Dense stereo matching

Assign thermal parameters and occupancy schedule

Generate dense point cloud

Energy model generation

Point cloud to SketchUp model

Fig. 1. Proposed UAV-based geometry data acquisition workflow

2.1 Pre-flight Preparation and In-flight Data Acquisition The field test employed a DJI Matrice M210 V2 drone, calibrated per the user manual. Weighing 4.8 kg, including a pair of TB55 batteries, the UAV is equipped with a 12 MP resolution 1/1.7 CMOS sensor. The UAV offers a maximum flight time of 34 min without payload. With three pairs of batteries, the UAV enabled a total flight time of around 90 min, including take-off and landing time. A pre-flight checklist adapted from a previous study [5] ensured operational safety and efficiency: (1) Confirm the case building is not in a no-fly zone, (2) Obtain the drone piloting license, (3) Conduct site observations in advance to avoid potential obstacles, such as trees, wires, and tall buildings, (4) Plan the flight path according to the actual situation, (5) Make sure outdoor environmental conditions are suitable for flight, avoiding rain, snow, and heavy wind, (6) Explain the test’s objectives and procedures

UAV-Based Geometry Data Acquisition

37

to the households to get cooperation, and notify occupants to minimise radio and Wi-Fi interference, (7) Check the batteries of the UAV and the controller. Flight path planning is fundamental for a survey mission. Flight pattern, altitude, and overlap are essential to achieving accurate and comprehensive results. Researchers [6] outlined four typical types of flight patterns applied in 3D surveys and modelling: polygon, grid, double grid, and circular patterns. Each pattern served distinct survey objectives and provided unique advantages. Another study [4] mentioned that the flight height differed from different missions, and both grid and elliptical patterns were effective. According to their findings, a minimum overlap of 70% was recommended for grid patterns, while elliptical patterns should have a higher overlap percentage of 90 to 95%. Regarding effective altitudes for 3D modelling, heights of 18, 22, and 27 m, or twice the height of the building and twice the size of the building footprint, were recommended. Table 1 details flight path parameters applied in UAV-based building energy assessment research. Table 1. Flight path parameters applied in UAV-based building energy assessment Equipment

Height

Overlap

Flight pattern

Photo numbers

Ref.

Intel Falcon 8+

15 m

n/a

Polygon

102

[3]

n/a

n/a

>60%

Elliptical

>1000 and 152 manually selected

[7]

DJI Phantom 4 Pro

70 m

n/a

n/a

89 + laser scanner

[8]

DJI Inspire 1

n/a

90%

Strip

n/a

[9]

DJI Matrice 200

n/a

95%

Strip+ Polygon

n/a

[10]

DJI Mavic 2

15 m

90%

Triple grid

246

[11]

The case buildings conform to Chinese construction standards with a height of around 20 m. Initially set at 40 m, the flight height was adjusted to 60 m because of a neighbouring 13-floor community with an average height of 40 m. A 70% overlap grid pattern was employed to ensure comprehensive data acquisition. The DJI Pilot app facilitated the flight. While alternative software like Litchi and Pix4D Capture exist for flight planning, DJI Pilot was chosen due to its seamless integration of flight planning capabilities and insurance coverage provided by DJI. UAVs show remarkable efficiency in capturing close-range images, especially on rooftops, where traditional surveys could pose safety concerns. The field survey was conducted on 13th July 2023, around 4 p.m., on a cloudy and windless day, ensuring optimal weather conditions for UAV operations. The entire survey was operated within 1 h. 1,011 photos were acquired, covering the northern part of the Sanxiang Xincun community, comprising 30 buildings. The UAV operation involved two AOPA-certified drone pilots who ensured adherence to legal and safety guidelines.

38

M. Jin and M. Cimillo

2.2 Post-flight Data Processing Photogrammetric Reconstruction Various commercially available software programs, including Pix4D, DroneDeploy, and Agisoft Metashape, facilitated the automatic oblique photogrammetric reconstruction. The mesh models or point clouds produced by these software packages can be further processed in other 3D modelling software for rendering and visualisation. However, these outputs cannot directly serve as geometric input for BEM, and additional steps are needed to transform them into a compatible format for BEM. After capturing RGB photos, Agisoft Metashape software was utilised for post-flight photogrammetric reconstruction of the case area. The software automatically detected and assigned tie points, refining the initial pose to achieve precise global camera position and rotation estimations. A 3D point cloud was computed through dense stereo matching. No physical ground control points were applied in this research since the aim was to generate a geometry model to assist building energy assessment rather than precise 3D reconstruction. The first step was to align the 1,011 photos, which took approximately 20 min. Subsequently, the software automatically generated the dense point cloud, varying processing time based on the desired accuracy level. The high-accuracy point cloud of the 30 buildings took around 4 h to complete. Before exporting the point cloud, the case area underwent manual cutting to remove noise points, such as trees, cars, and floating objects. Specifically, the No.11 building was manually isolated as the case building to validate the workflow. The resulting file was then exported in LAS format, containing 43,961,206 points for further analysis and use in building energy assessment. 2.2.1 Geometry Model Generation SketchUp was selected as the platform due to its user-friendly interface and the availability of Scan Essentials and Open Studio plug-ins. These plug-ins ensure seamless integration of the point cloud data with the geometry and energy models within a single software environment. By utilising the point cloud data as a reference, the geometry model accurately represented the physical features of the building, including external walls, roofs, and windows, facilitating the subsequent energy assessment. Other Data for BEM Non-geometric data are significant in energy assessment as they provide information about the thermal parameters of the building envelope, HVAC system, and lighting system. Although direct measurements may not always be feasible, they can be reasonably estimated based on construction regulations and relevant standards. In contexts where systematic databases were developed – a good example is TABULA[12]. To retrieve the thermal parameters of buildings in Suzhou, national design standards JGJ 134-2001 [13] and JGJ 134-2010 [14] provide valuable information with the U-value of external walls, windows and air change rate under different construction periods. Construction year information for the case building could be sourced from reliable real estate websites. Occupancy schedules could be derived from the Annual Development Research Report on Building Energy Efficiency in China 2021 Urban Residential Special Topic. Climate data could be collected from the EnergyPlus website, and these mentioned data could be integrated into an energy model manually.

UAV-Based Geometry Data Acquisition

39

3 Results and discussion Figure 2 illustrates the detailed workflow process proposed in this study. Notably, remodelling the No.11 sample building using point cloud data within SketchUp proved highly efficient, requiring under 20 min.

Fig. 2. Detailed UAV-based geometry data acquisition workflow

This efficiency suggests the potential to remodel all 30 buildings within 10 h, and the whole workflow will be finished within 15 h. The processing time depends on computer hardware, while enhanced computational power will shorten the task duration. Such speed is advantageous in large-scale projects, while researchers [15] estimated that the traditional manual workflow would take 120 h to complete the geometry model generation of a large industry building. Table 2 presents a comparison of building dimensions obtained from different sources. With construction drawings lacking, data was collected from Baidu Map by manually tracing building footprints in AutoCAD, manual measurements using a laser range finder on site, and the experimental workflow model. The accuracy of the proposed workflow is compared with the method using Baidu Map (currently the most viable option for large-scale modelling), while the manual measurement will be used as a baseline, being the traditional method deemed sufficiently accurate but inefficient. The comparison underscores the higher accuracy of the proposed method; for example, the measurement error for the building volume is reduced to 5.2% from 48.7%. This is one of the essential parameters, as it affects other non-geometrical aspects of the simulation, such as the ventilation rate. The experimental workflow could also retrieve the WWR easily, which is unavailable from Baidu Map. In conclusion, UAV-acquired data surpasses open-source maps in accuracy and integrity in a safe and efficient way. The model geometry required for BEM simulations is acquired during the survey and recreated in the SketchUp environment in the proposed workflow. This geometry model is transferred directly to an energy model within the same software. While specific simulation properties can be manually entered, the geometry data serve as the input for further steps, significantly reducing the time required for data preparation. BEM can be categorised into multi-zone or single-zone simulations. Each building can be treated as a single zone for large-scale building energy assessments. Since UAVs can only retrieve the external envelope, further adjustments to the thermal zones based on indoor layouts are necessary for specific assessments of individual flats. Thermal zones can be estimated based on window locations, considering information from typological databases or real estate websites.

40

M. Jin and M. Cimillo Table 2. Comparison between building dimensions obtained from various sources

Feature

Baseline: Manual measurement

Baidu Map (Error)

Experimental Workflow (Error)

Length of East/West wall

10.16 m

6.18 m (39.2%)

10.46 m (2.95%)

Length of North wall

60.19 m

50.41 m (19.4%)

61.59 m (2.3%)

Length of the main entrance

3.7 m

3.0 m (23%)

3.90 m (5%)

Building Floor Area

3735.78 m2

1930.36 m2 (48.3%)

3859.8 m2 (3.3%)

Building Height

17.5 m

17.4 m (0.6%)

17.8 m (1.7%)

Building Volume

10896.03 m3

5592.37 m3 (48.7%)

11464.25 m3 (5.2%)

Facade Surface Area

2759.19 m2

2173.49 m2 (21.2%)

3086.98 m2 (11.9%)

4 Conclusion and Limitations The proposed workflow in this research covers pre-flight considerations, in-flight data acquisition, and post-flight processing. It aims to serve as a comprehensive guideline for future energy assessment modellers, ensuring a systematic approach to the entire process, from equipment preparation to flight path planning, including considerations of height and pattern, to in-flight operations, to the photogrammetric processing and finally to the geometry model generation. The proposed workflow covers every step to ensure accurate and efficient data acquisition. The post-flight processing, utilising Agisoft Metashape, undergoes several automated stages, leading to the reconstruction of a high-quality dense point cloud and photo-realistic 3D model of the case building. Compared to previous studies, more attention was paid to the geometry model generation. Furthermore, different from the existing research, which utilised BIM software Recap and Revit [11], the manual generation of the geometry model in SketchUp facilitates its smooth transfer for further energy assessment. This aspect serves as the study’s first objective, validating the feasibility and effectiveness of the proposed workflow for future energy modellers. The second objective is to validate the 3D geometry model output based on point clouds against actual measurements and open access sources, ensuring the generated geometry’s integrity, accuracy, and reliability. In conclusion, the research provides a comprehensive analysis and discussion of the geometric model and other inputs utilised in BEM and Urban Building Energy Modelling, thereby avoiding time-consuming and costly processes. However, certain limitations are acknowledged. UAV technology faces challenges, including battery life, flight regulations, licensing, and privacy concerns. Photo processing efficiency depends on computer hardware, potentially demanding higher computation power for larger projects. Furthermore, as reality capture and machine learning evolve, the transfer of point clouds to geometry models may become more automated.

UAV-Based Geometry Data Acquisition

41

References 1. IEA: Buildings (2022) [cited 18 Mar 2023]. Available from: https://www.iea.org/reports/bui ldings 2. Schönberger, J., Frahm, J.-M.: Structure-from-Motion Revisited (2016) 3. Benz, A., et al.: Framework for a UAS-based assessment of energy performance of buildings. Energy Build. 250, 111266 (2021) 4. Rakha, T., Gorodetsky, A.: Review of Unmanned Aerial System (UAS) applications in the built environment: towards automated building inspection procedures using drones. Autom. Constr. 93, 252–264 (2018) 5. Rakha, T., et al.: Heat mapping drones: an autonomous computer-vision-based procedure for building envelope inspection using unmanned aerial systems (UAS). Technol. Architect. Des. 2(1), 30–44 (2018) 6. Pepe, M., Alfio, V.S., Costantino, D.: UAV platforms and the SfM-MVS approach in the 3D surveys and modelling: a review in the cultural heritage field. Appl. Sci. 12(24), 12886 (2022) 7. Groesdonk, P., et al.: Remote Sensing For Building Energy Simulation Input – A Field Trial (2019) 8. Etxepare, L., et al.: Advanced intervention protocol in the energy rehabilitation of heritage buildings: a Miñones Barracks case study. Sustainability 12(15), 6270 (2020) 9. Bayomi, N., et al.: Building envelope modeling calibration using aerial thermography. Energy Build. 233, 110648 (2021) 10. Rakha, T., et al.: Building envelope anomaly characterisation and simulation using drone time-lapse thermography. Energy Build. 259, 111754 (2022) 11. Delgado, J.M.P.Q., et al.: BIM and BEM interoperability-evaluation of a case study in modular wooden housing. Energies 16(4), 1579 (2023) 12. Ballarini, I., Corgnati, S.P., Corrado, V.: Use of reference buildings to assess the energy saving potentials of the residential building stock: the experience of TABULA project. Energy Pol. 68, 273–284 (2014) 13. China Academy of Building Research, C.U.: Design Standard for Energy Efficiency of Residential Buildings in Hot Summer and Cold Winter Zone JGJ 134-2001. China Architecture & Building Press (2001) 14. China Academy of Building Research: Design standard for energy efficiency of residential buildings in hot summer and cold winter zone JGJ 134-2010. China Architecture & Building Press (2010) 15. Garwood, T.: Closing the Performance Gap in Building Energy Modelling through Digital Survey methods and Automated Reconstruction. Department of Mechanical Engineering. The University of Sheffield (2019)

Cellular Automata as Design Tools for Artificial Ecologies Yiming Liu(B)

and Christiane M. Herr

Southern University of Science and Technology, Shenzhen, China [email protected], [email protected]

Abstract. The built environment functions as a facilitator of a healthy relationship between humans and the natural environment. The ability of cellular automata (CA) to model naturally occurring phenomena that express in spatio-temporal patterns offers opportunities to both simulate and generate complex spatial arrangements based on local morphological rules. This local relationship-driven property resonates with the local generative dynamics found in natural systems. To address the intricacies associated with cellular automata rule definitions within the context of ecological urban form, we construct a “multi-layered” CA as a theoretical model for exploring ecological performance-oriented architectural form design. The resulting model accommodates both conventional top-down design strategies and the locally driven relational networks of ecological considerations. An analysis of two case studies not only examines the possibilities of both simulating and generating ecologically oriented, integrated architectural and urban form using CA-based strategies at different stages of the design process, and offers designers a new way to model and evaluate ecologically sensitive constructed environments. Keywords: Cellular Automata · Cellular Automata Rules · Ecological Design · Algorithmic Design

1 Introduction Cellular automata (CA) are discrete models in which spatially determined state changes over time generate dynamic patterns. CA systems are typically simple in their setup: In each iteration of the CA system, predefined transition rules (TR) embedded in each individual cell determine the change of local cell states based on the states of their neighbouring cells. CA and their dynamics were first discussed as “mathematical games” in the 1940s by Stanislav Ulam [1] to study crystal growth, and later by John von Neumann [2] to study robotic self-reproduction [3]. Due to limited computing capacity and power at the time, it was not until 1980 that Wolfram conducted a systematic study of onedimensional CA [4]. In this process, Wolfram defined key CA terminology describing key CA elements, including CA grid, states, neighbourhoods as well as characteristics of the processes by which basic rules lead to complex results. Conway’s Game of Life model forms one of the most well-known two-dimensional CA models [5]. The Game of Life’s intriguing dynamics illustrate how changes in sets of simple transition rules and © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 42–49, 2024. https://doi.org/10.1007/978-981-97-0621-1_6

Cellular Automata as Design Tools for Artificial Ecologies

43

initial states can generate intricate dynamic patterns. It further illustrates the difficulty to predict CA dynamics based on the initially determined transition rules, and invites an explorative and experimental approach to CA-based research. Due to these properties, the Game of Life has also inspired a broad variety of speculative implementations for design purposes. Over the past decades, the possibilities of modelling and simulating with CA have led to their application in various natural, physical and chemical phenomena. CA-based methods are also widely used to analyse spatial structure of regions or cities to evaluate and predict urban development. Yeh and Li [6] demonstrated how CA models that implement contextual relationships of local neighbourhoods in combination with several constraint rules can also be used to generate new urban forms. The aim of this paper is to present a framework of multi-layered CA model to generate an ecological performance-oriented architectural form design. We argue that the complexity of CA rule definitions can be addressed by multi-layered CA. This method examines the interaction of two types of rules, which are top-down rules based on global constraints and bottom-up rules based local interactive processes. Moreover, we elaborate that an urban ecological framework could be implemented by applying multi-layered CA to integrate the modelling of urban man-made landscapes and built forms.

2 A Brief Literature Review CA models can be implemented in any discrete and homogeneous assembly of cells. Already early on, Schrandt and Ulam [1] developed three-dimensional CA and corresponding transition rules in their original work, and presented compelling sculptural results. More generally, three-dimensional CA outcomes can be generated in several ways, typically including the following three ways: First, CA transition rules can be executed simultaneously by cells distributed homogeneously in three-dimensional space to generate changing spatial configurations with each iteration. Second, three-dimensional shapes can be generated as the spatial traces of a progressing two-dimensional CA growth process. The third approach to generating geometry through CA is based on layered transition rule sets applied in sequence to determine the form change of the three-dimensional CA system, which allows adjustments according to different architectural purposes in the early stages of architectural design [7]. Overall, CA offer an innovative and understudied approach to study architectural generative process. 2.1 CA-Based Architectural Design Strategies As a generative design approach, CA can support design through establishing strategic rules while allowing the fast generation of design options. In architecture, CA form a classic and frequently reinvented algorithmic design paradigm supporting a range of applications ranging from abstract to concrete [8]. There have been some previous studies on spatial generation based on Conway’s Game of Life. For example, Coates et al. [9] introduced new cell states, the “dead” or “alive” of cells in the game of life can be interpreted as the “void” and “full” spatial units, or the volumetric “cube” cells and

44

Y. Liu and C. M. Herr

surface-based “plate” cells in design proposal. Krawczyk [10] adjusted the size and shape of cell units, such that CA could also adapt to the site environment through customized rules [11]. Cell properties were further adapted by Khalili Araghi and Stouffs [12], who used cell states to represent natural light and accessibility and defined cellular automata as decentralized space expansion systems consisting of a large number of simple units with local connectivity, with the state of units depending on the state of their neighbours. Despite continuing interest, however, their radically different operational logic and generative capacities have so far limited CA applications in architectural design beyond ambitious Master projects and experimental form generation. Reasons for these shortcomings are found in two characteristic properties of classic CA systems: their locally and contextually driven generative logic and their typical implementation as homogeneous cellular systems devoid of context models. The first characteristic is the primary reason for the lasting attraction of CA generated spatio-temporal patterns in the eyes of designers, who are typically fascinated by the intricacy and unpredictability of CA. The same characteristic of unpredictable, locally determined patterns also renders CA very difficult to use for directed, goal-oriented design beyond simple form or dynamics investigations. This is compounded by the limited capacity of CA to translate design goals which may depend on multiple variables at the same time into the characteristic simplicity of CA states and rule sets. As a result, CA experiments are rarely found in applied design beyond the scope of Master projects, explorative research or experimental workshops. The second characteristic refers to the difficulty of adapting CA into models that respond to context in ways that could offer divergence from homogeneous CA models that are used in science-oriented simulation applications [11]. 2.2 CA Models in Ecological Planning and Design While the locally-driven dynamics of CA models are not easily coordinated with typical top-down goal-oriented planning and design paradigms, they resonate much better with the localized generative dynamics that can be found in natural systems, where local relationships drive expressions of environmental properties within the parameters set by broader climatic, environmental and ecological systems. This property has been exploited previously in modelling and simulation approaches found in the sciences, including biology and ecology. For example, Colasanti and Grime [13] explored a CA model based on CSR theory, which is used to simulate the dynamic spatial pattern developed by local expansion and succession among plant individuals or populations. Chen [14] presented a CA model by adjusting the cell properties to stimulate the competitive growth relationship between predator and prey in the food chain. Accordingly, a rethinking of planning paradigms based on CA dynamics offers an opportunity to translate spatio-temporal processes found in natural contexts into the context of the hybrid space of artificially created natural contexts. This particularly applies to urban and architectural form planned with ecological goals, which needs to accommodate both conventional top-down design strategies and the locally driven relational networks of ecological considerations. This new context is rapidly gaining in relevance as cities are seeking to create environments beyond human-centric concerns [15], as expressed in many recent post-anthropocentric “urban rewilding” initiatives, biophilic design and urban biodiversity initiatives.

Cellular Automata as Design Tools for Artificial Ecologies

45

This paper maps opportunities and challenges for designing environments with both natural and artificial characteristics using CA based methods. To this end, it examines new types of layered CA models employed as part of recent architectural and urban design strategies. Based on the analysis of previous work in the field in the preceding sections, the following sections examine limitations and advantages of CA models in the context of architectural and urban design applications. This survey and analysis of CA behavioural types provides insights into how generative CA models in design have adapted CA models developed for science-based simulation models. The analysis characterizes the suitability of different types of CA models to particular planning and design tasks, and contributes an aspect of CA research that is missing from the field so far, contributing towards a broader and systematic study of CA for design purposes.

3 CA-Based Design of Artificial Ecologies: Two Case Studies Humanity is witnessing a fundamental shift in the composition of the environment, where urban landscapes blend human influence with cultivated natural elements. Cities aim to create environments in which natural elements are reintroduced not as superficial decorative additions but as a co-determinant of urban form. The gradual realization that designed urban form can accommodate and be shaped by natural ecosystems, gives rise to the concept of “artificial ecology”. Artificial ecologies support and emulate the symbiotic interactions observed in natural systems, where a balance is achieved through the fusion of goal-directed planning mechanisms and locally driven self-regulating developmental processes. CA models offer flexibility for simulating complex interactions between different design elements, providing insights into emerging patterns and behaviours in evolving systems. In the following, we discuss two characteristic examples illustrating the multi-layered application of CA rules in applied design processes. By adjusting CA rules at different layers, designers can generate, simulate and control various complex phenomena that are locally driven. 3.1 A Layered CA-Based Urban Ecological Growth Model The first case study is a proposal for a new approach to introduce green spaces into urban concentrations developed by Hu and Song at the University College London, as part of their Master of Urban Design studies [16]. CA were introduced into this project because dynamic CA rules can model to the dispersal rules of plant populations based on competition for resources among individual plants and neighbouring plants on temporal and spatial scales. Hu and Song took their project as an opportunity to design a system that encompasses a nature-led approach to facilitate the integration of biodiversity within urban environments and create buildings that closely integrate with nature. This study simulated the process of plant growth through CA rules derived from plant growth strategies, such as resource capture and utilization (Fig. 1 top left). At the next step, in order to generate plant allotment, the configuration originally generated by the CA system is transformed manually into an innovative type of multi-layered CA with distribution rules for different plant types. Each layer of CA is assigned specific rules, cell sizes, and cell states, representing a particular plant type (Fig. 1 bottom left).

46

Y. Liu and C. M. Herr

This approach allows for the simulation of complex interactions between different plant species and their environments. By utilizing selective results from the previous steps, Hu and Song created simulations closely matching the characteristics and dynamics of plant colonization within buildings. Additionally, the analysis of building environments, such as solar conditions, nutritional resources and spatial availability, were used before the three-dimensional generation process. Hu and Song describe the resulting CA-based design process as both open and goal-oriented, since each layer of the CA model is based on input data selected by the designer while CA rules are dynamic and specified based on simulated plant behaviours.

Fig. 1. The forest erosion project developed by Hu and Song

3.2 A Layered CA-Based Space Planning System for La Defense The second case study is the “Carcassonne system”, an experimental application of CA developed by Liu et al. [17] during their Master of Architecture studies at University College London. This project proposes a CA-based approach forming self-organizing patterns with the aim of creating functional floor arrangements to explore interactions between different ruleset layers with limited manual intervention. By testing the potential of density, accessibility, and functional clusters in the building environments, and simulating connectivity rules between card boundaries with similar properties in the card game of Carcassonne, they also developed an adaptive multi-layered CA tool. This tool is mainly divided into two types of rules, local rules and global rules. For the local rules part, the observation of the functional organization characteristics of general office plans led Liu et al. to change conventional CA rules by creating proprietary cell states. In the design process, the connection between functions is a way to determine suitable neighbourhood clusters. If there are no cells that can be connected to the existing assemblies, these cells are interpreted as “empty” cells (Fig. 2 left). If no constraints are applied to CA-based generative dynamics, the process of generating plans would expand infinitely. In addition, this approach based on an algorithmic

Cellular Automata as Design Tools for Artificial Ecologies

47

generative approach can produce a large number of potential solutions that meet specific constraints and requirements. Therefore, some limiting factors needed to be introduced as global rules to ensure that the generated plan can meet the original design goals. Liu et al. further used constraints in the form of starting point, traffic system, boundary and evaluation process, with the designer then systematically adjusting the space function to adapt it to the surrounding environment to build more significant and desired space forms (Fig. 2 right), yet this method also has some limitations on representing and simulating phenomena that involve interactions or dependencies over large spatial or temporal scales. In the next step, other agent-based algorithms were used in the twodimensional space generated by the CA-based method to generate three-dimensional building structures, where the building volume is generated based on the basic configuration of functional modules, open spaces, and other forms of public space. In the process of generating the building frame, Liu et al. adjusted the height, direction and surrounding environment of the whole building form by controlling ecological variables such as sunlight, temperature, moisture, and exposure. The three-dimensional space generated by successive CA-based steps seeks to accommodate the space requirements of ecological buildings.

Fig. 2. An experimental adaptation of the Carcassonne game using the CA rules by Liu et al.

4 Discussion: Layered CA for Artificial Ecological Design In classical CA systems, each cell follows constant transition rules based on the states of neighbouring cells, which typically produces intricate and unpredictable results. However, this CA approach lacks the key ability to respond to context dynamically, which renders them less suitable for certain applications in architectural and urban design. The case study analysis shows that the layered CA-based generative system allows each cell to follow more flexible and locally responsive transition rules. By integrating context

48

Y. Liu and C. M. Herr

awareness, this type of CA model can address complex architectural challenges and generate design solutions that are more aligned with specific requirements and constraints. Addressing the relationship between local interactions and global system behaviour, CA is used in stages with layers of alternating stages of human intervention and adjustment. This circular feedback process retains CA dynamics yet allows for a relatively high degree of design control. These new, layer-based CA models, allow for some speculation on new possibilities of CA application in the generation and simulation of ecological networks to obtain new types of ecological urban and architectural form. This new approach to CA-based design translates CA characteristics for a planning context that accommodates not only top-down control but also localized pattern formations and connectivity networks. In this paper, we present this initial “multi-layered” CA model that integrates bottom-up localized generative logic with overarching planning goals. The theoretical model is complemented with an in-progress report of first prototypes of layered CA development and an initial review of the potential of this design strategy in the context of ecological design. The customized CA systems we introduced here adopt a generative design strategy, and have, beyond this paper, only been described by Herr and Li [18] previously in the creation of microhabitats supporting microbial diversity in bio-receptive façade elements. Herr and Li employed a layered generative CA system in order to adapt building surfaces to environmental requirements for microbial colonization, with a focus on the relationship between microbial communities and design factors creating favourable building surface conditions. Herr and Li further proposed differentiated local and global rules alongside dedicated design control strategies, to generate desired physical shapes in analogy to natural microbial growth patterns.

5 Summary Beyond the exploration of possibilities at the initial architectural and urban design stage, CA also have the potential to optimize and adjust design process in other ways. By introducing various types of circular feedback loops, designers can adjust CA rules and parameters to accommodate goal-oriented design strategies. Based on an analysis of previous published work and the two case studies presented in this paper, we introduce a multi-layered CA as a method to generate ecologically oriented, integrated architectural and urban form, extending previous research in the field. The current state of multi-layered CA for design purposes is still at a preliminary stage, with the interaction between global rules and locally-generated expressions still requiring a broader theoretical framework for multi-layered CA capable of connecting buildings and surroundings in a more generalizable and transferable way.

References 1. Schrandt, R.G., Ulam, S.M.: On Recursively Defined Geometrical Objects and Patterns of Growth. Cellular Automata, pp. 232–243. University of Illinois Press, Urbana, IL (1967) 2. von Neumann, J.: Theory of Self-Reproducing Automata. University of Illinois Press, Urbana, IL (1966)

Cellular Automata as Design Tools for Artificial Ecologies

49

3. Jeffress, L.A.: Cerebral Mechanisms in Behavior: The Hixon Symposium. John Wiley & Sons (1951) 4. Wolfram, S.: Statistical mechanics of cellular automata. Rev. Modern Phys. 55(3), 601–644 (1983) 5. Gardner, M.: Mathematical games – the fantastic combinations of John Conway’s new solitaire game ‘life.’ Sci. Am. 223(4), 120–123 (1970) 6. Yeh, A.G., Li, X.: Urban simulation using neural networks and cellular automata for land use planning. In: Richardson, D.E., van Oosterom, P. (eds) Advances in Spatial Data Handling. Springer, Berlin, Heidelberg (2002). https://doi.org/10.1007/978-3-642-56094-1_33 7. Herr, C.M., Karakiewicz, J.: Algogram: automated diagrams for an architectural design studio. In: Dong, A., Moere, A.V., Gero, J.S. (eds.) Computer-Aided Architectural Design Futures (CAADFutures) 2007, pp. 167–180. Springer, Dordrecht (2007). https://doi.org/10.1007/9781-4020-6528-6_13 8. Herr, C.M., Fischer, T.: Systems for showing and repurposing: a second-order cybernetic reflection on some cellular automata projects. J. Math. Syst. Sci. 3(2013), 201–216 (2013) 9. Coates, P., Healy, N., Lamb, C., Voon, W.L.: The use of Cellular Automata to explore bottom up architectonic rules. In: Eurographics UK Chapter 14th Annual Conference, 26–28 Mar 1996. Eurographics Association UK, London (1996) 10. Krawczyk, R.J.: Architectural Interpretation of Cellular Automata. In: Soddu, C. (eds.) The 5th International Conference on Generative Art 2002, pp. 7.1–7.8, Generative Design Lab, DiAP, Politecnico di Milano University, Milano (2002) 11. Herr, C.M., Ford, R.C.: Cellular automata in architectural design: from generic systems to specific design tools. Autom. Constr. 72, 39–45 (2016). https://doi.org/10.1016/j.autcon.2016. 07.005 12. Khalili Araghi, S., Stouffs, R.: Exploring cellular automata for high density residential building form generation. Autom. Constr. 49, 152–162 (2015). https://doi.org/10.1016/j.autcon. 2014.10.007 13. Colasanti, R.L., Grime, J.P.: Resource dynamics and vegetation processes: a deterministic model using two-dimensional cellular automata. Funct. Ecol. 7, 169 (1993) 14. Chen, Q.: Cellular automata. In: Jørgensen, S.E., Chon, T.-S., Recknagel, F. (eds.), WIT Transactions on State of the Art in Science and Engineering. pp. 283–306. WIT Press (2009) 15. Samuelsson, K.: The topodiverse city: urban form for subjective well-being. Front. Built Environ. 7 (2021) 16. Hu, Y.Y., Song, Y.Q.: Forest Invasion Plan: La Defense in 2125. Master of Architecture Urban Design Thesis Project. UCL Bartlett School of Architecture. https://bpro2022.bartlettarchucl. com/rc11-hidden-dimensions/forest-invasion-plan. Last accessed 10 Aug 2023 17. Liu, Y.M., Liu, Z.C., Zheng, Y.H., Xie, Y.C.: Threading city. Master of Architecture Urban Design Thesis Project. UCL Bartlett School of Architecture. https://bpro2021.bartlettarchucl. com/rc11-ai-and-the-case-of-la-defense/threading-city. Last accessed 10 Aug 2023 18. Herr, C.M., Li, C.: Articulating facade microbiomes at human scale: a cellular automata driven bioreceptive facade design approach to communicate a new dimension of urban health. HUMAN-CENTRIC. In: Proceedings of the 28th International Conference on ComputerAided Architectural Design Research in Asia (CAADRIA) 2023, vol. 1, pp. 281–290, Ahmedabad, India (2023). https://doi.org/10.52842/conf.caadria.2023.1.281

Deployable Origami Wall with Patterned Knit Panels Virginia Ellyn Melnyk1,2(B) 1 Tongji University, Shanghai, China

[email protected] 2 Virginia Polytechnical University, Blacksburg, VA, USA

Abstract. This paper explores the process of creating a unique triangulated origami folding wall, highlighting its relevance to design as research and research through design. By integrating hand-making techniques with digital methodologies, the project challenges traditional design methods in architecture. The design process involved a process of hand to digital and digital-to-hand for the different elements of the project. Using Rhino3D and Grasshopper to simulate various pattern designs, which were then ultimately fabricated by hand. Meanwhile, paper origami hand models inspired the overall frame structure, which was later modeled, in Rhino3D and Grasshopper to simulate the folding process. While designs were simulated both digitally, the use of digital tools is used during different phases of the design, emphasizing the potential of parametric thinking alongside manual creation. Hand-making techniques, including domestic knitting machines and handmanipulated stitches, were used to craft detailed and customized panels. Varying pattern densities created a dynamic interplay of light and shadow. The design for the pattern is based on a binary pattern of stitches and floats, harkening back to early coding and jacquard looms and punch cards. These knitted panels are attached to the folding frame design to create a wall system that is spatially dividing but also plays with porosity. The design allows some light and shadow play as well as varying density and depth between the layers. The project demonstrates the potential of blending computational design with manual development in architectural design. While working through this research on the relationships between physical and digital production as a means for design artifacts. Keywords: Digital Design · Physical Craft · Textiles · Knitting · Parametric Design · Lightweight Structures

1 Introduction This paper explores the process of designing and creating a triangulated origami folding wall, serving as a case study for design research and research through design. It explores the fusion of hand-making techniques and digital design methodologies, reflecting on traditional design methods within architectural practice and the design process. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 50–60, 2024. https://doi.org/10.1007/978-981-97-0621-1_7

Deployable Origami Wall with Patterned Knit Panels

51

In this case study, the design process involved utilizing Rhino3D and Grasshopper software to simulate and iterate various design patterns, enabling precision and adaptability compared to conventional approaches. By employing parametric design principles, a system of triangle frames was developed, allowing for seamless adjustments and customization to meet specific spatial requirements. This flexible design approach ensures the wall’s versatility, making it suitable for diverse applications, including temporary or mobile structures. While the initial designs for the textile patterns were digitally simulated, the project’s conceptual foundation emerged from manual exploration. Rather than seeking a direct replication of computer-generated patterns, hand-developed design patterns served as conceptual frameworks for stitch and pattern development. This intentional integration of manual craftsmanship and parametric thinking represents the potential of harmonizing digital and manual processes. The hand-making techniques employed in crafting the knitted panels exemplify a progressive intersection of traditional craftsmanship and digital innovation. Drawing inspiration from early coding practices such as Jacquard loom punch cards and binary programming, the project utilizes a domestic knitting machine and hand-manipulated stitches to create panels infused with intricate details and personalized touches. The overall wall form is inspired by the principles of origami, the folding mechanism of the wall representing a departure from conventional design methods, reimagined within a digital context. Meticulous consideration was given to ensuring compatibility between the folding frames and hand-knitted panels, enabling integration between the rigid frame and the soft textile. This paper examines the shift in reconsidering the digital within architectural design by merging computational precision and adaptability with the artistic depth of handmaking techniques, the project underscores the potential of blending traditional craftsmanship with digital advancements. It emphasizes the significance of manual development in design processes while displaying the transformative capacity of computational design methods in architectural innovation. Ultimately, it explores the intriguing interplay between digital craft, pattern design as binary coding, and the reciprocal inspiration between computers and artisans.

2 Historical Context To frame this project within a historical context, it is important to highlight the significance of textiles in architecture, the value of craft and making, and the role of digital craft and design. These aspects draw upon renowned scholars and their works in the field. 2.1 Textiles in Architecture Gottfried Semper’s writings emphasize the cultural and structural importance of textiles in shaping architectural forms [1]. He recognizes textiles as a fundamental element in architectural design, highlighting their ability to define and partition spaces, as well as their role in providing structural support and ornamentation.

52

V. E. Melnyk

Traditional architectural examples demonstrate the incorporation of textiles into architecture, such as the nomadic structures of Central Asia, yurts or tents used by the Mongolians. These portable dwellings utilize textiles as flexible and lightweight materials, enabling easy transportation and assembly. In these examples, textiles play a vital role in providing shelter, insulation, and visual aesthetics, highlighting the adaptability and versatility of textile architecture [2]. Another example is the works of Frei Otto, who is known for his exploration of lightweight and deployable structures. Otto’s innovative designs, including the German Pavilion at Expo ’67 in Montreal, Canada, displayed the potential of textiles in architectural applications. The pavilion featured a tensile membrane roof structure made of lightweight fabrics stretched across a steel frame, allowing for large spans and creating a visually striking canopy [3]. The importance of lightweight and deployable architecture is further demonstrated by the work of architects like Shigeru Ban and his use of paper tubes and fabric membranes in disaster relief structures [4]. By exploring these examples and the theoretical underpinnings provided by Semper, it becomes evident that textiles have played a crucial role in the history of architecture. They continue to inspire contemporary architects to create lightweight, deployable structures that respond to changing needs and environmental conditions. 2.2 Craft and Textiles The role of craft and making in architecture and textiles is largely significant, particularly when considering the unique contributions of feminist craft movements. Richard Sennett’s book, “The Craftsman”, delves into the importance of craftsmanship and the tangible skills acquired through hands-on engagement [5]. Sennett emphasizes how craftsmanship shapes architectural traditions and emphasizes the significance of skilled labor in the design and production process. Craft, as exemplified by practices such as knitting, not only preserves traditional techniques but also fosters a sense of community and empowerment through making [5]. Further exploration of craft and making is highlighted by the work of Juhani Pallasmaa in “The Thinking Hand” [6]. Pallasmaa emphasizes the sensory and experiential qualities of craftsmanship in design. This approach emphasizes the connection between the hand, the mind, and the creative process, emphasizing the significance of tactile engagement in architectural practice [6]. Within the realm of textiles, knitting stands out as a particularly significant craft form. Knitting has a rich history of being associated with both functional and artistic expressions. It has been recognized for its capacity to create intricate patterns, textures, and forms. Recently, knitting has also become a symbol of feminist craft-making, empowering individuals through creativity, self-expression, and cultural resistance [7]. Feminist craft movements, including the influential subversive knitting movement, have challenged traditional gendered roles within craft and textile practices. They have fostered a renewed appreciation for the value of handwork, reclaiming domestic craft as a site of creativity and cultural commentary. These movements highlight the social and political dimensions of craft, promoting inclusivity, diversity, and the celebration of women’s artistic contributions [8].

Deployable Origami Wall with Patterned Knit Panels

53

By acknowledging the importance of craft, textiles, and knitting in architectural practice, we embrace a holistic approach that values the experiential, tactile qualities of making. Integrating feminist perspectives on craft expands our understanding of the cultural, social, and historical implications of textile arts in architecture, enriching the dialogue surrounding craft making, gender, and the built environment. 2.3 Digital Craft Walter Benjamin’s essay “The Work of Art in the Age of Mechanical Reproduction” explores the profound changes brought about by technological advancements, specifically the reproducibility of artworks through mechanical means [9]. While Benjamin’s focus is primarily on visual arts, his ideas resonate with the concept of craft in relation to tools. Benjamin’s observations on the loss of aura and authenticity in mechanically reproduced artworks raise pertinent questions about the role of digital craft. Regarding digital craft, Malcolm McCullough’s “Abstracting Craft: The Practiced Digital Hand” explores the fusion of traditional craftsmanship and digital methodologies [10]. It reflects on the transformative potential of digital tools and their impact on design practices. McCullough’s work highlights how digital craft can extend the capabilities of the human hand and expand the possibilities of design and fabrication. Mario Carpo’s theories on algorithmic design are also pertinent to this discussion. His book “The Digital Turn in Architecture” explores the historical connection between coding and early textile designs [11]. Carpo examines the influence of binary patterns in Jacquard looms and their relationship to architectural design. His insights offer valuable perspectives on the integration of digital tools, algorithms, and pattern designs in architectural practice. Additionally, “Form + Code in Design, Art, and Architecture: Introductory Book for Digital Design and Media Arts” by Barendse, Reas, and McWilliams provides further exploration of the relationship between form, code, and design. The transformative potential of coding in generating intricate patterns, generative art, and architectural forms have become more possible due to digital tools [12]. It offers examples and case studies that demonstrate the fusion of form and code in the realm of digital design and media arts.

3 Design Process The overall design process is broken into two main parts. The design for the origami wall, which is constructed ridged frame. In addition, the design for the textile panels to infill the wall frames. Each of these two parts takes on a different approach based on their design specificity and how they were to be fabricated. 3.1 Origami: Physical and Digital Design Process The concept of a folding wall stems from the exploration of architectural and interior spaces that require adaptability and change. Taking inspiration from lightweight transportable architecture, the design entails utilizing a frame structure wrapped with textiles

54

V. E. Melnyk

to create enclosure. The resulting wall is a lightweight, movable element that can divide space or be easily folded up and removed. Rather than aiming for complete solidity in the materials, the design embraces the idea of providing transparency and lightness. This portion of the design utilized a process that starts with physical conception, progresses to digital models and simulations, and culminates in the final physical construction. The design embodies a seamless integration of craft, making, and computational design methodologies. This iterative approach allows for the exploration of form, structure, and functionality. Meanwhile exploring the relationship of maker and designer with digital tools and digital craft.

Fig. 1. Samples of Origami Models. Credit Author.

Paper Folding Design. The initial inspiration for the folding wall design draws from the art of origami and its significance in architecture. Origami, with its focus on geometric folds and intricate patterns, has long been admired for its ability to transform flat sheets into three-dimensional structures. In architectural practice, the principles of origami have been applied to create dynamic and adaptable forms, enabling efficient use of materials and space [13]. Building upon this legacy, the folding design incorporates angular folding and origami patterns to enhance its aesthetic appeal and functionality. Introducing these design elements can achieve a sense of depth and visual interest, engaging the viewer with its geometric intricacies [14]. The utilization of angular folding techniques goes beyond the typical straight-line zigzag folds found in conventional folding walls, allowing for a more captivating and unique design outcome. The first study models for the wall were made out of paper and tested these various folding designs and ideas. By looking at them and making them by hand the designer can build a relationship with the forms and understand how the panels will move to open and close. Another important feature was how it would stand and balance as a successful wall. See Fig. 1. Digital Folding Design. In addition to the physical construction process, the folding design was simulated in Grasshopper to visualize the movement and unfolding of the design. This digital simulation aided in refining the folding mechanism and ensuring smooth operation when deploying and folding the wall structure. See Fig. 2.

Deployable Origami Wall with Patterned Knit Panels

55

Fig. 2. Grasshopper Model of Origami Folding Design. Credit Author.

To further develop the design into a design with a material thickness that could be fabricated, a Rhino3D model was created at a proposed human scale. Scaling the design to wall size necessitated considering several constraints. Firstly, standard wood dimensions available at local home improvement stores were taken into account. Secondly, the size of the vehicle required for transporting the wall to the exhibition space was considered. Lastly, the size requirements of the allotted space at the Frascari Symposium VI: Finishing: The Ends of Architecture, held from March 31 to April 1 at the Virginia Tech Washington-Alexandria Architectural Center, where the wall was to be exhibited. The Rhino3D model facilitated the creation of a schematic plan for the frame design. The frame consists of ten triangles connected with canvas fabric hinges. Construction of the frame occurred in the woodshop at Virginia Tech using standard woodworking tools readily available.

3.2 Knitting: Digital and Hand Craft Processes The knitting process involved in this project uses a knitting machine, which automated the creation of the knitted fabrics. The machine consists of a bed of latch-hook needles that work together to form rows of stitches in the fabric. Various stitch types can be produced, including the knit stitch, float, and tuck stitch [15]. Stitches in knitting refer to the basic units that make up the fabric, formed by wrapping the yarn around the needle and pulling it through a loop to create a new loop. This repetitive process constructs the fabric, while floats occur when stitches are skipped or not worked, resulting in longer strands of yarn that span across the fabric [15]. In this phase of the design process, the integration of digital to physical making emphasizes the transformative potential of bridging the virtual into tangible realms. By embracing the abilities offered by digital tools and technologies into the design process. However, it also recognizes the significance of human error and the serendipitous “glitch”

56

V. E. Melnyk

that can emerge during the translation from digital to physical. This understanding fosters a dynamic and exploratory environment that stimulates innovative design solutions, drawing inspiration from Francesca Hughes’ book, “The Architecture of Error: Matter, Measure, and the Misadventures of Precision.“ [16].

Fig. 3. Parametric Design for Knitting Pattern. Credit Author.

Parametric Pattern Design Process. The design process for the knitted panels involved Grasshopper in Rhino3D to create patterns that explore the interplay between solid and transparent areas. By establishing a set of rules that determined the relationships between stitches and floats, the design achieved a dynamic play with transparency. A grid spreading definition was employed in Grasshopper to manipulate lines to flow around attractor points, where the dense areas of lines represented stitches and the voids represented where floats would occur. This established a proportional relationship between the number of stitches and floats, resulting in a range of pattern outcomes. See Fig. 3. The design for stitches versus floats was then translated into a grid layout to be used as a knitting pattern. Swatches and Scaling. To ensure the appropriate size for the knitted panels, a rectangular swatch sample was created using the designated yarn. This sample, measuring 30 stitches by 30 rows, allowed for testing and measurement to determine the final dimensions. By stretching the swatch and measuring the resulting dimensions in tension, the average size of an individual stitch could be calculated to fit the triangular frame. Resulting in the proportions to be translated to the final digital pattern design. See Fig. 4. Physical Knitting Process. The processes of fabricating the wall translated the digitally designed pattern and used the knitting machine to manually knit and manipulate the stitches into the desired design pattern.

Deployable Origami Wall with Patterned Knit Panels

57

Fig. 4. Swatch Samples. Credit Author.

The triangular design of the panels was incorporated by using a narrowing technique, achieved by transferring the stitches at the material’s edges every few rows to reduce the number of needles in use. Starting with a width of 108 stitches, the stitches at the edges were gradually transferred to the neighboring internal needle, resulting in a reduction of width by two stitches every eight rows. This process continued until only eight stitches remained on the knitting bed to form the top point of the triangle panel. To optimize production speed, the parametric pattern design was used to inspire adjustments every eight rows, creating elongated shifts instead of changes in stitches and floats at every row. This modification aimed to facilitate the fabricator’s task of manipulating the knitting machine carriage and needle activation using a transfer tool. While the manual knitting process took several hours per panel, totaling approximately 60 h, some mistakes and dropped stitches occurred along the way. These errors, inherent to the craftsmanship involved, were embraced as part of the process, contributing to the uniqueness of each panel.

4 Final Deployable Wall Outcomes The resulting final design comprised 10 triangular Frames and 20 knitted panels, with 1 panel attached to the front and another to the back of the frame. The separation of the panels by the depth of the frame created a dynamic space where the layers of patterns could interact and shift in response to perspective and parallax. To highlight the different layers, the front, and back panels were knitted using two different yarn colors: a blue and a pink wool mix yarn of medium weight with 2 ply. To secure the knitted panels to the frame, small tack nails were used, spaced every few inches. To ensure even tensioning the installation process by first attaching the three corners, followed by the middle of the panels. Subsequently, the middle of each length was attached until the nails were spaced 2–3 inches apart. Approximately 680 nails were utilized for each panel’s attachment to the frames. The final dimensions of the wall were 2 m (6.4 ft) long by 1.8 M (6 ft) tall and .33 m (13 in) deep. When collapsed, the dimensions were reduced down to 2 m (6.4 ft) by

58

V. E. Melnyk

.48 m (19 in) by .27 m (11 in), facilitating convenient transportation and deployment. See Fig. 5. With a weight of 8 kg (17 lb), the wall proved to be lightweight and easily maneuverable, allowing a single person to carry and unfold it into the upright standing position.

Fig. 5. Final Design Deployment. Credit Author.

During the installation, it was observed that the textile dimensions estimated for the wall were slightly tight, resulting in a scalloped effect along the edges where the material was fastened to the frame. Additional nails could be added to address this issue, but it would require an ongoing process. Exploring alternative mechanisms for attaching the textile to the frame could potentially yield smoother connections, reducing the visual distraction caused by the color contrast between the frame and the knit material. This would shift the focus towards the intricate knit pattern and its overall design rather than the edge condition. The final knitted material displayed a linear design characterized by shifts in aperture and transparency. This design allowed for a balanced interplay of light and visual connection between the sides of the wall while still maintaining a sense of spatial division. The resulting texture and play of shadows further contributed to the overall aesthetic. These aspects resonate with Gottfried Semper’s ideas of different materials, colors, light, and texture in architectural design, and establish a connection between the human body, notions of enclosure, and the experience of lightness through vision and porosity see Fig. 6.

Deployable Origami Wall with Patterned Knit Panels

59

Fig. 6. Details of Porosity and Shadow Patterns. Credit Author.

5 Conclusion In conclusion, this project serves as a case study in design research and research through design, exploring the dynamic interplay between textiles, architecture, and digital design. By integrating various theoretical perspectives and practical methodologies, it offers valuable insights into the potential of textiles as a medium for architectural expression. Inspired by the ideas of Gottfried Semper and his emphasis on weaving, wrapping, and material effects, this project reflects on the historical significance of textiles in shaping architectural forms [1]. The design process involved working both digitally and physically, highlighting the iterative nature of design research. The computational design methods employed in the parametric pattern generation facilitated the translation of digital designs into the physical realm, while the handcrafting of the knitted panels brought forth the artistry of craftsmanship [5]. Through the exploration of transformable architecture and lightweight structures, this project exemplifies the potential of textiles in creating dynamic and adaptable spaces. The folding wall frame design, inspired by origami principles [14], demonstrates the seamless transition from physical to digital and back to physical, blurring the boundaries between the virtual and the tangible. The knitting pattern design, reminiscent of early coding practices, highlights the integration of digital technologies with traditional craft techniques. This project also highlights the importance of research as an integral part of the design process. By approaching the design problem with a research mindset, it becomes a catalyst for innovation and exploration. The combination of design research and research through design allows for a deeper understanding of the potential of textiles in architecture and opens up new possibilities for creative expression [16]. In summary, this project serves as a testament to the interplay of textiles, architecture, and digital design. It exemplifies the potential of research-driven design in pushing the boundaries of architectural expression. By seamlessly integrating digital and physical processes, it embraces the richness of craft and the transformative power of textiles, ultimately contributing to the ongoing discourse on the role of textiles in contemporary architecture.

60

V. E. Melnyk

References 1. Semper, G.: The Four Elements of Architecture and Other Writings. Cambridge University Press (1989) 2. Kronenberg, A.: Portable architecture. Wiley (1998) 3. Otto, F. (1995). Frei Otto: Complete Works: Lightweight Construction—Natural Design. Birkhäuser 4. Ban, S. (2014). Shigeru Ban: Paper in Architecture. TOTO Publishing 5. Sennett, R.: The Craftsman. Yale University Press (2008) 6. Pallasmaa, J. (2009). The Thinking Hand: Existential and Embodied Wisdom in Architecture. John Wiley & Sons 7. Adamson, G.: The Invention of Craft. Bloomsbury Publishing (2018) 8. Hemmings, J. (2012). The Textile Reader. Berg 9. Benjamin, W.: The Work of Art in the Age of Mechanical Reproduction. In: Arendt, H. (ed.) Illuminations: Essays and Reflections, pp. 217–251. Schocken Books (1936) 10. McCullough, M. (1996). Abstracting Craft: The Practiced Digital Hand. The MIT Press 11. Carpo, M.: The Digital Turn in Architecture. John Wiley & Sons (2012) 12. Barendse, J., Reas, C., & McWilliams, C. (2010). Form+Code in Design, Art, and Architecture: Introductory Book for Digital Design and Media Arts. Princeton Architectural Press 13. Miura, K., Tachi, T., Uehara, R.: Origami and its applications in architecture and engineering. Int. J. Space Struct. 30(3–4), 175–185 (2015) 14. Jackson, P. (2011). Folding techniques for designers: From sheet to form. Laurence King 15. Guagliumi, S. (2008). Hand-manipulated stitches for machine knitters. BookSurge Publishing 16. Hughes, F.: The architecture of error: Matter, measure, and the misadventures of Precision. The MIT Press (2014)

Critical Social Computing for Digital Design Muhammad Talha Muftee1,2(B) 1 Ghent University, 9000 Ghent, Belgium

[email protected] 2 COMSATS University Islamabad, Lahore Campus, Lahore, Pakistan

Abstract. Advancements in computational technology have ushered in a new era of architectural design, characterized by the integration of digital tools. However, this rapid pace of technological innovation has also widened the digital divide, making advanced design solutions less accessible, particularly in underprivileged regions and marginalized communities. Meanwhile, data and algorithmic processes profoundly impact the post-digital built environment. With increasingly complex problems of the built environment, such communities are at risk unless this gap is addressed. This paper underscores the significance of regional computing systems, referred to as social computers, and their emergence in addressing these disparities. To allow architects and experts to act as facilitators of resilience and transformation, critical social computing for design is proposed as a strategy to develop meaningful collaborative practices through digital tools. Through illuminating case studies, instances are identified where architects and experts collaborate as co-coders of social computing, fostering bottom-up approaches to address increasingly intricate challenges in a post-digital world. Keywords: Digital Design · Social Computing · Participatory Design

1 Background 1.1 Rapidly Evolving Digital Design Landscapes With breakthroughs in computer sciences, software development, and human-computer interfacing (HCI), many computational tools have found applications within architecture, leading to new possibilities of design and collaboration [1]. Each iteration of technological innovation occurs at an exponential rate, driven by two main factors. Firstly, each developmental stage of digital tools enables faster innovation, as new computational systems pave way for the next generation of advancements. Secondly, progress takes place in an intangible manner through networks of information. In the age of information, increasing connectivity carries a low-cost barrier, resulting in a larger pool of resources and information, further accelerating technological development. Consequently, it is challenging to critically engage with an ever-changing digital design landscape. While digital tools amplify creative output, even minor misjudgements, miscalculations, and biases can have significant consequences on our environment. It is only in retrospect that we can assess the impact of any technology. Therefore, © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 61–68, 2024. https://doi.org/10.1007/978-981-97-0621-1_8

62

M. T. Muftee

architects applying emerging digital tools for community empowerment or sustainability need to involve all stakeholders in an open and meaningful co-creation process. However, this requires a thorough understanding of local conditions regarding technological access, digital cultures, and how to embed digital tools to encourage ownership. 1.2 Non-Linear Digital Divide Digital divide refers to the gaps in access to advanced technologies across regions, sectors, and groups. However, defining a scale of digitization is a challenging task and limited dichotomies of developed/developing regions only obscure regional nuances. Even within regions where innovative technologies are easily available or developed, digital divide can still exist at a granular level. Since design is a process of ideation and conceptualization, architects can integrate emerging digital tools within their workflows. This creates an even bigger gap with other stakeholders, user groups, and industries. Closing the gap through technology transfers between different regions or different industries is not sufficient since development occurs at an exponential rate. Groups at the leading edge of innovation accelerate at a much higher rate compared to groups trying to keep up. The net digital divide therefore increases with time, changing the baseline definition of what it means for a group or community to be technologically sophisticated. In such situations, architects creatively find ways to collaborate and translate design data into instructional drawings or pre-existing industry standards. Digital communication, social media, and platforms such as Building Information Modelling (BIM) help in creating shared information networks and gathering feedback. With limited integration however, designers only manage to automate some parts of their processes without drastically changing the conventions of design and construction [2]. One example of architects trying to modify digital design methods to suit local construction practices are the several parametric brick design projects that have been built in Tehran in recent years [3].

2 The Social Dimension of Digital Design During the last few decades, digital tools have significantly changed how architects collaborate with other experts through simulations, modelling, and visualization applications. These tools allow for shared virtual spaces of negotiation and collaboration. However, contrary to common perceptions, appropriation of digital tools is often determined by social structures of experts as opposed to precision or productivity. Concurrence with expert intuition, project hierarchy, organizational structures, and social capital often determine which digital tools are more likely to be incorporated into design practices [4]. On the other hand, inhabitants or users integral to formation of the built environment are often not part of these collaborative processes. Either they are completely excluded, or feedback is limited; recorded but not incorporated.

Critical Social Computing for Digital Design

63

2.1 Exclusion by Digital Design While better sensing technologies and digital tools can process increasingly diverse types of information, aspirations and needs of local communities are often not encoded. Mario Carpo in his book The Second Digital Turn: Design Beyond Intelligence points to the contradictory application of digital tools in the post-internet age. The Internet has been one of the biggest technologically driven shifts for the human civilization, creating a global network of knowledge production in which no single individual is just a consumer of information but a participant. Carpo notes that for many years, digital tools have featured collaborative mechanisms which can leverage internet to create immense participatory practices, but this has not happened so. Even in instances when BIM is used, it is used to collaborate selectively with stakeholders who are in a more privileged position than non-expert user groups [1]. 2.2 Contextual Disconnect Lack of community participation potentially obscures crucial contextual information for an architect who relies on abstract digital models of the built environment for social design. Technologies are often perceived as neutral because of their explicit syntax and representation but architects are not entirely free from bias. Even adept digital tool users in architectural domains have limited capabilities to develop digital models that accurately represent complex social realities. Effects of such bias are most noticeable when architects or other experts are working with marginal communities without acknowledging their digital cultures and practices. Resulting in models based on assumptions and alienating the local communities as stakeholders of the environment. To begin designing for any community, it would be imperative to understand how it defines itself and facilitate its decision-making processes through networks of communication without unknowingly excluding one or the other subgroups within that community. In a post-digital society, people no longer define community based solely on physical proximity. The new urban tribes are based on selection of peers in social networks without common spatial links. They are tied to common grounds formed by exchange of information related to interests, past experiences, and shared histories [5]. This intangible yet essential layer will remain obscure unless socially motivated design practices are made to deal with imbalance of agency and local digital cultures. 2.3 Lack of Meaningful Participation Gabriel Arboleda in Sustainability and Privilege: A Critique of Social Design Practice provides detailed accounts of several social design projects with communal collaboration in mind, with outcomes showing a lack of meaningful feedback process. Either local communities are not involved, or they are invited to collaborate at a point when all major design decisions are predetermined by experts; only choices left to make of superficial nature [6]. The author suggests that unpredictability is the only metric for determining meaningful participation in a participatory design process. Since local communities can be complex and non-deterministic, any design process which respects agency of the local community should not have a predictable outcome.

64

M. T. Muftee

In digital practice, meaningful participation can be observed in virtual communities that form around open-source software development. Since such practices are based on ideas of participation and transparency, they are noteworthy examples of collective production. In his book Rhetorical Code Studies: Discovering Arguments in and around Code, Kevin Brock investigates open-source software as social products, developed by diverse groups of people. An ideal open-source project resists the top-down imposed authorship of the primary developer by forking the source code [7]. Forking is defined as the creation of a new branch of source code with a distinct development community. In this manner, the project does not alienate the community and responds to shared ideas and values. Moreover, having multiple co-authors on the same project with an openly accessible code allows individuals to identify potential issues and make corrections. Based on similar mechanisms of open-source code and forking, “WikiHouse” was released as an open-source building system in 2011. Since its inception, the project has been forked into separate branches to suit specific regions or needs [8].

3 Social Computing and Design 3.1 Defining Critical Social Computing Conventional approach for digital design consists of abstracting context in the form of variables, inputs, outputs, functions, relations, and conditions. This allows for solving complex problems but at the cost of bypassing regional knowledge of categorization, understanding, and hierarchies [9]. In the age of ubiquitous computing and internet access, communities around the world are post-digital networks with blurred boundaries between sociocultural and computational layers, turning them into clusters of collective intelligence [10]. Every community is thus an emergent social computer and complex real-world problems are solved by human agents within digital networks [11]. Social computing is conventionally defined as the application of software to analyse and augment social interactions. However, this definition is limited in its scope because cultural aspirations, sociopolitical structures, and tacit knowledge motivate communities to appropriate technologies [10]. These social computers are region-specific and consist of informal exchanges though digital communication. In many cases, marginal groups form social computers of resilience against external forces of displacement. For example, informal trade groups in Durban, South Africa at risk of being displaced from vital commercial spaces, rely to digital communication networks to adapt, and negotiate within marginal spaces [12]. However, there is a lack of understanding regarding importance of social computing within participatory social design. With continuous proliferation of autonomous tools in many facets of mainstream architecture practices, a critical approach is needed to make sure that digital design tools deployed for community empowerment are done in a sensitive manner. Instead of appropriating computing platforms and trying to conform communities to digital standards, the reversal might be more productive for transformative collaboration. As a possible mechanism for participatory practices, this paper proposes “Critical Social Computing” to assess and develop architectural processes aimed at empowering local communities. The lens of social computing can help dispel myths of digital divide

Critical Social Computing for Digital Design

65

within marginal communities by acknowledging post-digital realities. In a world where algorithms have profound impact on the built environment and its stakeholders, architects need to be mindful of impact of digital spaces on physical spaces and vice versa. 3.2 Case Studies: Applications of Social Computing in Design With concerns of climate change, displacement, housing inequity, welfare, and architecture’s role in developing a resilient environment, social design projects serve as important avenue for architects to develop more responsible practices. However, lack of critical engagement with local communities and their aspirations can result in unintended consequences despite well-meaning intentions of experts [6]. Moreover, tools and technologies applied for social design are often restricted either due to cost barriers or emphasis on vernacular aesthetics. The examples of social design selected to demonstrate the application of social computing are notable for acknowledging local communities as emergent post-digital agencies. In each of these cases, the role of the architect or the expert is of a facilitator, co-coding systems of resilience through digital tools. Digital Matatus—Nairobi, Kenya. A collaborative effort between University of Nairobi, MIT, and Columbia University, the Digital Matatus project aims at facilitating existing informal transit networks. Since informal transit networks operate in a bottomup manner often responding to immediate local needs, information regarding routes can be unreliable. The project makes use of GTFS (General Transit Feed Specification) data collected from collaborators using smartphones to map out routes and detect disruptions. By making a vital transit system more visible and accessible to wider public, the project is also able to use data for integrating these systems in future development of the urban fabric. Moreover, collaborators involved in the project allow for tools and datasets to be openly available for replication in other regions and possible improvements [13] (Fig. 1).

Fig. 1. Community-led GTFS data collection and example of transit map on website of Digital Matatus [13].

African Fabbers School—Douala, Cameroon. An architecture and design education program directed by Architect Paulo Cascone, the African Fabbers School (AFS) has set up studio spaces with access to digital fabrication tools offering citizens and local

66

M. T. Muftee

students hands-on learning with design and making. focusing on advanced design and fabrication tools allows AFS to build upon tacit knowledge and craft of regional materials while promoting sustainable circular systems. By creating an open-ended platform of tools and knowledge, the project is critical in exploring ideas of community empowerment, resilience, and innovative approaches to digital tectonics enriching the domain of digital design [14] (Fig. 2).

Fig. 2. Overview of the African Fabbers School shown on COdesignLab website [14].

Hyderabad Urban Lab—Hyderabad, India. The Hyderabad Urban Lab (HUL) is an interdisciplinary research and design collective situated in Hyderabad, India. HUL makes use of digital tools, design, and data analytics to serve and empower people of Hyderabad. One of their key functions is to use data and mapping through local collaboration to perform audits at a city-wide scale. By encouraging citizens to collect, process, and map the built environment, They identify discrepancies and challenge misinformation propagated by developers and government officials regarding marginal communities. Moreover, HUL also uses its skills and resources to continually assess and manage public datasets related to the built environment to make sure that they are in accordance with the actual conditions of the city and openly accessible, allowing wider public to make use of data for problem solving. by disseminating knowledge about data and its implications through social media and workshops, HUL has created a collaborative platform where people gain a sense of ownership of their data and spaces [15] (Fig. 3).

Critical Social Computing for Digital Design

67

Fig. 3. Community mapping facilitated by HUL with local spatial knowledge contradicting preconceived notions [15] (CC BY 4.0).

4 Conclusions The case studies demonstrate how advanced digital tools and data-driven practices can be established to provide diverse communities with systems of resilience that are embedded within existing social computers. In such cases, the architect or the expert does not impose a singular solution but helps facilitate and sets up systems of critical inquiry. Through in-depth investigation of social computing, better informed decisions can be made regarding what tools to implement and how to set up methodologies that work better physical and virtual spaces. To sum up, the defining features of critical social computing for design are as follows: • Critical Cartography. Tools and mechanisms to map local context as a counteraction to large scale GIS systems allowing for people to take ownership of data and using it to dispel normative algorithmic practices [7]. • Modular Design Systems. Embedding architects as creators of classes of digital design tools as opposed to buildings, and spaces. Designing with local communities instead for communities by augmenting and enhancing existing information networks, digital spaces, and practices with digital tools [16]. • Meaningful Participation. Ensuring that participatory projects are open to forking and inherently unpredictable. This also requires that dialogical processes are favoured instead of conventional feedback tools such as questionnaires and surveys which end up imposing reductive choices [6]. • Documenting Dialogue. Making use of correspondence, multimedia, oral history to record participatory processes temporally similar to comments documenting issues, changes and requests within open-source communities. Allowing for transparency and openly accessible records that can inform other practitioners and stakeholders while ensuring that gaps can be identified retrospectively [17].

68

M. T. Muftee

With social computing and emerging digital tools, meaningful collaborative processes can be established for increasingly complex problems of the built environment. By using collective intelligence, systems of empowerment through forking and critical cartography, co-creation can help innovate in a bottom-up manner. Moreover, by situating emerging tools and methodologies in local contexts, digital design can be diversified resulting in innovations that respond to local needs.

References 1. Carpo, M.: The Second Digital Turn: Design Beyond Intelligence, 1st edn. MIT Press, Massachusetts (2017) 2. Hauck, A., Bergin, M. Bernstein, P.: The triumph of the turnip. In: Fabricate: Rethinking Design and Construction, pp. 16–21. UCL Press, London (2017) 3. Canepa, S.: Beyond the façades in Tehran: When tradition meets innovation. J. Civil Eng. Architec. 16(4), 183–194 (2022) 4. Loukissas, Y.A.: Co-Designers: Cultures of Computer Simulation in Architecture, 1st edn. Routledge, Oxon (2012) 5. Foth, M., Sanders, P.: Impacts of social computing on the architecture of urban spaces. In: Augmented Urban Spaces: Articulating the Physical and Electronic City, pp. 73–91. Ashgate Publishing Limited, Aldershot (2008) 6. Arboleda, G.: Sustainability and Privilege: A Critique of Social Design Practice, 1st edn. University of Virginia Press, Virginia (2022) 7. Loukissas, Y.A.: All Data Are Local: Thinking Critically in a Data-Driven Society, 1st edn. MIT Press, Massachusetts (2019) 8. Forks wikihouseproject/Skylark, https://github.com/wikihouseproject/Skylark/forks. Accessed 26 July 2023 9. Mcmahon, C.F.: Predictive machines: data, computer maps, and simulation. In: A Second Modernism : MIT, Architecture, and the ‘Techno-Social’ Moment, pp. 436-473. MIT Press, Massachusetts (2013) 10. Evans, J.: Social computing unhinged. J. Soc. Comput. 1(1), 1–13 (2020) 11. Robertson, D., Giunchiglia, F.: Programming the social computer. Philos. Trans.: Math., Physical Eng. Sci. 371(1987), 1–14 (2013) 12. Odendaal, N.: (D)urban space as the site of collective actions: towards a conceptual framework for understanding the digital city in Africa. In: Augmented Urban Spaces: Articulating the Physical and Electronic City, pp. 255–272. Ashgate Publishing Limited, Aldershot (2008) 13. DIGITAL MATATUS, http://digitalmatatus.com/. Accessed 15 Aug 2023 14. Cascone+Laddaga/CODESIGNLAB, http://codesignlab.org/fr/. Accessed 05 Jul 2022 15. Devulapalli, H., Jonnalagadda, I.: A civic mapping project in an indian megacity: the uses and challenges of spatial data for critical research. In: This is Not an Atlas: A Global Collection of Counter-Cartographies, pp. 120–125. Transcript Verlag, Bielefeld (2019) 16. Cascone, P., Galdi, F., Giglio, A., Ciancio, E.: Architectural self-fabrication. Int. J. Parallel Emergent Distrib. Syst. 32(S1), S39–S53 (2017) 17. Brock, K.: Rhetorical Code Studies: Discovering Arguments in and around Code, 1st edn. University of Michigan Press, Michigan (2019)

Data-Responsive Architecture in Urban Open Space: Sensing Social and Environmental Data and Regulating Spatial Configuration in Real-Time Hyunjae Nam(B) Architectural Association, London, United Kingdom [email protected]

Abstract. The design experiment aimed to examine the application of dataprocessing techniques to regulate a responsive architectural space. The experiment involved simulating an active system of adaptive space that comprised kinetic structures capable of responding to real-time changes in social events and environmental conditions situated in an open space in New York City. The study tested the application of rule-based algorithms that utilise predefined rules to determine the behaviours of kinetic structures by manipulating data. The rule-based algorithm processed the input of urban open data that captured the phenomenon in the open space, resulting in controlling the spatial configurations of kinetic structures. This involved the act of categorisation, classifying the location, schedule, and type of social event data, as well as calculating numeric values of weather parameters such as temperature, cloud cover, humidity, and wind speed to optimise spatial qualities with respect to light/shading, weather conditions, and access. The challenge of the experiment was to create an algorithm that was not only reactive to the input data in real time but also intelligently responsive, providing optimised conditions of an architectural space that adapts to the contexts of both social and environmental changes. The results of the experiment demonstrate the potential for the development of computational techniques to create intelligently responsive spaces in the built environment. Keywords: Responsive Architecture · Real-Time Optimisation · Rule-Based Algorithm · Data Manipulation · Context-Aware System

1 Introduction With the advancement of information technologies, various types of spatiotemporal data have become available for monitoring real-time occurrences in cities. Using such data, the design experiment aimed to develop a set of algorithms to program the behaviour of kinetic structures in an architectural model to be contextually responsive to surrounding conditions. Cedric Price and Nicholas Negroponte pioneered the investigation of self-regulating, feedback-based and physically controllable spaces [1]. By integrating © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 69–76, 2024. https://doi.org/10.1007/978-981-97-0621-1_9

70

H. Nam

sensor and actuator technologies, recent architectural design projects have realised interactive and responsive architectural spaces [2]. The majority of interactive architecture and responsive landscapes have focused on being promptly reactive to the motions of people as well as to instant changes in environmental conditions. In this research, the examination focused on establishing logic-based data processing between sensing and behaving systems in order to enable an architectural model to be contextually responsive to the real-time occurrences in an open space. For the simulation, Bryant Park, one of the most popularly used open spaces for various public events in New York City, was selected. In the algorithm, the act of categorisation classifies text-based data, and the ruleset enables real-time decision making to control the architectural model’s skins and structures. A rule-based algorithm was established to enhance the responsiveness of architectural spaces, not only to be reactive to certain stimuli, but also to be reasonably flexible for users’ activities and environmental conditions. This paper introduces the principles of a data-processing technique that enables responsive architecture to automate adaptive behaviours for both social and environmental data. It describes a method of data manipulation that maps the sequence from sensing temporary occurrences through data to the motions of an architectural model’s compositions.

2 Background 2.1 Architectural Interactivity, Responsiveness, and Intelligence The vision of architectural interactivity, responsiveness, and intelligence was deemed to originate from experiments designed in the 1960s applying cybernetics to architectural spaces, which was to facilitate the constant processing of input data of users and the output of changes of spatial configurations [1]. Cedric Price’s generator project aimed to exhibit how architectural spaces connected to a central computer device could improve spaces through self-generated decisions. Moreover, Nicholas Negroponte sought the convergence of the digital interface and the physical space through the “SEEK” project. These projects attempted to use the technologies of an operating system that could control physical components to achieve architectural intelligence. Their ideas continue to influence the way to think about the relationship between technology and the built environment. In the past two decades, responsive technologies that utilise sensors and actuators have been utilised in architectural practices, exploring the dynamic qualities of landscapes and challenging static design solutions [2]. Collaboration among architects, artists, media designers, and urban planners has led to the development of urban machines that serve as mediums between the materiality of physical components and the immateriality of information [3]. The realm of architectural design has been extended to deal with the dynamic conditions of physical space. However, the limitations of such projects remain, and so the outputs of actions are still simply reactive. The widely-available data which can be accessed via electronic devices, wireless networks, and information resources have facilitated the evolution of the application of sensing and controlling technologies in architectural spaces [4].

Data-Responsive Architecture in Urban Open Space

71

2.2 Data-Driven Urban Development Along with the greater number of data collection methods, the advancement of urbanscale data-processing techniques has influenced the ideal scenario of urban development for the efficient management of occurrences within urban fabrics [5]. Batty argued that the increased collection of data related to human activities shifts the focus of city development “from longer-term strategic planning to short-term thinking about how cities function and can be managed” [6]. Urban data visualisations have been used to monitor digitally collected urban data as well as to provide resources that present the different types of contexts situated in cities [7]. The development of models for visualising data has allowed the emergence of new types of knowledge and the cultivation of a set of visualisation techniques that calculate, manage, and act upon information, driving computational approaches to urban intelligence [8]. Recently, diverse applications of information and communication technologies (ICTs) associated with the terms “intelligent” or “smart” have been promoted to manage information flows in buildings and cities, using urban data as resources. Meanwhile, the issue of urban development has been addressed by discussing how the integration of digital and physical systems can enhance the quality of citizens’ lives. By visualising and analysing location-based data producing spatial knowledge, digital maps have been developed to detect diverse dimensions of the status of cities. Such digital tools have enabled citizens to interact more immediately with short-term occurrences in cities. Nevertheless, physical environments have continued to be built to maintain long-term use for their activities. To improve the urban fabric and physical conditions of existing cities, smart technologies have not been sufficiently applied, and the urban development process needs to focus on the liveability dimensions of cities [9]. Urban data have presented the sources which can be used to understand cities through various dimensions. The challenge of urban development built upon the application of ICTs lies in how such technologies can be applied to the liveability of cultural and historical urban fabrics [10].

3 Design Experiment 3.1 Visualisation of Social Data in Bryant Park The collection of spatiotemporal data has been used to depict a wide range of occurrences, happenings, or temporary situations in cities, enabling the observation and management of short-term changing states. Using such data, this research examined how information on short-term social events and environmental parameters could be manipulated in an algorithm to regulate an architectural space to provide optimised spatial conditions for outdoor activities. It focused on developing a set of algorithms to facilitate self-decisionmaking processes between the data and the model’s behaviours. The data visualisation was carried out to depict the overall view of social activities in Bryant Park and to implement the design strategy. It identified the collected raw data to make them easily readable. A tag cloud was generated to present an overall view of the most repeated social events in the open space. They were grouped into four categories: workshop/class (red), performance/show (yellow), fitness/exercise (green), and

72

H. Nam

party/dance (blue). The bigger words represented events that occurred more frequently (see Fig. 1).

Fig. 1. Tag cloud of the social events occurring in the four locations in Bryant Park (East 42nd Street Allee, Fountain Terrace, Upper Terrace, and Le Carrousel)

In addition, the data visualisation matching with the territorial condition depicted the frequency and duration of social events in the specific locations in the open space (see Fig. 2). The higher peaks represented longer duration, and the larger circles represented the more frequently occurring events. As shown in Fig. 2, the workshop and class types of the small group activities were arranged in the side areas adjacent to the central lawn, and the fitness and exercise types of activities were often held on the lawn and the main

Fig. 2. Data visualisation of social events in Bryant Park (depicting the frequency and duration of events)

Data-Responsive Architecture in Urban Open Space

73

front entrance area. Besides these five areas, sport-related activities were occasionally organised. 3.2 Categorisation of Environmental Data in Bryant Park Along with the observation visualising social data, the visualisation of the weather data parameters, which were collected via US Zip Code 10018, was carried out. The categorisation of the data was applied to a weighting calculation method to regulate the architectural model’s kinetic structures. The levels in the weather data were classified by referring to the standards of weather parameter levels (e.g. the journal written by Sasaki et al. [11] for temperature; the ASHRAE 55–2004 for humidity) (see Fig. 3, left). For example, for temperature, the range between 12 °C and 22 °C was identified as a comfort level; for humidity, values below 20% and above 80% were regarded as critical levels. To establish a calculation method in the algorithm, based on the divided levels, the numerical values in the parameters were redefined to be plus or minus numbers. They were used for the calculation resulting in the final numbers that adjusted the angle of the architectural model’s skin panels (see Fig. 3, right). For example, a temperature of 36 °C, belonging to the discomfort level, was defined as the high minus number that is −81.2, whereas a temperature of 20 °C was defined as −30.

Fig. 3. Categorisation of weather parameters (left) and weighting numbers based on weather parameter levels (right)

3.3 Programming Decision-making Processes With the two datasets, which were directly inserted via URLs and extracted from APIs (data resources: https://data.cityofnewyork.us; https://www.worldweatheronline.com), a set of algorithms was established using Rhino Grasshopper to test the real-time simulation of an architectural model’s behaviours (see Fig. 4). In the algorithm, the input of texts from the social event data was distinguished into the allocated locations and further into four activity types. This route was set to continue to select the predefined alternatives that open and close the model’s kinetic structures. The input of descriptions of weather conditions (e.g., sunny, partly cloudy, rainy) was used to decide whether to completely close the model’s skins for rainy contexts or to calculate the right angle

74

H. Nam

of skins for the factors of temperature, humidity, and cloud cover levels. Whilst the categorised contexts in the social event data selected an option to move the structural frames for the purpose of access control, the categorised weather parameters determined whether the skin components needed to move towards opening or closing for shading control.

Fig. 4. Flowchart of the sequence of input of social and environmental data, decision-making processes, and output of architectural behaviours

4 Real-Time Simulation For the real-time simulation, the algorithm was set to update the two input datasets every 7 min to present the current conditions. The ruleset indicated the behaviours of structures and skins by following updated inputs. The output of numerical values resulted from the selection of alternatives of structures, and the calculation for the skin’s angle was used to move a prototype physical model that was one of the vertically foldable structures (the first front one in the array) rendered in the digital model. With a microcontroller (Arduino) and actuators (servo motors), the motions of the physical model were synchronised with the simulation of the digital model (see Fig. 5). The Arduino board was

Data-Responsive Architecture in Urban Open Space

75

connected to a computer, and the servo motors were placed in the physical model. The values calculated using the rulesets in the algorithm were sent to the Arduino board, and the servo motors received them as signals to rotate the frame structures and the skin panels to the specified positions for both the digital and physical models.

Fig. 5. Real-time simulation of the prototype physical model

5 Conclusion The research examined the design of a set of algorithms to enable an architectural model’s kinetic structures to be responsive to both social and environmental contexts in real time. The predefined rule included the processes of matching and categorising

76

H. Nam

texts, as well as the calculation of assigned weighting numbers based on the levels of weather parameters. Instead of using local sensor data, this research utilised text-based data directly extracted from open data sources via APIs. The examined method presents a possible way to obtain current-time conditions not only for environmental contexts, but also for social events in a specific location, and to manipulate the data so that they can be used for regulating architectural components. Despite the limitations of the model’s self-learning system, the research verified the real-time simulation established by the rule-based algorithms that facilitate self-decision processes. The method can be further employed as one of the underlying methods for the development of a self-regulating system for an architectural model’s dynamic behaviours of kinetic structures.

References 1. Steenson, M.W.: Architectural Intelligence: How Designers and Architects Created the Digital Landscape. MIT Press, Cambridge (2017) 2. Cantrell, B.E., Holzman, J.: Responsive Landscapes: Strategies for Responsive Technologies in Landscape Architecture. Routledge, New York (2015) 3. Signore, M. D., Gernot R.: Urban Machines: Public Space in a Digital Culture. ListLab. (2018) 4. Augusto, J. C., Callaghan V., Cook, D., Kameas, A., Satoh, I.: Intelligent environments: a manifesto. Human-centric Comput. Inf. Sci. 3(12), pp. 1–18. Springer. Heidelberg. (2013) 5. Picon, A.: Smart Cities: A Spatialised Intelligence. John Wiley & Sons Ltd., Hoboken, NJ (2015) 6. Batty, M.: Big data, smart cities and city planning. Dialogues Human Geogr. 3(3), 274 (2013) 7. Ratti, C., Claudel, M.: The City of Tomorrow: Sensors, Networks, Hackers, and the Future of Urban Life. Yale University Press. (2016) 8. Halpern, O.: Beautiful Data: A History of Vision and Reason Since 1945. Duke University Press. (2014) 9. Allam, Z., Newman, P.: Redefining the Smart City: Culture, Metabolism, and Governance. Smart Cities. pp. 4–25. (2018) 10. Allam, Z., Dhunny, Z.: On big data, artificial intelligence and smart cities. Cities 89, 80–91 (2019) 11. Sasaki, R., Yamada, M., Uematsu, Y., Saeki, H.: Comfort environment assessment based on bodily sensation in open air: relationship between comfort sensation and meteorological factors. J. Wind Eng. Ind. Aerodyn. 87(1), 93–110 (2000)

Design of the Intelligent Bridge Drainage Monitoring and Control System Danni Zheng , Yiheng Feng, and Li Li(B) Southeast University, Nanjing, China [email protected]

Abstract. When a bridge spans over water, traffic accidents or hazardous chemical spills may result in pollutants entering the waterway through the runoff on the bridge surface, causing water pollution. This study aims to address the shortcomings of existing bridge drainage systems, such as environmental unfriendliness, low efficiency, and difficult maintenance, and to achieve intelligent and informational control of bridge drainage valves. To achieve this, we developed a bridge drainage intelligent control system based on the existing drainage system of the bridge, using Internet of Things (IoT) technology, machine learning technology and information visualization technology. The system consists hardware modules for detecting bridge surface conditions, long-distance wireless communication modules, intelligent drainage valves, and a server monitoring terminal. When a special situation occurs on the bridge surface, the server monitoring terminal will present real-time information on the bridge and send timely instructions to control the intelligent drainage valves to prevent pollutants from being discharged into the water. This study explores a safe, efficient, and environmentally friendly bridge drainage mode, contributing to the construction of a digital-era urban bridge information management system. Keywords: Emergency drainage of bridges · bridge drainage control · IoT technology · Machine learning · Data visualization

1 Introduction 1.1 Research Background With the rapid development of bridge construction and the continuous increase in motor vehicles in China, there have been frequent incidents and accidents involving hazardous chemicals on bridge surfaces. Hazardous chemicals often possess toxicity, corrosiveness, and flammability, and if they spread into the external environment after transportation accidents, they can have adverse effects on ecology, production, and daily life. The pollutants from traffic accidents and hazardous chemical spills, as well as residues from passing vehicles, will be washed into the bridge drainage system and directly discharged into water bodies during bridge surface cleaning and rainwater flushing. Such water pollution poses a significant threat to aquatic ecosystems, public health, and overall © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 77–84, 2024. https://doi.org/10.1007/978-981-97-0621-1_10

78

D. Zheng et al.

environmental integrity. Traditional bridge drainage systems are not only environmentally unfriendly but also inefficient and difficult to maintain. These limitations highlight the need for innovative solutions that not only address pollution issues but also meet the demands of the digital age. 1.2 Importance of Drainage Control To prevent and control pollution to sensitive water bodies from hazardous chemical transportation accidents, China’s “Notice on Strengthening the Environmental Impact Assessment of Highway Planning and Construction” (Circular [2007] No. 184) stipulates that for bridges crossing second-level protected areas of drinking water sources, quasiprotected areas, and water bodies of Class II and above, a bridge surface runoff collection system should be installed on the bridge, and sedimentation tanks should be set on both sides of the bridge to collect and treat bridge surface runoff in case of pollution accidents, ensuring the safety of drinking water. In actual bridge environmental assessments, there are strict requirements for bridge drainage, and if the corresponding standards are not met, the bridge cannot be put into use. 1.3 Research Objectives Currently, the development of smart cities is heading towards safety, intelligence, and sustainability. Smart bridges, as an integral part of smart cities, integrate advanced information technologies such as the Internet of Things (IoT), big data, and cloud computing. Therefore, this research aims to bridge the gap between traditional bridge drainage practices and advancements in the digital era. The primary objective is to develop an intelligent, information-driven bridge drainage control system that seamlessly integrates with existing infrastructure. This study focuses on the development and implementation of a novel smart bridge drainage control system that can proactively respond to specific situations on the bridge through IoT and machine recognition technologies.

2 Literature Review 2.1 Current Bridge Drainage Systems The bridge drainage monitoring and control system can be divided into two aspects: bridge drainage system design and monitoring control system design. The former focuses on researching and designing bridge drainage equipment. For example, Yang Wenjuan [1] and others conducted research on drainage pipe channel dimensions, spillway valve control methods, and wastewater collection pool design. This research further explores the intelligent diversion of pollutant-carrying and pollutant-free runoff. The monitoring control system is currently used in various fields, including community security, fire warning, and structural detection. Bridge safety monitoring has provided valuable experience and technical support, including hardware module selection and warning system construction, for the bridge drainage monitoring system. The existing bridge intelligent drainage monitoring control system is mainly applied in retrofit projects. For example,

Design of the Intelligent Bridge Drainage Monitoring

79

in Nanjing’s Qixia Mountain Bridge[2], a low-pressure pneumatic control system was developed to automate the drainage cover based on the original drainage system and manual cover. The control logic laid the foundation for this research. 2.2 Object Detection and Tracking Technology With the continuous improvement of object detection technology, detection-based tracking has become the mainstream paradigm in the field of multi-object tracking. ChienYao Wang [3] and others introduced Simple Online and Realtime Tracking (SORT) and YOLOv7 in their work. SORT is a practical multi-object tracking method, especially suitable for vehicle monitoring at high frame rates. YOLOv7 is a real-time object detector with excellent speed and accuracy and has a wide range of applications. SORT and YOLOv7 provide important technical support for the camera’s use of machine recognition technology to monitor and judge object behavior, providing significant solutions for vehicle monitoring, tracking, and stationary detection.

3 Project Implementation 3.1 Integration of IoT Technology, Visualization, and Machine Learning The Internet of Things (IoT) comprises four layers: perception, network, platform, and application. In bridge drainage systems, the sensors in the perception layer can transmit real-time bridge information to the network layer for analysis and processing. At the platform layer, machine recognition technology is utilized to identify vehicle abnormalities. The intelligent monitoring terminal at the application layer can connect to the platform layer through the network, visually presenting the status and performance of the bridge drainage system for administrators’ intuitive understanding of the system’s operation. The integration of these elements provides a more intelligent and efficient drainage solution, bringing broader prospects for the development of smart bridges and smart cities. 3.2 Design Concept Controlling pollution in bridge runoff is an urgent priority. Furthermore, this article also emphasizes the efficiency of bridge maintenance. Manual interventions in bridge maintenance are often disrupted by bridge traffic and rainfall, leading to high maintenance costs and slow issue resolution. Therefore, we have developed an automated system based on IoT and machine recognition technologies to intelligently manage bridge drainage, eliminating the need for on-site personnel for drainage operations and condition monitoring. The primary objective of this system is to manage bridge runoff in an efficient and environmentally friendly manner.

80

D. Zheng et al.

3.3 Overall Framework The project takes the Bailong River Bridge as an example, located in Longnan, Gansu, China, spanning the Bailong River with a length of approximately 210 m. During the operation of the bridge, rainfall can lead to the entry of pollutants into the receiving water body [4]. Therefore, the design of bridge drainage maintenance should consider both emergency events on the bridge surface and rainfall conditions to ensure that pollutants on the bridge surface are not discharged into the water under the bridge and can be effectively collected. The following are the drainage control solutions for rainfall and emergency events (Fig. 1).

Fig. 1. Drainage control process flowchart

Rainfall. When rainfall starts, rainwater washes the bridge surface and passing vehicles, carrying pollutants from both vehicles and residues on the bridge surface. To prevent direct discharge of bridge surface runoff into the river, all valves are kept in the closed position during rainfall, guiding water containing pollutants through drainage channels into a collection pool. After 20 min of rainfall, all valves are opened, allowing the bridge surface runoff to flow freely into the river. Emergency Events. When accidents, such as collisions or leaks involving vehicles on the bridge, especially oil tankers carrying hazardous chemicals, the hazardous substances can cause severe pollution to the river’s water. Consequently, when such accidents occur, the smart cameras on the bridge can swiftly detect irregularities in vehicle behavior and promptly relay this information to the server. Subsequently, the server triggers the closure of valve groups near the accident site, effectively preventing pollutants from the vicinity of the vehicle’s drainage outlets from entering the river. Once the camera detects that the vehicle has left, the backend server controls the valve groups to reopen. Remarkably, this entire process unfolds without the need for on-site personnel, leading to significant reductions in response time and labor costs. Additionally, emergency buttons are installed at regular intervals on-site, allowing drivers or on-site personnel to activate them when accidents occur. in such cases, the server also promptly initiates emergency procedures.

Design of the Intelligent Bridge Drainage Monitoring

81

4 System Implementation 4.1 Hardware Design In this solution, the perception layer, consisting of a weather station and video surveillance equipment, collects and transmits real-time environmental data to the processing layer. The network layer ensures data transmission, employing wireless technology to communicate with the cloud platform and control drainage valves. The platform layer processes data, making intelligent decisions based on real-time information and issuing control instructions as needed to prevent water pollution. The application layer serves as the user interface, offering real-time monitoring, data display, and remote control capabilities. The implementation of the system requires various hardware components, including bridge drainage equipment, intelligent drainage valves, wireless gateway control boxes, cameras for monitoring vehicle stops, on-site emergency buttons, and a cloud server for sending and receiving instructions (Fig. 2).

Fig. 2. System design schematic

Wireless Communication Modules. Considering the large span of the bridge and its location in a mountainous area, wireless communication modules need to meet the requirements of long-distance transmission, low power consumption, and high transmission quality. After comparing bluetooth, Wi-Fi, ZigBee, and LoRa Wireless Communication Technologies, LoRa is chosen for its advantages, such as ultra-long transmission distance, extremely low power consumption, relatively low cost, and reasonable security [5]. Therefore, the LoRa wide-area network transmission technology is adopted in the proposed solution. lora modules are used in the wireless gateway control box, intelligent drainage valves, and emergency buttons to enable communication and information exchange between various modules. Wireless Gateway Control Box. The wireless gateway control box is equipped with lora wireless communication modules and a 4G gateway. The LoRa module can communicate with the LoRa modules in the emergency buttons and smart drainage valves, While the 4G gateway can transmit the information received by the LoRa module to the backend

82

D. Zheng et al.

server. Simultaneously, the 4G gateway receives instructions from the server and transfers them to the drainage valves through the lora module. Intelligent Monitor. The intelligent monitoring system comprises multiple recognition nodes installed on the bridge surface, each covering a 200-m range. These nodes collaboratively perform real-time surface monitoring and provide road information feedback (Fig. 3). When unusual road conditions occur, the system can autonomously detect and report to the server, automating data collection and feedback. Each node includes a high-definition camera, edge computing device (Jetson Nano), and 4G network device. The edge computing device captures camera images and uses YOLOv7 neural network algorithms for vehicle location and DeepSort for trajectory tracking. It also employs a heat map algorithm to progressively identify prolonged vehicle stops. The combination of tracking and heat map algorithms can detect extended vehicle parking on the bridge, signaling a potential emergency, and promptly report it to the server (Fig. 4).

Fig. 3. Camera monitoring of vehicles on the bridge surface

Fig. 4. Intelligent camera workflow diagram

Intelligent Drainage Valves. Intelligent drainage valves are located at the bridge’s drainage outlets, with one installed every five meters, and they are controlled remotely by the backend server. The backend control system assigns a unique identifier to each valve and manages them, enabling the opening or closing of individual valves or combinations through command transmission. The valves have an initial status of being open. In the open position, bridge surface runoff can flow directly through the drainage outlets into the area beneath the bridge. In the closed position, bridge surface runoff is collected in a central pool. Each drainage valve is equipped with LoRa modules, and when the backend server sends commands, the LoRa module on the wireless gateway communicates with the LoRa module on the drainage valve to transmit the control signals.

Design of the Intelligent Bridge Drainage Monitoring

83

Emergency Buttons. The emergency buttons are equipped with LoRa modules, allowing on-site personnel to trigger them in case of emergencies on the bridge. When an emergency button is activated, its LoRa module sends the information to the back-end server, which then processes the information. Weather Stations. The weather station receives meteorological data and transmits it to the backend control system. The backend control system issues valve control instructions 20 min After the weather station reports the start of rainfall. 4.2 Software Design The intelligent monitoring system enables remote device management and real-time on-site information access through both mobile and desktop platforms. It leverages the Django backend framework and the React frontend framework to provide system management, data administration, and data visualization functionalities(Fig. 5).

Fig. 5. Visualized dashboard of bridge drainage control information on large screen.

System Management Module. The system management module comprises user management and device management. The backend management system is divided into administrators and regular users. Administrators possess permissions for device management and data CRUD, while regular users lack such privileges. The device management module is a crucial component of the backend management system, encompassing the management of lora gateway devices and valves. In Django, user and device data models can be created by defining Models, and the Django Admin backend is employed for data administration. Data Management Functionality. Data management functionality primarily involves storing and managing user data, valve control records, and meteorological data using databases such as MongoDB. In Django, the interaction between data models and the database can be handled using Django ORM. After defining Models, Django automatically generates the corresponding database tables and provides APIs for data CRUD operations. Data Visualization Module. The data visualization module presents the status information of bridge drainage equipment through a large-screen data display. The large

84

D. Zheng et al.

screen integrates comprehensive information, including equipment statistics, valve status diagrams, valve control records, camera surveillance images, and meteorological information. In react, interactive frontend interfaces can be created, and react components are utilized to display information on the large screen. Additionally, with the help of the Echarts charting plugin, data retrieved from the MongoDB database can be seamlessly converted into JavaScript objects or arrays, allowing easy visualization of data into intuitive charts and graphs, making information more illustrative and easy to comprehend.

5 Summary and Prospects At present, the drainage system of Bailong River Bridge has been successfully put into use. The significance of this research extends beyond the scope of bridge drainage management, as it not only achieves environmentally friendly and efficient drainage management but also demonstrates significant potential for integration with urban bridge management systems, contributing to the development of holistic and digitally-driven urban infrastructure methods. Furthermore, the adaptability of this system allows for its expansion to other management projects, thus promoting sustainable practices across different environmental settings. However, our study has its limitations. Implementing intelligent drainage systems on a large scale necessitates addressing technological challenges, considering costs, and addressing potential integration issues with existing infrastructure. In future research, areas such as network security, equipment updates, and addressing more complex traffic issues will become focal points of investigation. We believe that our research contributes to the broader goal of constructing technologically advanced and ecologically responsible cities.

References 1. Yang, W., Chen, Y., Wu, P., & Zhao, J.: Runoff drainage system design for highway bridges crossing sensitive water. J. Chang’an University, 60–67 (2020). https://doi.org/10.19721/j.cnki. 1671-8879.2020.04.006 2. The City Transportation Group in the Same Industry in China’s First Crossing the River Bridge Hazardous Chemical Leakage Emergency System. (2020). Nanjing Communications Group WeChat Public Platform. https://mp.weixin.qq.com/s/AkS6Fzdit8iAPN4JWvy0DQ 3. Wang, C., Bochkovskiy, A., Liao, H.M.: YOLOv7: Trainable bag-of-freebies sets new stateof-the-art for real-time object detectors. arXiv (Cornell University). (2022b) https://doi.org/10. 48550/arxiv.2207.02696 4. Li, Q., Qian, R., Gao, J., Huang, J.: Environmental impacts and risks of bridges and tunnels across lakes: an overview. J. Environ. Manage. 319, 115684 (2022). https://doi.org/10.1016/j. jenvman.2022.115684 5. Fan, Z.: A fire safety research and design of real-time monitoring system (a master’s degree thesis, Southeast University). https://kns.cnki.net/kcms2/article/abstract?v=InnWydrwIfIr ZLykqEF96ZcRu5N3zolSFBpswq3HpQWoX1zv-4irKFiaYHWWS9_n4qagLnEiU7wrSGh 3w5u2GileJArEtwy4VTdrxxmAF7u_JORDKyIO4M9CxazZ1eiWorDLmd48Has=&unipla tform=NZKPT&language=CHS (2022)

SAUCE – SpAcevehicle-bUilding Connectivity Evaluation An Adjacent Matrix Based Digital Tool for Internal Connectivity Evaluation and Improvement in Spacevehicle-Buildings Zhelun Zhu(B) Xi’an Jiaotong-Liverpool University, Suzhou, China [email protected]

Abstract. Spacevehicle-building represents a peculiar design problem with many stringent requirements, including unusual surrounding conditions and severe cargo limitations. The latter is caused by the launch system capabilities and impacts on the internal room’s dimensional and mass characteristics. Therefore, one of the Spaceship’s design challenges consists of its internal room organisation considering the needs under holistic perspectives. In particular, it is crucial “how” internal rooms are related since their optimal use depends on well-defined connections. SAUCE proposes an automatised tool that evaluates internal connections’ performance in Spacevehicle-Buildings by digitalising the Adjacency Matrix. This method is used in architectural design to represent the relationship between rooms while its digitalisation will bring the readability and execution by a machine. The proposed tool provides a computable and objective assessment of “how” the internal functions are related to each other, describing a Spacevehicle-Building internal organisation from spatial layout perspective. Implementation of this workflow include feeding the scoring system to a metaheuristic algorithm-based generative process to investigate optimal combinations between the functions, providing support to the design process. Keywords: Space Architecture · Generative Design · Digital Design · Adjacent Matrix · Spatial Layout Design

1 Introduction and Research Objectives Crewed Space explorations represent one of the most ambitious challenges of the Contemporary era. The endeavour started in the 1960s and even today, there are still many unresolved issues. During the Space staying, terrestrial living beings must deal with many hostile and unusual conditions like extreme temperature, radiation exposition, impact with micrometeoroids or orbital debris, as well as the absence of vital resources, including air, water, food and “room” (Martinez 2007; Whitmore et al. 2013). The latter is strictly related to the Spacevehicle’s capabilities to provide solutions in Extra-Terrestrial Environments (ETEs) and directly influence the Space inhabitants living quality (Zhu et al. 2020). © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 85–92, 2024. https://doi.org/10.1007/978-981-97-0621-1_11

86

Z. Zhu

Nowadays launch systems are featured with stringent cargo limitations, impacting the Spacevehicle-Building internal rooms’ dimensional and mass characteristics. These features directly influence their functionalities and user experiences, causing many psychological stresses – for instance the feeling of confinement, isolation, and lack of privacy – bringing to anxiety, nostalgia, judgement impairment, and related physiological issues, affecting negatively astronauts’ wellbeing and jeopardising the success of the entire mission (Whitmore et al. 2013). In this vein, one of the design challenges consists of the room organisation within the Spacevehicle-Building’s payload fairing considering its needs under holistic perspectives and inhabitants’ perception. It has been pointed out the crucial role of “how” the internal rooms are connected in influencing their optimal uses and improving the user experience (Simon et al. 2011). For instance, astronauts complained about the constant noise produced by some technical devices near their rest area (NASA 2007) and on the other hand, studies have pointed out that the view on greenhouses can reduce the perception of isolation and confinement, bringing benefits in term of psychological relieves during long period missions (Häuplik-Meusburger et al. 2014). The resulting design problem includes furnishing proper internal functions and establishing their layout under an optimal configuration within the Spacevehicle-Building’s cargo limitations. This problem is featured with fuzzy variables, given by many possible combinations between internal functions’ number and their connections, as well as the need to deal with mutual interests between different systems, leading to a complex design problem. As can be imagined, manual iterations and assessment requires tremendous effort and would be exposed to subjective decisions if based only on human operators. The present research investigates a digital and automated method to evaluate internal connections in Spacevehicle-Buildings. It analyses the relationship between different functions to evaluate the entire configuration objectively. During this process, it will be considered the peculiarities of Space conditions and the potentiality of the proposed tool will be investigated.

2 State of the Art In “Architect’s Data”, E. Neufert introduced the graph representation in architectural design to describe the relationship between different areas (Neufert 1980). This strategy consists of a set of connected vertices, providing a visual representation of how n number of rooms are related to each other. Furthermore, the graph strategy finds the Adjacent Matrix as a compact way to represent the relationship between its vertices. The Adjacent Matrix is used in architectural layout representation and comprises n x n elements in n rows and columns. Each element expresses the relationship between the correspondent vertices and consequently, the resulting matrix is symmetric (Fig. 1). Traditionally, the Adjacent Matrix is manually related to the design process through human operators, which could be affected by their subjective evaluation and limitations (Lorenz et al. 2015). To this regard, Coraglia proposed a digital workflow able to evaluate the room performance according to its relationship with the external view (Coraglia et al. 2021). SAUCE has been developed as a crucial part of Author’s Previous Work (APW) called PASTA – Parametric Approach for Space Transplanet hAbitations (Zhu et al. 2022) that

SAUCE – SpAcevehicle-bUilding Connectivity Evaluation

87

Fig. 1. An example of Adjacent Matrix (Lorenz et al. 2015).

investigates an automatised framework to organise internal rooms in a SpacevehicleBuilding. It leverages on generative design approach and adopts an evaluation system to lead and select satisfying outcomes. During this process, the internal connectivity performance has been considered as one of the parameters that affects the user experience. Nevertheless, it has been considered all the modules in a storey equally, without differencing physical connectivity from the visual one. Furthermore, there hasn’t been considered the relationship with the adjacent modules in different storeys, which could compromise some functionalities.

3 Methodology The connectivity performance in SAUCE is achieved by digitalising the Adjacent Matrix and expressing the relationship between the rooms in numbers, making it machinereadable. Then the sum of all connections’ evaluation of a given configuration provides an objective evaluation of “how” the internal rooms are related, describing the overall space layout organisation. This work starts with the definition of an Adjacent Matrix for Spacevehicle-Building. It leverages results from “Specific Design Adjacency Considerations” provided by NASA’s Man-System Integration Standards (NASA 1995), which point out that astronauts’ activities should follow a logical sequence and avoid interference. Crucial consideration consists in distributing the functions according to crew members’ involvement (individual or group) and their privacy requirements (public or private). For simplification purposes, this work considers the classification proposed by Wickman and Anderson (2009) and gathers astronaut’s activities into corresponding modular Functional Space Units (FSU) which are: • Work Area, hosting the mission command and research activities; • Rest Area, provides rest and privacy to the astronauts; • Greenhouse, food supply during the Space exploration and user experience improvement. • Health and Hygiene Area, including medication, services and exercise rooms; • Dining Area, where meals are prepared and eaten. Here also include space for group entertainment activities;

88

Z. Zhu

On the other hand, the technical and landing areas host crucial functions for the Spaceship and their position is established by the manufacturing company. The first host equipment while the second is where the external hatch and the spacesuit are collocated and stored. In particular for the landing area, it also allows the entrance and exit from the Spaceship during Extra Vehicular Activities and, therefore, includes an airlock for pressure regulation and adequate room for astronauts’ preparation. Due to their requirements and peculiarities, these functions are not involved in the internal configuration design since they are fixed from the outset and can’t be separated. Therefore, their position is externally established in the present workflow and occupies the entire storey. The resulting Adjacent Matrix is shown on the right side of Fig. 2 where the relationship between the rooms are indicated as: • • • •

proximate, functions that require immediate connection; close, in a reasonable distance; remote, indicating undesirable and interference connections; indifferent, which collocation does not interfere with their utilisation.

Fig. 2. LEFT: Consideration for the Relative Locations of Space Module Functions Based on the Results of Functional Relationships Analysis (NASA 1995), with the classification in FSU by the author in SAUCE; RIGHT: Adjacent Matrix for a Spacevehicle-Building (Author).

SAUCE has been validated using as a case study the Starship, the Spaceship developed by SpaceX. It is featured with internal available volume assimilable to a 9.00 m diameter cylinder, corresponding to 1,000 mc of payload fairing. The latter is subdivided into modular rooms, then assigned to FSUs according to the framework proposed in APW (2022). In particular, the rooms have been simplified to alleviate the calculation process and are represented with different colours to differentiate each function. It has been used a BIM environment since it allows components’ ontological identification and multi-dimensional data management (Succar 2009).

SAUCE – SpAcevehicle-bUilding Connectivity Evaluation

89

Fig. 3. Available volume in a Starship and internal modules representation.

Figure 3 provides the graphical representation of the available volume – payload fairing – within the Starship and the distribution logic proposed in PASTA. As an example, it has been highlighted the module with “ID_Number” = 4, its adjacent modules and the correspondent relationship. As can be observed, it is possible to individuate the adjacent modules given a module’s “ID_Number” and its remainder from four (number of modules per storey) “k” (Table 1). Table 1. Indexes to “ID_Number” to individuate the adjacent modules under different “k”. k = −1

k=0

k=1

k=2

Proximate

+1; +3

+1; −1

−1; +1

−1; −3

Close

+2; +4; −4

+2; +4; −4

+2; −4; +4

−2; −4; +4

Remote

−1; −2; −3; +5; +6; +7

−2; −3; −5; +3; +5; +6

+2; +3; +5; −3; −5; −6

+1; + 2; +3; −5; −6; −7

It has been proposed a workflow that allows the import of the Adjacent Matrix as an external input. This operation enables a more flexible and collaborative design process using a spreadsheet, which can be loaded in the BIM platform – in this case Revit 2024 – through an interface in Dynamo. The file is read as a list of lists, identified with the correspondent indexes where the lower level indicates the element’s row position in the Adjacent Matrix while the upper one the column position. Therefore, indexes can be used to identify the relationship between two rooms in the Adjacent Matrix and have been assigned to FSU as “Functional ID”.

90

Z. Zhu

The connectivity evaluation process is repeated for each module in a configuration according to Fig. 4. Benefits of using BIM methodology emerge in data and ontological association of the generated objects (Zhu et al. 2021). In particular, every generated module has been assigned two parameters which are: “ID number”, used to identify the object and for its collocation in the payload fairing; and “Functional ID”, which addresses to its position in the AM. These data are stored in dictionary form to enable their extraction by a key. Since the modules are sequentially collocated, the “ID number” allows the individuation of the adjacent modules and consequently, the correspondent “Functional ID”.

Fig. 4. SAUCE workflow.

For demonstrative purposes, there have been generated 100 configurations with a stochastic algorithm and Fig. 5 reports a series of connectivity performance examples evaluated according to the SAUCE workflow.

SAUCE – SpAcevehicle-bUilding Connectivity Evaluation

91

Fig. 5. Connectivity evaluation examples using SAUCE workflow.

4 Discussion and Conclusion The present research has investigated an automatised method to evaluate the connections’ performance in a Spacevehicle-Building. Advancements consist in digitalising the Adjacent Matrix strategy, allowing its readability and execution from a machine. The proposed workflow enables a flexible and collaborative design process since the Adjacent Matrix is externally defined and can be updated according to emerging needs. Compared to the APW in 2022, a more comprehensive – considering the adjacent modules in different storeys – and detailed – differencing connections type – evaluation has been developed. The scoring system used in the SAUCE workflow has been proven as valid design support in expressing the connectivity performance. There have been pointed out several configurations (for example in 62 and 74) with high value, which means their above-theaverage performance and worthy of further investigation. Nevertheless, the real situation is a multi-objective problem that requires the satisfaction of additional performances (e.g., cargo limitation, number of crew members, radiation exposure) under a functional relationship since their parameters could influence each other. The scoring system can be fed to a metaheuristic algorithm-based generative process to investigate optimal configurations of a Spacevehicle-Building (Zhu et al. 2022). In this context, SAUCE can lead to the best solutions-finding process. Benefits from this framework consist of automating part of the design process and providing support for complex problems’ resolution, releasing designers from repetitive tasks and enhancing their creativity, especially at the early stages of the design process. SAUCE has been developed for Spacevehicle-Building, featured with limited internal volume and specific requirements. Nevertheless, the connectivity evaluation also interests many terrestrial buildings where the connection between functions is vital for

92

Z. Zhu

their correct utilisation. This performance is particularly relevant in complex buildings, such as hospitals and common habitations. In these cases, it is essential to properly manage users’ flows to prevent undesirable intersections and avoid possible conflicts.

References Coraglia, U.M., Zhu, Z., Fioravanti, A., Simeone, D., Cursi, S.: A new relation matrix as a fruitful meta-design tool how to overcome typological limits. In: Proceedings of the International Conference on Education and Research in Computer Aided Architectural Design in Europe, vol. 1, pp. 295–302 (2021) Häuplik-Meusburger, S., Paterson, C., Schubert, D.: Greenhouses and their humanizing synergies. Acta Astronautica 1–14 (2014) Lorenz, W.E., Bicher, M., Wurzer, G.X.: Adjacency in hospital planning. IFAC-PapersOnLine 28(1), 862–867 (2015). https://doi.org/10.1016/j.ifacol.2015.05.118 Martinez, V.: Architecture for space habitats. Role of architectural design in planning artificial environment for long time manned space missions. Acta Astronautica 60(4–7 SPEC. ISS.), 588–593 (2007). https://doi.org/10.1016/j.actaastro.2006.09.034 NASA. NASA-STD-3000, Man-System Integration Standards. Section 8 Architecture: Vol. I (1995) NASA. Final Report of the International Space Station Independent Safety Task Force (2007) Neufert, E.: Architects’s Data Second (In). Blackwell Science Ltd. (1980) Simon, M., Whitmire, A., Otto, C.: Factors Impacting Habitable Volume Requirements for Long Duration Missions. NASA (2011) Succar, B.: Building information modelling framework: a research and delivery foundation for industry stakeholders. Autom. Constr. 18(3), 357–375 (2009). https://doi.org/10.1016/j.autcon. 2008.10.003 Whitmore, M., et al.: Evidence Report: Risk of Incompatible Vehicle/Habitat Design Human Research Program Space Human Factors and Habitability Element (2013). https://humanrese archroadmap.nasa.gov/Evidence/reports/HAB.pdf Wickman, L., Anderson, G.: Activity-based habitable volume estimating for human spaceflight vehicles. In: IEEE Aerospace Conference Proceedings, pp. 1–7 (2009). https://doi.org/10.1109/ AERO.2009.4839707 Zhu, Z., Coraglia, U.M., Fioravanti, A.: PASTA – parametric approach for space transplanet habitations, a generative powered design process for internal configurations in Spaceships. ECAADe 2, 47–55 (2022) Zhu, Z., Coraglia, U.M., Simeone, D., Fioravanti, A.: Spaces Identity Evaluation aNd Assignment - SIENA A duck typing approach for automatic recognition and semantic enrichment. Towards a New, Configurable Architecture - Proceedings of the 39th ECAADe Conference - University of Novi Sad, Novi Sad, Serbia, 8–10 September 2021, vol. 2, pp. 341–350 (2021). https://doi. org/10.52842/conf.ecaade.2021.2.341 Zhu, Z., Fioravanti, A., Coraglia, U.M.: Space vehicle-building design process issues and models. A framework. In: Ghaffarianhoseini, A., Ghaffarianhoseini, A., Nasmith, N. (eds.) Imaginable Futures: Design Thinking, and the Scientific Method, 54th International Conference of the Architectural Science Association 2020, pp. 986–995 (2020)

Digital Experience Session

Challenges and Opportunities in Using Digital Pedagogy for Game-Based Architecture Education: A Case in China Silvia Albano(B)

, Wan Meng , Wenruo Xu , and Na Li

Xi’An Jiaotong-Liverpool University, Suzhou, China {silvia.albano,na.li}@xjtlu.edu.cn, [email protected], [email protected] Abstract. Digital pedagogy (DP) has received increasing attention from different disciplines to study the dedicated use of contemporary digital technologies (e.g., virtual learning environments and digital media platforms) for inclusive and personalized learning and teaching. Applying DP appropriately can help teachers to shift from traditional instruction methods towards more interactive and engaging approaches for high-quality education. Under the umbrella of DP, digital gamebased learning (DGBL) represents a pedagogical approach that utilizes interactive and immersive digital games to nourish the development of critical thinking and problem-solving skills for student-centred active learning. However, how teachers can apply DGBL effectively in architecture education is still being determined. We conducted a qualitative case study to explore student engagement after implementing the DGBL pedagogy supported by H5P escape room learning activities in an architecture course with 121 university students. Our study found that the game elements and the characteristics of Architecture education influenced students’ learning engagement in four aspects (behavioural, emotional, cognitive and social). The main research findings contribute to digital pedagogy by extending people’s understanding of the DGBL’s impact on learner engagement with rich empirical data. Limitations and practical implications were discussed for future development. Keywords: Architectural Education · Digital game-based learning · H5P · Escape room · Engagement

1 Introduction and Literature Review In recent years, the field of education has witnessed a significant transformation fuelled by the rapid advancement of digital technologies. Several studies and research explore how to integrate digital and immersive tools to enhance the existing workflow of students and teachers by highlighting advantages and disadvantages in education and practice and pursuing different goals [6, 7]. One of the main aims of implementing this new approach in Architecture education is to bridge the gap between theoretical concepts and real-world applications. However, successfully integrating these two methods in the Architecture realm remains a complex endeavour, often presenting numerous challenges and untapped opportunities. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 95–102, 2024. https://doi.org/10.1007/978-981-97-0621-1_12

96

S. Albano et al.

H5P is an open-source technology that provides a versatile platform for creating and delivering interactive content that can be seamlessly integrated into learning management systems and web-based applications. Therefore, this study explores mainly the following research question: In what ways the use of H5P-assisted DGBL in curriculum design improves learning engagement? 1.1 Digital Game-Based Learning and Architecture Education Digital Game-Based Learning was first coined by [19] as “the coming together of serious learning and interactive entertainment into a newly emerging and fascinating medium”. Over the past two decades, the inclination towards DGBL has gained momentum in the educational context, particularly online and blended educational contexts. DGBL is seen as a tool that can benefit architecture education. From the original French École des Beaux-Arts movement to the giant leap of Bauhaus ideas to our present-day emphasis on interdisciplinary instructional methods, the long history of Architecture education has evolved around the project-based praxis and the student-centred constructive approach [25]. The professional practice, as a significant and yet influential segment of education in the study of Architecture, helps in many ways; it allows students to understand the theoretical knowledge at a practical level, get acquainted with current trends in domestic or foreign professional practice, develop teamwork and problem-solving skills [20]. This lays the foundation for the application of DGBL in Architecture education. Firstly, DGBL effectively promotes the development of students’ problem-solving skills [2, 12], harmonising architecture education’s educational purpose. Furthermore, DGBL experts believe that the true power of digital games lies in using real-world context and authentic tasks [16, 24]. This allows learners to apply what is learned by solving real problems from the field and activate relevant prior knowledge, aligning with Architecture education’s main focus. Another benefit of DGBL is the self-oriented trial and error process. In Architecture education, students employ various techniques, including modelling and sketching, to generate personal expressions [25]. Similarly, DGBL creates a safe environment for the learners to try without fear of failure [16]. 1.2 Impact of Digital Game-Based Learning on Student Engagement Engagement refers to “psychological investment in and effort directed toward learning, understanding, or mastering the knowledge, skills, or crafts that academic work is intended to promote” [18]. It is a meta-construct that includes different dimensions and indicators. The models of engagement proposed by different researchers have consistently addressed three common sub-constructs that include a) cognitive engagement, which involves the mental effort investment in learning and the idea of self-regulation or the use of cognitive strategy; b) behavioural engagement, which is related to observable behaviours of active participation, such as effort, concentration, attendance to classes, asking questions and contributing to class, commonly understood as time “on task”; and c) emotional engagement, which refers to feelings and emotional reactions, such as interest, enjoyment, enthusiasm, feelings of belonging, and value of learning [9–11, 15]. More recently, a MOOC (Massive Open Online Courses) engagement scale further distinguished social engagement, centred on learner-instructor and learner-learner

Challenges and Opportunities in Using Digital Pedagogy

97

interactions, from behavioural engagement as the fourth dimension of learner engagement in MOOCs [5]. All the dynamically interrelated dimensions of engagement are essential mediators of interest in game-based learning [9]. In this line of reasoning, high engagement entails heightened concentration, enjoyment, commitment, effort, and positive effect toward the activity, showing a significant positive footprint on learning and teaching [13]. This study focuses on the four components of learner engagement: behavioural, cognitive, emotional, and social while examining the relationship between learner engagement and a DGBL intervention in an Architecture course.

2 Research Methodology 2.1 Research Setting The four-year full-time BEng-Architecture at the Department of Architecture at Xi’an Jiaotong-Liverpool University (XJTLU) in Suzhou (China) for the academic year 22/23 has started to experiment in that sense using the H5P tool by choosing a 5-credit taught module. The selected module is named ARC306 Professional Practice, which introduces students to the role and responsibility of the architect as a professional and in the construction industry, together with exposing them to concepts about the management of an Architecture practice. During the previous academic years, this module has recorded low levels of engagement and interest by students that have become evident throughout the collection of low attendance rates and discontinuous levels of engagement, and interaction with the teaching team. In light of these special conditions and the additional learning and teaching issues faced during the last three years of the Covid-19 pandemic, during the second half of the academic year 22–23, the ARC306 module included among its contents an additional support section to help students consolidate and internalize the knowledge and learning outcomes provided by the module consciously and maturely. Specifically, the activity selected by the various packages of options included in the H5P tool was that of interactive videos linked to a system of escape rooms. A series of three short videos (maximum 5 min each) related to the fundamental topics of the module were made, referring to the essential part of the final assignment requested for completing the module successfully. The main aim of the activity was that students watched these videos answering questions on the main topic embedded in them. To continue watching the video and especially the following ones, the students had to provide the correct answer; otherwise, they would get stuck on the video watched. In this way, students could resolve doubts and strengthen the essential concepts to complete their final submission and achieve the fundamental module’s learning outcomes.In this research, the main strategy relied on a previous study [26] employing a new digital technology, HTML 5 Package (H5P), to design and deliver learning in a professional practice class within the Architecture curriculum through online digital escape rooms on the Moodle-based virtual learning environment, namely the learning mall. The primary motivation of the Module Coordinator for introducing the online digital escape room supported by the H5P technology was to improve students’ learning experience and performance. Providing a highly engaging activity can help students achieve the learning outcomes in the authentic content of DGBL and increase

98

S. Albano et al.

the course participation rate that in these last years registered a quite low attendance percentage and a discontinuous level of engagement. 2.2 Participants, Data Collection and Analysis The non-probability sampling technique was employed in this study to obtain feedback faster, simpler, and more practically [27]. 10 senior undergraduate students, 6 females and 4 males, who attended the escape room in the Architecture professional practice course, voluntarily participated in the one-on-one interviews. Before data collection, all participants received an explanation of the project to ensure that they were fully aware of the research aims. The consent form and participant information sheet were provided and signed in advance. Qualitative data collection methods were employed with semistructured individual interviews to examine different dimensions of learner engagement in the course and key drivers of engagement created by the features embedded within the DGBL activity. Qualitative research facilitates a deeper understanding of the causality between people’s beliefs and understandings, their behaviour, and the context of the intervention being implemented [17]. Additionally, the interview has been the mainstay of qualitative research to give a “voice” to persons for authentic experiences that are important to the present study’s research questions [4, 23]. The one-on-one interviews lasted 20 to 30 min each. All interviews were audio-recorded. Two researchers conducted the interviews via online meeting, transcribed the audio recordings into text, translated the Chinese text into English if the interview was conducted in Chinese, and manually cross-checked the auto-transcription and translation results before analysing the data. The recorded interview transcriptions were coded with NVivo 12 qualitative analysis software. Data analysis adhered to the thematic analysis approach [22], with a hybrid approach incorporating deductive and inductive coding [8]. A predetermined code manual was developed based on the research questions and literature review. The final version of the code manual consisted of two parts. First, the four dimensions of learner engagement (behavioural, cognitive, emotional and social) were divided into 11 subcategories. Second, the coding scheme focused on identifying factors that enhance learner engagement in the DGBL setting. Data analysis was conducted separately, which allowed the independent coder to gain familiarity with the coding scheme. After confirming and consolidating the identified coding categories, the researchers met online to compare their analysis and classify results. The two independent coders discussed all disparities until an agreement was reached. Before the thematic analysis was finished, two rounds of coding comparisons were examined together with a literature search. Overall dimensions relevant to the research questions were identified and further analysed.

3 Results 3.1 Student Engagement To identify students’ performance on academic engagement following the use of H5P, the researchers gained insights from the literature and empirical data (interviews and open-ended survey questions). This study conceptualized academic engagement in an explanatory matrix across four dimensions: (1) behavioural engagement, (2) emotional engagement, (3) cognitive engagement (4) social engagement.

Challenges and Opportunities in Using Digital Pedagogy

99

Behavioural Engagement. The behavioural engagement dimension concerns students’ observable actions and their participation and involvement in educational activities. It entails time on task, students’ behaviours regarding rules and expectations, and student participation in learning activities, including effort, persistence, concentration, attention, asking questions, and contributing to the discussion. Student state excerpt examples: Student A: “I may use it for a long time”. Student B: “I will follow its order, take notes while watching the video, and then answer the questions”. Student C: “When I use the H5P, I will ask questions if there’s something I don’t understand”. Emotional Engagement. Emotional engagement refers to students’ affective reactions in the classroom, including interest, boredom, happiness, sadness, and anxiety. Some conceptualize it as identification with school, belonging (a feeling of being essential to the school), and value (an appreciation of success in school-related outcomes). Student state excerpt examples: Student D: “I think H5P is a fun learning activity that makes it a bit easier to learn, you don’t have to memories, and you can remember the knowledge more easily.” Student E: “I think it’s one of the more emerging learning experiences that I’ve been exposed to, so I’m just interested in it, and that would then lead me to be very willing to learn about it and continue to watch these related learning videos.” Cognitive Engagement. Cognitive engagement defines as the levels of processing theory, including the idea of deep versus shallow engagement. Deep engagement involves actively using prior knowledge and intentionally creating more complex knowledge structures by integrating new information with prior knowledge. Shal-low engagement involves rote processing and other intentional cognitive actions that are more mechanical than thoughtful. Student state excerpt examples: Student F: “Because I use it at the end of the course to help with revision, and because many of our exams are taken at the end of the course, H5P provides an active learning and independent exploration process that allows students to work independently to review their knowledge, which can help improve performance and aid our learning.” Student G: “I think it’s good to take notes while watching the video because you might miss a section in the process, so you can go back and watch it again.” Social Engagement. Social engagement is centred on learner-instructor and learnerlearner interactions. It highlights the importance of collaboration, social learning, interaction and communication patterns in online discussion and motivation for attending offline meetings. Student state excerpt examples: Student H: “The most helpful thing is that I can use it to communicate with my teachers later, which has helped me a lot.” Student I: “Because each group of students may have different ideas when reading the same material, you can use this platform to see different perspectives on the same knowledge point and generate some collisions of ideas. The teacher will also better understand each student’s ideas because usually no one answers or interacts in class,

100

S. Albano et al.

but in this way, the teacher will better understand the students’ ideas and give better feedback.” 3.2 Factors that May Influence Students’ Engagement The game elements have been divided into four relevant elements that address cognitive and affective aspects: (a) motivational elements (i.e. elements that influence players’ thoughts, actions and reactions regarding meaningful play and learning); (b) interactive elements (i.e. elements that provide players with opportunities to engage and participate in gameplay activities); (c) fun elements (i.e. elements that provide players with a sense of fun and excitement); and (d) multimedia elements (i.e. elements that engage players through physical and/or multi-sensory interaction) [13]. Students perceived H5P as a gamified approach to learning similar to a breakout game, which would provide more fun and game rewards and a more experiential feel through multimedia technologies such as video interaction. Students are, therefore, motivated to use H5P for learning. Architectural education can be broken down through play, and the quality of education improves with this process, making nonformal education more suitable for lifelong learning than formal education in terms of operational flexibility and student attitudes to education. Students can experience first-hand the fundamental principles of Architecture, ‘the creation of events’ and ‘the construction of environments’, through the ‘environmental and spatial transformations’ that characterise informal education. Student state excerpt examples: Student J: “H5P is building a full aspect scenario based on knowledge, we can get a more practical feel for how it works, and it feels very interactive, so I like it.” Student I: “I think it’s a breakout-like format, which is quite interesting.” Student E: “As this course is mainly about how to build a building, H5P uses this interactive approach to learn a lot about building construction, such as some safety hazards and what we need to be aware of during the actual operation.” Student F: “The course itself is a vocational education-related course, which will help you to understand the real industry.”

4 Discussion and Conclusions Thanks to the results obtained through this research study, it is possible to affirm how the introduction of DGBL, specifically the H5P tool, stimulates positive reactions by students and tutors. This approach is supported by the existing body of recent literature demonstrating the potential benefits of embedding these game technologies for improving the learning experience’s quality and enhancing student engagement. In this case, students will not feel uncomfortable about their possible failure during the first attempts. Otherwise, students will feel more motivated to re-do the activity to enhance their final results and, as an essential consequence, improve the quality of the knowledge acquired, understanding previous mistakes and possible misunderstandings. The feeling of inadequacy and performance anxiety that students generally experience attending classes or modules will be replaced by a positive sense of self-improvement based on the principles of emotional engagement [9–11, 15]. The description above

Challenges and Opportunities in Using Digital Pedagogy

101

represents a clear example of how DGBL raises students’ focus and active commitment since it facilitates learning engagingly and joyfully. Furthermore, the qualitative data collected in the results reveal that the design and provision of DG pedagogy, through the H5P tool, in a lecture-based module of the Architecture curriculum can effectively boost students’ behavioural engagement. It is evident in the interview report how students changed their attitude regarding the interaction with tutors, peers, and the module’s tasks. Understanding the positive benefits of utilizing gamified elements in their learning experience, they started to wish to be more active during the class session sharing doubts or considerations to clarify any possible misunderstanding. DGBL has also detractors and critics. For example, [3] explains that the introduction of games into the contemporary learning system has been abused by the Companies that produce the games themselves for profit, given the rampant popularity recorded in recent years. These analyses can generate mistrust about the effectiveness of implementing game design principles in education because they highlight an absence of profound design studies and research. On the other hand, it is also true that despite the weaknesses mentioned above by the misuse of gamification, its principles introduced in the educational field are promising to improve students’ educational experience and spread complex concepts appealingly [1]. To conclude, this study made practical and theoretical contributions to improve students’ engagement and learning experiences within and beyond Architecture education. In addition, this research showcased an excellent practice of digital games as elements that provide a safe and friendly environment for students [21]. Overall, this paper contributes to the existing body of knowledge by shedding light on the challenges and opportunities associated with using digital pedagogy for game-based Architecture education. As a future enhancement plan, there is the intention to expand the planning of gaming activity to the whole module to involve students since the earlier stage of the course topics’ dissemination.

References 1. Awan, A., Lombardi, D., Agkathidis, A., Ruffino P.: Efficacy of gamification on introductory architectural education: a literature review. In: eCAADe2022. KU Luven, Ghent (2022) 2. Backlund, P., Hendrix, M.: Educational games - are they worth the effort? A literature survey of the effectiveness of serious games. In: 2013 5th International Conference on Games and Virtual Worlds for Serious Applications (VS-GAMES), Games and Virtual Worlds for Serious Applications (VS-GAMES), 2013 5th International Conference on, pp. 1–8 (2013) 3. Bogost, I.: Why Gamification is Bullshit. The Gameful World: Approaches, Issues, Applications. The MIT Press, Cambridge (2014) 4. Cohen, L.: Research Methods in Education (L. Manion & K. Morrison (eds.) 8th edn. Routledge (2018) 5. Deng, R., Benckendorff, P., Gannaway, D.: Learner engagement in MOOCs: scale development and validation. Br. J. Edu. Technol. 51(1), 245–262 (2020) 6. Faraj Al-Suwaidi, M., Agkathidis, A., Haidar A., Lombardi, D.: Application of immersive technologies in the early design stage in architectural education: a systematic review. Archit. Plan. J. (APJ) 28(3), Article no. 27 (2022)

102

S. Albano et al.

7. Faraj Al-Suwaidi, M., Agkathidis, A., Haidar, A., Lombardi, D.: Immersive technologies in architectural education: a pedagogical framework for integrating virtual reality as the main design tool in a fully virtualised architectural design studio environment. In: Digital Design Reconsidered - Proceedings of the 41st Conference on Education and Research in Computer Aided Architectural Design in Europe (eCAADe 2023), Graz, 20–22 September 2023, vol. 1, pp. 89–98 (2023) 8. Fereday, J., Muir-Cochrane, E.: Demonstrating rigor using thematic analysis: a hybrid approach of inductive and deductive coding and theme development. Int J Qual Methods 5(1), 80–92 (2006) 9. Filsecker, M., Kerres, M.: Engagement as a volitional construct: a framework for evidencebased research on educational games. Simul. Gaming 45, 450–470 (2014) 10. Fredricks, J.A., Blumenfeld, P.C., Paris, A.H.: School engagement: potential of the concept, state of the evidence. Rev. Educ. Res. 74(1), 59–109 (2004) 11. Greene, B.A.: Measuring cognitive engagement with self-report scales: reflections from over 20 years of research. Educ. Psychol. 50(1), 14–30 (2015) 12. Gros, B.: Digital games in education: the design of games-based learning environments. J. Res. Technol. Educ. 40(1), 23–38 (2007) 13. Henrie, C.R., Halverson, L.R., Graham, C.R.: Measuring student engagement in technologymediated learning: a review. Comput. Educ. 90, 36–53 (2015) 14. Jabbar, A.I.A., Felicia, P.: Gameplay engagement and learning in game-based learning: a systematic review. Rev. Educ. Res. 85(4), 740–779 (2015) 15. Jimerson, S.R., Campos, E., Greif, J.L.: Toward an understanding of definitions and measures of school engagement and related terms. Calif. Sch. Psychol. 8(1), 7–27 (2003) 16. Kebritchi, M., Hirumi, A.: Examining the pedagogical foundations of modern educational computer games. Comput. Educ. 51, 1729–1743 (2008) 17. Maxwell, J.A.: A Realist Approach for Qualitative Research. Sage (2012) 18. Newmann, F.M., Wehlage, G.G., Lamborn, S.D.: The significance and sources of student engagement. In: Newmann, F.M. (ed.) Student Engagement and Achievement in American Secondary Schools, pp. 11–39. Teachers College Press, New York (1992) 19. Prensky, M.: Digital Game-Based Learning. McGraw-Hill & Paragon House, New York (2001) 20. Radovi´c, G., Frki´c, S., Nemetschek, V.: Contributions of digitization and professional practice to the study of architecture. In: Uskov, V.L., Howlett, R.J., Jain, L.C. (eds.) Smart Education and e-Learning 2020. SIST, vol. 188, pp. 605–613. Springer, Singapore (2020). https://doi. org/10.1007/978-981-15-5584-8_51 21. Li, N., Reis, C.: Using H5P to enrich online learning engagement in a postgraduate certificate program teaching. In: ElSayary, A., Olowoselu, A. (eds.) Overcoming Challenges in Online Learning. Routledge (2023). https://doi.org/10.4324/9781003342335-14 22. Saldaña, J.: The Coding Manual for Qualitative Researchers (3E [Third edition]). SAGE (2016) 23. Sandelowski, M.: Reembodying qualitative inquiry. Qual. Health Res. 12(1), 104–115 (2002) 24. Sanchez, A., Cannon-Bowers, J.A., Bowers, C.: Establishing a science of game based learning. In: Sanchez, A., Cannon-Bowers, J.A., Bowers, C. (eds.) Serious Game Design and Development: Technologies for Training and Learning, pp. 290–304. IGI Global (2010) 25. Ta¸scı, B.: Theoretical framework for using digital games in architecture education. Archit. Res. 6(3), 51–56 (2016) 26. Unsworth, A.J., Posner, M.G.: Case study: using H5P to design and deliver interactive laboratory practicals. Essays Biochem. 66(1), 19–27 (2022) 27. Vehovar, V., Steinmetz, S., Toepoel, V.: Non-probability Sampling (2016)

Digital Creativity in Urban Interventions: Using Technology as an Engagement and Idea Inducing Tool Daria Belkouri1(B)

and Theodoros Dounas2

1 Robert Gordon University, Aberdeen, Scotland

[email protected] 2 University of Antwerp, Antwerpen, Belgium

Abstract. The urban planning discipline is increasingly turning to specialized technologies to better understand the multiple and complex processes within cities. Automation is already reshaping infrastructure and urban ecosystems. This study seeks to reconciliate technological agency and creative research methods by utilising laser scanning technology to increase community participation in planning and speculate about urban possibilities of using walking and pedestrian viewpoints as drivers for urban design. This research employs hybrid methods, on the one hand the simple act of walking and on the other hand visualising the urban space using a 3D laser scanner. This mixed technique makes it possible to increase the perception of individuals with respect to urban space, becoming more aware of it and coming into greater contact with it generating ideas about spaces encountered on the route. Keywords: Laser Scanning · Urban Walking · Participation · Urban Design

1 Introduction - Technology as Creative Agent Digitisation is everywhere [1]. With the proliferation of (often Artificial Intelligence [A.I] generated) data there seem to exist a concern that it can be misleading, inaccurate, emotionless and dull [2]. Automation is already reshaping and altering our economy, culture, and urban ecosystems [3] yet, its potential might rest in employing the technology to augment and complement our ability to solve problems, ignite progress and utilise technology to create equitable, inclusive and just urban environments. There are numerous instances of art and design installations that represent the hybrid of digital technology and real objects integrating the digital into the physical, challenging existing paradigms in a creative and critical way. For example, the artists Heinrich and Palmer have used laser scanning in architectural immersive installations combining point cloud data, video projections and sound effects [4, 5] treating large architectural objects as porous membranes to show the fleetingness of scanned objects and to reveal and explore different interpretations of existing spaces by often showing them rescaled and from unexpected perspectives. Thomas Pearce’s [6] study, on the other hand, explored © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 103–110, 2024. https://doi.org/10.1007/978-981-97-0621-1_13

104

D. Belkouri and T. Dounas

the phenomena of fictional point clouds in a three-dimensional laser scan created by split dimensions at the edge of an object. Faulty measurements were utilised as productive points that enabled designers to actively create the edge noise, via precisely perforated screens. The artists utilised the errors to convey experiments in actual urban context [6] where fictional geometric content was elaboratively placed into supposedly “realist” point cloud data set. By challenging the practicability of the technology, new meanings and possibilities of spaces were revealed through experimentation producing a hybrid yet deeper understanding of the context. These examples of artistic experiments with technology served as an inspiration to this study. On one hand the truly inclusive view of creativity would encompass the agency of both humans and technology [7]. Yet, is there still space for utilising this hybrid creativity in an algorithmic future and age of digital reproduction? The coupling of humans’ creative agency with the torrent of possibilities provided by the technology should serve as a hybrid tool to optimise the work processes and provide space for new sustainable developments and innovation ([2] on application in urban planning). Thus, nurturing our agency to embrace the technological advancements in urban narratives, we introduce our study that uses laser scanning to elicit ideas and speculate about urban possibilities via the analysis of the act of walking within the urban space and its ability to increase community participation.

2 Context - Using Technology as an Engagement Tool The urban planning environment is increasingly turning to specialized technologies to address uses related to sustainability, society, security, transportation, infrastructure and governance [2] to better understand the multiple and complex processes within cities. Yet, in the current digital world, there can often be observed a shift disengaging from the operational towards the sensorial and sensitive engagement with the physical world [8]. What frequently becomes a focus of research attention ‘are the different sensorial qualities and embodied affordances’ with further explorations into subjective ambiences and spatial possibilities generated by those reflections that we could experience while inhabiting our cities [8]. It can be argued that this ‘permeability’ with surroundings could potentially be endangered by the algorithmic agency depriving citizen participatory and co-creation voices. This research therefore seeks to reconciliate technological agency to induce participatory and creative mobile practices – conjoining the active mobility and laser scanning technology. It investigates how co-creating urban walkability may be enhanced through a method in which digital representation of the walking experience and immersion in both real and virtual settings offer a novel approach to participatory urban design. While part of a wider study, this paper presents findings from engaging primary school children in the research, based on the presumption that ‘[a]lthough we make all kinds of conscious decisions around the nurturing of our children, it seems that the way in which they are travelling receives only limited attention’ [9]. Including children in research meant that the scope of the study was enriched with creative yet important voices. Therefore, this research seeks ways of inclusive urban designing and imagining to avoid the reality where people ‘are reduced to patterns of data’ in automated planning processes [10] through the

Digital Creativity in Urban Interventions

105

lens of engaging gamification in urban context, and the potential relationships between civic gaming processes, digital technologies and smart urbanism [11–14]. From the situationist “derive” and “flânerie” with ‘insightful reflective gaze on urban realities’ [15, p.399], which aspired to challenge the status quo by integrating play, spontaneity, intuition and critical thought underlining gaming’s revolutionary and rising potential [11] to indeed, a number of cities throughout the world [16, 17] that are currently generating ideas aimed at meaningfully engaging the public in the planning and design processes by utilising digital technology to foster connections between people and the urban areas - they highlight the significance of play and enjoyment to activate a creative urban investigation [11, 18].

3 Aim and Objectives This study explores the complexity of experiential aspects of walking through creative use of technology to establish whether the re-discovery of often familiar context would strengthen the idea that the process of deepening the connection to and subjective ‘interpretation’ of our cities could possibly improve our experience of inhabiting urban spaces [19, 23]. This research looks at how we perceive space when walking. It investigates how the urban environment may promote and positively affect walking by seeing the city from an abstract perspective and employing novel technology and qualitative data gathering techniques. Walking and mapping is used as a study tool to record reality and uncover new ways of appreciating the urban environment. The study’s objectives include testing the response to technology and the feasibility of its application, as well as recording the experience to the real and abstract urban contexts while walking to develop an enhanced understanding of context and inspiration for urban solutions.

4 Methodology Our method extends the impact of the point cloud representation, beyond the technical survey, where accuracy and consistency are the scope. Through the point cloud data, we have created a highly precise yet abstract rendering of the city, which acts as an ephemeral representation. This has been created with the use of a portable laser scanner, rather than a stationary device. The act of creating and gathering the data takes place through walking, where the activity of walking does not conform to simply getting from one place to another but is itself an engagement integral to perception of the environment. Hence the output from laser scanner - point cloud image - becomes an artefact that encapsulates the new knowledge generated by walking, but also acts as a further focal point of engagement in the second part of our methodological apparatus, where we engaged in “walkshops” with local school children. The data collection was divided into two sequential parts. The Phase 1 was performed solely by the first author where the journeys in the city were recorded and captured with a mobile laser scanner to depict the experience of walking in the city. The data output was then processed by the first author and presented in various visual forms to act as a springboard to test the response to digital technology and feasibility of its application.

106

D. Belkouri and T. Dounas

The objective for using the laser scanning in a novel way was to expand human ability to perceive the context beyond the initial visible elements – discovering the city anew, recomposing and reconfiguring the experience of being in the city, potentially triggering creativity and co-creation element. Phase 2 encompassed organising ‘Walkshops’ for primary school children to expose participants to the re-imagined visions of the city in order to explore potential change in walking behaviour. The existing urban environment was presented in an unexpected way [25, 26 on visual parallelism and artistic qualities of mapping] to promote city discovery and inspire children’s own visions for the space. This activity was complimented by the walk-along semi-structured interviews that encouraged contemplation and observation of the immediate surroundings. The first author also developed the ‘postcards’ method where the informants were asked to share their thoughts on cards containing images representing certain scanned areas within the city on one side and a question on the reverse (Fig. 1). This task was envisaged for the walking and talking aspect – ‘walkshops’ to evoke responses that static and only verbal interviews would not potentially achieve.

Fig. 1. Examples of postcards [modelled and drawn by the first author, 2022]

The school participating in the research was recruited following a positive response to the email explaining the research subject and inviting children to participate in the study. Once the school was identified the first author pre-recorded the walking routes in the area with a portable laser scanner and strategically defined the length of the route (to last no more than 45min to accommodate the sensory abilities of pupils) and to follow the looped route surrounding the school grounds. The route was then visualised on the map to mark the ‘stops’ and sent to the school for approval. The consent forms were emailed out prior to research activities and distributed by the teachers who also handled organisational aspects such as a number of pupils per adult. The children were equipped with high vis vests - as per school’s health and safety requirements. The first author was present at all times explaining and reiterating the rules of the ‘walking game’ at each stop. Breaks were strategically chosen with sufficient surface area (pavement or green patch) to allow for all of the children to gather safely together and have room to pause and write down their answers (Fig. 2). On the day of ‘walkshops’ the first author introduced the technology to the children in the classroom explaining that the activity was arranged to resemble an outdoor urban game. The first stop along the route was outside the school building with postcard

Digital Creativity in Urban Interventions

107

depicting a scan of a mature tree which served as a conversation prompt to talk about the importance of greenery in urban areas. The children were tasked with finding the exact frame of the scan in the real-world environment. The playful approach to interview combined with walking encouraged children to share their thoughts and feelings in an unusual way. The walkshop facilitator’s instructions promoted active, reflexive absorption of surroundings, reflection on the intricacies of the context that might elicit emotions, and resonance with earlier experiences throughout the journey. During the walks children commented on the exact spots where they catch ‘Pokemon’ – alluding to PokemonGo! urban game of augmented reality whereby children (as well as adults) walk the city streets and look out for virtual elements placed in the exact points referable on the actual map of the surroundings [20]. It could be argued that this activity of playful cartography and augmented reality applications found a huge resonance in children and adults alike (with myriad of mapping games, applications, social networks or locative artworks etc.) where connection is found between technology, people, spatial surroundings and maps [14].

Fig. 2. Map of the route and children at first ‘stop’ [drawn by first author, 2022]

Some children also reminiscent on old nursery when walking past it or identified shops where they used to go with parents and where they would often meet their friends. The study revealed that friends and memories associated with physical spaces encountered along the routes are inextricably linked and critical to each other. It can be said that the access to past experiences and memories inspired by the act of walking deepens the experience of the walker and creates personal bonds with the surroundings [21]. Thus, the combination of abstracted space through point cloud images and deliberately ‘aware’ approach to walking within the study would seem to hold the potential to similarly create new associations and memories, in turn having an effect on perception of surroundings and the values attached to them. During the next stages of the ‘game’ the children were striving to position themselves at the actual spots of the point cloud frames from the postcards. This prompted a discussion on the use of scanning technology in an urban context as well as its representation. The first author encouraged a dialogue with the surroundings via the lenses of point cloud images revealing the juxtaposition of the familiar buildings and streets

108

D. Belkouri and T. Dounas

and leading the group to notice surprising elements of the context (Fig. 3). The children picked up on the nuances that can be observed while walking (‘a dog in the window’, ‘noticed flowers are growing again’) as well as details of the church that was depicted on the postcard mentioning ‘ornaments on the top’, ‘spiky bit on the top [of the church], or a ‘cross at the top of the door’. The context paralleled with point cloud images gave a wider perspective and ignited an interest in the route through the familiar spaces to some of the study participants. The children mentioned that they like playing games while walking ‘like I spy and unlucky 13’ or the possibility of just talking to a friend or buying an ice cream from local cafes indicating the importance of sociality of walking with possibility of pleasant intervals - like treasure hunts or just being playful ‘I would just run on my hands’ or ‘turn into giant and go very fast’.

Fig. 3. Examples of postcards from ‘walkshops’ [drawn by first author, 2022]

Environmental consciousness triggered by walking could also be observed as words like ‘environment’ and ‘ecofriendly ‘were in abundance and suggestions that ‘maybe instead of cars people could use roller skates, roller blades, bikes and skateboards’. The surroundings characterised by ‘more trees less cars’ were of significance as well as meditative, cognitive elements – somewhat therapeutic aspects of walking, have also been mentioned by the children treating walking as an opportunity to clear the mind, find solutions and explore unexpected places - ‘you see more of the city’. When asked, while walking, about any ideas of how the space could be used differently the creative potential of children was truly unfolded as the suggestions apart from ‘planting more plants and talking area’, ‘speed bumps or no road only pavement’ (in front of the school)’, ‘café in the playground’, ‘gymnastics obstacles’ included ‘water slide’, ‘ice cream van’, ‘fountain with fresh water’ and ‘tree houses with video games’. Yet, these answers prove that there is a torrent of practical improvements and creative ideas (some more realistic than others) but grounded on awareness of surroundings and observation of smells, textures, material aesthetics that were induced by the slow rhythm and potentially playful approach to engagement.

Digital Creativity in Urban Interventions

109

5 Discussion In this study, we investigated an experimental approach to participatory design by combining environmental capture in laser point clouds and walking as a method of inquiry. We employed a combination of ‘postcard’ survey approach in which participants were asked to immerse themselves in the context while walking and considering the parallel point cloud visuals created of the same context. We merged components of digital representation, walking as a research and design tool, and parallelism of virtual and actual surroundings in the study’s design. Our results were based on a relatively small sample of workshop participants as opposed to other techniques of collecting data via online surveys employing immersive VR [22] yet, yielding rich qualitative data on discovery of new spatial qualities, aspects of playfulness, changed behavioural perspectives and perception of urban environment. The results also provided insights into how urban spaces could be improved on the spot by participants’ ideas generated while walking. It can be argued, that the critical observation and engagement of study partakers have been enhanced by the creative use of technology to appreciate and capture the experience of urban walking practices, rather than utilising the technology to only produce a precise geometrical record. Instead, the innovative tools were used to enhance human capability to reflect and perceive beyond the mundane and visible city—re-discovering it, and reconfiguring the familiar, generating co-creation elements whilst walking and thinking about spatial design [23]. It is recommended that the influence of augmented and virtual reality as well as gamification on how we research but also perceive and potentially design the urban environment gives opportunity for further exploration [24].

References 1. Baciu, D.C.: Creativity and diversification: what digital systems teach. Thinking Skills Creativity 4(100885), 1871 (2021) 2. Yigitcanlar, T., et al.: Artificial intelligence technologies and related urban planning and development concepts: how are they perceived and utilized in Australia? J. Open Innov. Technol. Market Complex. 6(4), 187 (2021) 3. Thirgood, J., Johal, S.: Digital disruption. Econ. Dev. J. 16, 25–32 (2017) 4. Heinrich and Palmer: Casting Light. Point Cloud images (2019). https://heinrichpalmer.co. uk/project/casting-light/. Accessed 12 June 2023 5. Heinrich and Palmer: Travelling Light. Light installation (2016). https://heinrichpalmer.co. uk/project/297/. Accessed 12 June 2023 6. Pearce, T.: Orchestrating the edge: towards a noisy point cloud onto-epistemology. Des. Ecol. 4(1–2), 142–170 (2014) 7. Bejan, A.: Human evolution is biological & technological evolution. Biosystems 195 (2020) 8. Jensen, O.B.: Of ‘other’ materialities: why (mobilities) design is central to the future of mobilities research. Mobilities 11(4), 587–597 (2016) 9. Brömmelstroet, M., Nikolaeva, A., Glaser, M., Chan, C., Nicolaisen, M.: Travelling together alone and alone together: mobility and potential exposure to diversity. Appl. Mobilities 2, 1–15 (2017) 10. Choi, J.H., Forlano, L., Kera, D.: Situated automation. Algorithmic creatures in participatory design. In: PDC 2020, vol. 2, pp. 15–20 (2020)

110

D. Belkouri and T. Dounas

11. Vanolo, A.: Cities and the politics of gamification. Cities 74, 320–326 (2018) 12. Hollands, R.: Will the real smart city please stand up? Intelligent, progressive or entrepreneurial? City 12(3), 303–320 (2008) 13. Kitchin, R.: Making sense of smart cities: addressing present shortcomings. Camb. J. Reg. Econ. Soc. 8(1), 131–136 (2015) 14. Lammes, S.: Digital cartographies as playful practices. In: Frissen, V., Lammes, S., de Lange, M., de Mul, J., Raessens, J. (eds.) Playful Identities. The Ludification of Digital Media Cultures, pp. 199–210. Amsterdam University Press, Amsterdam (2015) 15. Aroles, J., Küpers, W.: Flânerie as a methodological practice for explorative research in digital worlds. Cult. Organ. 28(5) (2022) 16. Playable City. https://www.playablecity.com/. Accessed 17 July 2023 17. Zuckerman, O., Gal-Oz, A.: Deconstructing gamification: evaluating the effectiveness of continuous measurement, virtual rewards, and social comparison for promoting physical activity. Pers. Ubiquit. Comput. 18(7), 1705–1719 (2014) 18. Cowley, R., Joss, S., Dayot, Y.: The smart city and its publics: insights from across six UK cities. Urban Research & Practice (2017) 19. Lynch, K.: The Image of the City. Massachusetts Institute of Technology Press, Cambridge (1960) 20. Potts, R., Jacka, L., Yee, L.H.: Can we catch ‘em all’? An exploration of the nexus between augmented reality games, urban planning and urban design. J. Urban Des. 22(6), 866–880 (2017) 21. Calvert, T.: An Exploration of the Urban Pedestrian Experience, Including How it is Affected by the Presence of Motor Traffic. University of the West of England (2015). Ph.D. 22. Kasraian, D., Adhikari, S., Kossowsky, D., Luubert, M., Hall, B., Hawkins, J.: Evaluating walkable streets with a 3D stated preference survey. In: 99th Annual Meeting of the Transportation Research Board (2020) 23. Belkouri, D., Lanng, D.B., Laing, R.: Being there: capturing and conveying noisy slices of walking in the city. Mobilities 17(6), 914–931 (2022) 24. Graham, M., Zook, M., Boulton, A.: Augmented reality in the urban environment: contested content and the duplicity of code. Trans. Inst. Br. Geogr. 38(3), 464–479 (2013) 25. Amoroso, N.: The Exposed City. Routledge, New York (2010) 26. Tufte, E.R.: Visual Explanations. Images and Quantities, Evidence and Narrative. Graphic Press, Cheshire (1997)

Digital Hybridities: Theorising the ‘Social’ and the ‘Local’ of Fabrication Technologies in Craft Practice Matthew Holmes(B)

and Alejandro Veliz Reyes

University of Plymouth, Plymouth, UK {matthew.a.holmes,alejandro.velizreyes}@plymouth.ac.uk

Abstract. The digital design research community keeps moving towards increasingly techno-centric spaces. Largely development-focused, the evolution of technology adoption in our sector is often reported from performance and efficiency perspectives, instead of its efficacy – its implementation and adoption in realworld design practices and cultures. In that context, this paper presents a theory framework to investigate the adoption of digital fabrication technology among craft practitioners - emphasising the challenges among craftsperson’s activities protected by both heritage and cultural traditions developed outside authored and institutionalised contexts of digital design research. Through the coupling of theoretical strands originated in management studies (socio-materiality), media obsolescence, heritage, and crafts theory, our approach is demonstrated through the infusion of digital fabrication tools into the analogue world of letterpress printmaking. Through methods such as community engagement, codesign and prototyping, and graphic outputs, we reflect on issues emanating from uniting a technology that has already faced industrial obsolescence with newly developed digital tools, including the influence of heritage and cultural traditions, tensions around irreplaceability, the need for preservation, spatial and community manifestations, and knowledge transfer and translation across seemingly diverging ways of engaging with technology. The resulting approach challenges the standard hegemonic analogue/digital dichotomy and allows for the interrogation of a more ‘complexified’, nuanced field of hybrid practices encouraging a balance between preservation, tradition, and experimentation. Keywords: Socio-materiality · Digital Fabrication · Digital Craft · Letterpress

1 Introduction Technology and media undergoes a ‘life cycle’ comprising phases of innovation, mainstream adoption, and obsolescence [1]. During obsolescence media struggles for relevance and is affected by external factors such as competition from new innovations. Rarely media ever wholly disappear as it declines from industrial (or commercial) obsolescence, with examples of media returning from the apparent ‘dead’ (e.g. vinyl records). Instead, media is often repurposed by creatives [2] so ‘secondary functions and practices © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 111–117, 2024. https://doi.org/10.1007/978-981-97-0621-1_14

112

M. Holmes and A. V. Reyes

can become primary ones again; new or discarded forms of use are (re)discovered and artistically explored’ [1]. Within these new creative spaces, complex and nuanced factors are in operation - such as aesthetics, materiality, heritage, and tradition. Digital design research keeps, however, moving towards more efficient and performance-driven spaces, often in disregards of efficacy - real world implementations of technology within design practices and cultures. Research ‘on-site’ is rare, let alone observational studies aiming at understanding the effects of technology introduction in creative environments [3]. This research sits within that adoption process, contextualised in the city of Plymouth, UK. Here we study the incorporation of digital fabrication technologies by the craft of letterpress printmaking, further emphasised by spatial and urban regeneration plans, new spaces (both literally and metaphorically) for creatives practitioners, and community participation and knowledge cocreation around ‘the obsolete’ (Fig. 1) as part of an urban-wide regeneration program.

Fig. 1. Printmaking event with first author presenting the digital fabrication of letterpress types. (Image: M. Holmes).

Letterpress practitioners have begun adopting new tools to address obsolescence and danger of disappearance such as lack of educational and manufacturing infrastructure [4]. As digital fabrication has become accessible in the form of ‘personal fabrication’, questions arise surrounding the aesthetics of new hybrid practices, the agency of practitioners beyond the ‘original’, and how alternative modes of knowledge cocreation of intangible cultural heritage may ‘reconfigure, relocate and recalibrate’ innovation and production [5] into more nuanced, complexified spaces of digital practice. This ongoing research follows a practice-based approach to investigate the adoption of digital fabrication within emerging hybrid letterpress printmaking. The paper outlines an initial theoretical framework bringing together strands of management, media, craft and architectural theory into a bricolage of methods and provocations [6] to be tested via a live ‘research lab’. This is expected to be a vehicle for ‘research through design’ on-site facilitating knowledge cocreation, spatial manifestations and community participation on the rediscovery of novel, hybrid, post-digital practices.

Digital Hybridities

113

1.1 Core Definitions Within this project, ‘digital fabrication’ refers to CAD/CAM tools such as 3D printers, milling machines, and laser cutters. For instance, CNC milling has allowed letterpress practitioners to manufacture wood type - both in new and old typeface designs. ‘Hyperlocal’ in this context will here be defined as the use of these tools operated in-situ by practitioners themselves, critically remaining in close connection to the craft. This digital/craft hybridity challenges the common binarism of setting the ‘old’ in conflict with the ‘new’. As Key observes, this presents difficulties when working in the ‘spaces between’, and highlights wider problems with boundary settings and inclusion [7]. Similarly, the notion of ‘craft’ is a debated concept that has shifted significantly over time this research acknowledges the difficulty to isolate a single definition. A common core of many descriptors is the utilisation of the ‘hand’ and the application of a high degree of skill at the point of production. Shiner [8] however believes that this reduction of craft purely to skill is limited, and instead advocates for the inclusion of creative processes, demonstrating ‘autonomy in a field of knowledge’ [9].

2 Theory Framework In comparison to production technologies, craft is a complex and nuanced field consisting of the interplays between agency, tradition, processes and outputs which when combined derived into a resulting ‘aesthetic’ [10]. Heritage and tradition both share a common normatively described core of protection, preservation and transmission of culturallyvalued knowledge from one generation to the next. However, Cardoso [11] observes that crafts can come to a ‘position of terminal nostalgia’, whereby processes, materiality and aesthetics become effectively ‘locked in’ to a particular or agreed standard [12, 13]. Furthermore, the over-romanticising of craft practices may lead to issues of ‘misremembered pasts’ [14], protecting certain aspects through ‘invented tradition’ described by Ranger and Hobsbawm [15] as ‘a set of practices (…) which seek to inculcate certain values and norms of behaviour by repetition.’ This fabricated historical grounding that forms invented traditions is what can lead to institutional perspectives over traditional practices. An example of this is the “Authorised Heritage Discourse” [16], a framework which acknowledges that the interactions with heritage cultural assets are controlled by appointed experts, with a viewpoint that is highly monumentally and aesthetically driven [17], a ‘constructed authenticity’ [18] that signifies the process through perceived wear, marks, scars and damage forming an ‘aura’ of craft aesthetics [19], with problematic implications for machine-made objects and their new, hybridised material aesthetics. Letterpress is a craft space with noteworthy examples of such conceptual tensions, with new and emerging forms of material practice that stem from lack of consumables’ availability in industry (Fig. 2). Being a practice intrinsically bound to specific physical tools (such as the moveable metal type) and spaces, the loss of these would mark a fundamental change in the process itself. Goossens [4] remarks that ‘if we are no longer limited by the physicality of type, but accept digital designs which can be imposed onto a plate (…) then points of continuity between then and now become so faint that the practice might be thought of as something else entirely.’ In that sense, Risatti [20] asserts that a tool is not a craft object in and of itself - it is the means by which the craft object

114

M. Holmes and A. V. Reyes

is created. However, this is not so cleanly cut for those crafts that are deemed traditional and ‘locked-in’- the object-centric factors should not be overlooked due to its importance in underpinning physical productive processes. While unlikely that there would ever be a return to industrial scale manufacture of wood type, digital fabrication processes could be a potential replacement on a highly local scale.

Fig. 2. Resin 3D printed type pieces. Image: M. Holmes.

Here we identify our research problem. These areas of practice are more nuanced than the new/old or digital/analogue dichotomies. Complex conceptual interrelations do require a new theoretical framework able to update research originating from these emerging, hyperlocalised processes of adoption. On considering issues of tradition and ‘locked in’ crafts in relation to digital design, we will detail two areas in which this approach influences new, hybrid practices: ‘knowledge cocreation’ and ‘sociomaterial cultures’. An emphasis on knowledge cocreation comes as a result of the loss of the traditional information pathways that were associated with craft practices. There is a declining number of HE institutions retaining trade pathways such as printers’ apprenticeships, and craft-orientated technical colleges are increasingly rare [4]. This, again, has a twofold impact on the letterpress craft: a dismantling of many hundreds of years of highly regulated (and often secretive) trade practice [9], but also the current generation of printers are the first to no longer have ready access to skilled master printers, technicians and academics who underwent formal training in the craft, and that valuable knowledge is at risk of being lost [21]. The second issue relates to the material and estate needs, supply chains and new cultures of practice originated (largely) due to scarcity of specialist resources; a socio-material [22] dynamic of obsolescence and collective resilience and survival. ‘Type Founding and Manufacture’ joined the Heritage Crafts Red List at the endangered level in 2021, highlighting that access to craft-specific equipment is recognised as in decline. By the late 20th century as commercial hand-set letterpress began to decline, machinery that was displaced from industry suddenly flooded the market, being sold off cheaply, given away, or scrapped. Creatives and artists could then acquire them [4] allowing Letterpress to enter the media obsolescence stage of retrieval and

Digital Hybridities

115

reinvention. The side effect however has been the increased demand for now limited consumable resources, creating problematic economic barriers [21]. This need aligns with the capabilities of digital fabrication to bring manufacturing within the reach of the practitioner and embedding within supply chains [23], with many unavailable tools able to be replicated or emulated by digital fabrication resources (Fig. 3).

Fig. 3. Pop-up letterpress workshop within the renovation of the Millenium building, including a sample of 3D printed version of the “Coolvetica” typeface. Source: M. Holmes.

Here, then, we establish a parallel with the design disciplines. Due to industry pressures, the roles of craft practitioners have fundamentally shifted and expanded, with multiple roles now amalgamated into multi-skilled individuals [21]. Disciplinary and skills boundaries are blurred, simultaneously increasing the openness and sharing with broader communities, while expanding the knowledge and material complexities involving in the production of artefacts. This emerging socio-cognitive intelligence is recalled by Risner [24] as a ‘networked type of digital craft practice’, particularly relevant when applied to a craft whose existence is under pressures of obsolescence. These changing ways of working expand the agency of the practitioner, moving closer to Frayling’s concept of a ‘designer-craftsman’ as roles of designer and craftsperson merge [10]. This is confirmed by Dixon [21] who notes the ‘possibilities for a maker-intelligence which combines the material and the conceptual. Such an approach to thinking-through-making or what A.Telier describes as ‘situated doing’ moves beyond production-centred models’ and challenges recently emerging aesthetic exercises engaging with craft from purely ornamental or material perspectives. The use of digital fabrication tools, then, allows an approach of a hybrid ‘design-asinquiry’ [25] - both in the form of a replication of the traditional, but also as a developmental tool for evolving new aesthetics and materiality - which posits an alternative perspective on already established practices. By embracing hybridity, the development of practice can expand in a more appropriate, practice-centred and humane ways able to leverage the potential of a novel cultural and socio-cognitive [26] ‘making intelligence’. This links to concepts of media repurposing and the reframing of the obsolete, incorporating Sennett’s concept of ‘domain shift’ whereby tools and principles from one area

116

M. Holmes and A. V. Reyes

of practice can be translated and applied to another [27], as well as emerging research frameworks such as distributed cognition [26].

3 Discussion By bringing together theoretical strands originated in craft theory, socio-materiality, media and heritage discourses, this short paper makes the case for revisiting the relationship between digital and craft practices beyond the old/new dichotomies and technologically-centred approaches, acknowledging more inclusive spaces ‘in between’ informed by a deeper understanding of aesthetics, traditions, knowledge cocreation and the social implications of emerging, hybrid spaces. However briefly, we pose the challenge of not only theorising those spaces but also researching them on-site, improving our understanding of the field by cocreating new knowledge ‘with’ practitioners as opposed to ‘about’ practitioners. To achieve this, the study will follow two complimentary strategies – one, to work with practice-in-place, embedding digital fabrication technologies with already established practitioners within their workshops - and the other to create a real-world laboratory space in order to explore the divergence of this digitally recreated craft divorced of constraints applied by discourses of heritage and tradition. We expect that, however, this theoretical position will keep evolving as further fieldwork activity is conducted. Instead of a one-size-fits-all conceptualisation, we pose this bricolage of theoretical strands as a live construction, likely to evolve and accommodate further knowledge as this research proceeds. Particularly we expect insights to emerge from two distinct areas of investigation. First, we identify ‘the field’ as not only the context from which data is gathered, but a more complex space of practices, cultures and communities involved in the survival, adaptation and reinvention of ‘the traditional’. In Plymouth this includes broader developments around the emergence of new creative economies and urban regeneration projects, and the fitness of design practices within local, place-based growth and industrial legacies. This is acknowledged in Plymouth’s 2021–2030 Culture Plan which identifies the city’s ‘ability to use digital technology to engage communities and build cross-sector connections’ [28], aligned with local narratives on social innovation and municipal statecraft [29]. Secondly, this study builds upon a stream of research calling for more nuanced, informed and critical approaches to the digital investigation of making and making intelligence. We expect this work to contribute to both design and craft studies, but more broadly to challenge techno-centric and techno-solutionist approaches to making in areas such as digital manufacturing and modern methods of construction.

References 1. Schrey, D.: Analogue nostalgia and the aesthetics of digital remediation. In: Niemeyer, K. (ed.) Media and Nostalgia, pp. 27–38. Palgrave Macmillan (2014) 2. Harman, G.: Heidegger, McLuhan and Schumacher on form and its aliens. Theory Cult. Soc. 33(6), 99–105 (2016) 3. Veliz Reyes, A.: Augmented pedagogies. Ph.D. Thesis. University of Liverpool (2016) 4. Archer-Parré, C., Mussell, J.: Letterpress Printing: Past, Present, Future. Peter Lang, Oxford (2023)

Digital Hybridities

117

5. Smith, A., Hielscher, S., Dickel, S., Soderberg, J., Van Oost, E.: Grassroots digital fabrication and makerspaces: reconfiguring, relocating and recalibrating innovation? In: Ciarli, T., Rotolo, D. (eds.) SPRU Working Paper Series. University of Sussex (2013) 6. Sanders, E., Stappers, P.: Probes, toolkits and prototypes: three approaches to making in codesigning. CoDesign 10(1), 5–14 (2014) 7. Key, J.: Readymade or handmade? J. Mod. Craft 5(2), 204–215 (1997) 8. Shiner, L.: “Blurred boundaries”? Rethinking the concept of craft and its relation to art and design. Philos Compass 7(4), 230–244 (2012) 9. Jury, D.: Letterpress: new applications for traditional skills. Rockport (2006) 10. Frayling, C.: On Craftsmanship: Towards a New Bauhaus. Oberon Books, London (2017) 11. Cardoso, R.: Craft versus design: moving beyond a tired dichotomy. In: Adamson, G. (ed.) The Craft Reader, pp. 321–332. Bloomsbury, London (2018) 12. Gibson, C.: Material inheritances: how place, materiality, and labor process underpin the path-dependent evolution of contemporary craft production. Econ. Geogr. 92(1), 61–86 (2016) 13. Martin, R., Sunley, P.: Path dependence and regional economic evolution. J. Econ. Geography 6(4), 395–437 (2006) 14. Boym, S.: The Off-Modern. Bloomsbury, New York (2017) 15. Hobsbawm, E., Ranger, T.: The invention of tradition. Canto. Canto ed., 18. print ed., p. 320. Cambridge University Press, Cambridge (2010) 16. Smith, L.: Uses of Heritage. Routledge, London (2006) 17. Smith, L., Waterton, E.: Constrained by commonsense: the authorized heritage discourse in contemporary debates. In: The Oxford Handbook of Public Archaeology, pp. 153–171. Oxford University Press, Oxford (2012) 18. Flynn, D.: Laser cutting with character. Blue Notebook 3(1), 38–45 (2008) 19. Benjamin, W.: The work of art in the age of mechanical reproduction. In: Watson, S., Barnes, A.J., Bunning, K. (eds.) A Museum Studies Approach to Heritage, pp. 226–243. Routledge, London (2018) 20. Risatti, H.: A theory of craft: function and aesthetic expression. University of North Carolina Press (2013) 21. Amado, P.M.R., Silva, A.C., Quelhas, V.: Post-digital Letterpress Printing: Research, Education and Practice. Routledge, New York (2022) 22. Orlikowski, W.: Material knowing: the scaffolding of human knowledgeability. Eur. J. Inf. Syst. 15(5), 460–466 (2006) 23. Birtchnell, T., Urry, J.: Fabricating futures and the movement of objects. Mobilities 8(3), 388–405 (2013) 24. Risner, I.: The integration of digital technologies into designer maker practice: a study of access, attitudes and implications. Ph.D. thesis. University of the Arts London (2013) 25. Rosner, D.: Approaching design as inquiry: magic, myth and metaphor in digital fabrication. In: Sayers, J. (ed.) The Routledge Companion to Media Studies and Digital Humanities, pp. 511–520. Routledge, New York (2018) 26. Hutchins, E.: Cognition in the Wild. MIT Press, Cambridge (1995) 27. Sennett, R.: The Craftsman. Penguin Books, London (2009) 28. Plymouth City Council. Culture Report 2021-30 (Appendix A: Briefing Report) (2021) 29. Thompson, M.: Whatever happened to municipal radicalism? Trans. Inst. Br. Geogr. 48, 603–618 (2023)

Exploring an Evolving Architectural Pedagogy in the Age of Digital Creativity and Artificial Intelligence: Examining the Challenges to Critical Thinking John Latto(B) Xi’An Jiaotong-Liverpool University, Suzhou, China [email protected]

Abstract. The advent of digital creativity and artificial intelligence (AI) is rapidly changing the way we think, design and construct architecture. This transformative methodology, must also inevitably lead to a continued evolution in architectural pedagogy. Whilst these new technologies, computational tools and data-driven processes suggest new possibilities for creativity and innovation, there is an inherent perception they may also pose a threat to the development of the traditional design skills within architectural education. This paper seeks to explore the impacts of new ways of design and the importance of retaining creative critical thinking processes in design education. Keywords: Architectural pedagogy · design thinking · design methodology · digital tools

1 Introduction In 1954, in a letter between Charles Eames and Ian McCallum of the Architectural Review (London), Eames wrote; “The buildings and communities of the near future will be planned with the aid of some development of these theories (new technologies). Whether or not they are planned by architects may pretty well depend on the way architects today prepare to use such tools” [1]. This in many ways was indeed an astute prediction from Eames and certainly reflects the current rapid advancements and accessibility of machine learning and generative modelling and its cautious acceptance by society and ethical adoption by the design professions. With the such advancements of information and digital technologies, it is clear the availability of such technologies has profoundly influenced the way that projects are conceived, designed, delivered and constructed and in turn tremendously affects architectural practice and the education of students. For our students to be in a position to meet these future challenges, education must demonstrate agility and evolve. Onosahwo Ivendo and Alibaba [2] call for all educators to grow with these different technological tools and programs, to provide students with necessary digital awareness. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 118–127, 2024. https://doi.org/10.1007/978-981-97-0621-1_15

Exploring an Evolving Architectural Pedagogy

119

Having personally entered the architectural profession in the late 1970’s, at a time of drawing boards and T squares and subsequently graduating in the early 1980’s from an Architecture School in the UK which had only one, low performing computer, running rudimentary CAD software, there has been an observed shift and extensive development in the process of creating and delivering architecture. Witnessing the second digital wave in the 1990’s [3] and recalling the day where staff gathered around the only iMac computer in the practice, with the connection to this new phenomenon called the ‘world wide web’, we could not have envisaged the radical changes this would bring. Change however, brings many opportunities, but also brings fears, along with challenges to the way we think, operate and progress, as a profession and as a society. The way in which we respond and adapt to these opportunities and challenges, can optimize the processes of creation. Now faced, through the ever-increasing accessibility to machine learning and ‘new’ perceived threats and opportunities in design and making, this paper seeks to explore the potential impacts on the architectural curriculum and pedagogical philosophies and how these must inevitably develop in the new digital. It explores the background context of architectural education, exploring the teaching of core skills, such as creative and critical thinking [4, 5] and the integration and impacts of digital design methodologies and machine learning on architectural pedagogy. To address these concerns, a number of strategies are put forward. Firstly, suggesting the embedding of core design thinking skills with human-centered design philosophies, to encourage students to reflect on the ethical, functional and social implications of their designs. Secondly, proposing the integration of design methodologies, both analog and digital and design critique methodologies, which encourage divergent thinking, iterative design processes, and constructive criticism. Finally, the paper considers through several examples of new creative digital approaches and their positive applications in collaborative multi-disciplinary projects, which may be further extended into education.

2 Research Methodology In order to investigate the intersection of architecture pedagogy, digital creativity, and critical thinking, a comprehensive literature review was conducted. The research methodology for this paper primarily rely on a keyword search approach to identify relevant papers and publications. The criteria for analysis was based on the inclusion of keywords such as “architecture pedagogy,” “digital creativity,” and “critical thinking” within the title, abstract, or keywords of the selected papers. To ensure the currency and relevance of the literature, the principle range for papers and publications considered in this study will be from the year 2000 to the present. This timeframe allows for the inclusion of recent advancements in digital technologies and their impact on architectural education, while also considering the evolution of pedagogical approaches and critical thinking in the field. However, some earlier relevant publications were reviewed where they provided interesting points worthy of consideration and further examination. The literature review was conducted using various academic databases, such as Google Scholar, JSTOR, and Scopus, to ensure a comprehensive coverage of relevant scholarly articles, conference papers, and book chapters. The search strategy involved

120

J. Latto

combining the keywords mentioned above using Boolean operators (e.g., “architecture pedagogy” AND “digital creativity” AND “critical thinking”) to refine the search results and narrow down the focus of the study. The selected papers and publications were then critically analyzed and synthesized to identify common themes, trends, and insights related to study intentions of this paper. The analysis involved examining the theoretical frameworks, methodologies, case studies, and empirical evidence presented in the literature to gain a deeper understanding of the topic. Additionally, the research methodology considered the quality and credibility of the selected papers and publications. Peer-reviewed articles and publications from reputable academic journals and conferences were given greater priority, to ensure the reliability and validity of the findings.

3 Background to the Pedagogical Debate The most important experience of the architectural education is the design studio. It is in this educational culture where students acquire vital knowledge and skills, and develop their creativity. Through experimentation with problems simulating real-life practice, whilst gaining experience in the integration of theoretical with practical aspects of the architectural profession [6]. “It is the “signature pedagogy” of the profession, dominating the preparation of future architects encouraging them to think (like an architect), perform (like an architect) and to act with integrity” [7] Thomas Dutton further describes studios as ‘active sites’ where students are engaged intellectually and socially, shifting between analytic, synthetic and evaluative modes of thinking in different sets of activities [4]. The architecture studio in education, still largely follows the model originating from the Ecole des Beaux-Arts from the 19th Century, which is questioned by many as being a illfitting model in contemporary tertiary education [8]. However, the core skills mentioned above remain a key aspect of a student’s architectural design education. For some, intellectually, The Bauhaus stands for what is essential for Modernism, and is still held as a rigorous model for the present day. However, like Beaux-Art, the model was developed between 1919–1933 and may be considered unrepresentative of current socio-economic and cultural issues; The Bauhaus was a school of thought developed within a set of conditions, cultural, intellectual, or otherwise, specific to its own time. Anay and Ozten [9] go on to argue that digital design, one of the recent and most powerful challenges to Bauhaus pedagogy is perhaps the best illustrative case of this situation: From a certain perspective, digital design, by definition, is incompatible with the Bauhaus pedagogy. Yet from another perspective, which is shared by many, it is assumed that digital design yet another paradigm shift; this time, from Bauhaus to what Anay and Ozten refer to as “something else.” Anay and Ozten, explore what is left of the Bauhaus and its relevance to a current pedagogical model. They suggest that what we refer to as digital design today is a phenomena which is neither a design model nor a pedagogy, but rather a strong influence on the conventional models of design; coming with a set of demands towards a change in the production of architecture, in the architectural design thinking, and consequently, but more important for the present case, in the education of an architect. At the time of conception, The Bauhaus was a

Exploring an Evolving Architectural Pedagogy

121

paradigm shift in the education of an architect, formulated as an answer to the changing conditions of the Modern era, now at the beginning of “the third digital age” [3] we are witnessing a similar change, providing a new set of conditions and demanding a consequent transformation.

4 An Argument for Creativity and Critical Thinking As seen in the previous section, creativity remains a core learning attribute related to design problem-solving through the studio culture, This is because most design problems are ill-defined and therefore cannot be completely solved using routine problem-solving processes, either manually or computationally [10] Gero describes design as not only as synthesis of solutions, even optimal solutions, but also in the novel and unexpected solution which as a consequence of its existence changes our expectations. This may require a change in structure, behaviour of function, what Gero describes as ‘the essence of creative design.’ Creative thinking can aid to enrich and develop the abilities needed to tackle ill-defined processes, and generate exceptional solutions [11, 12]. These essential critical thinking abilities of the architect, are a complex multi-level set of cognitive and meta-cognitive actions performed consciously in the course of architectural design activity and aimed at creating high-quality original architectural solutions. A review and analysis by Tarasova [13] of various theoretical concepts of architectural thinking, identified three levels of critical: logical-psychological, meta-cognitive and philosophical methodological, which enable its specificity to be determined. She observed that the specificity of critical thinking consists in a multi-layered pattern of actions on operational, regulatory and attitudinal levels. The results of this study and further theoretical research into architect’s critical thinking make it possible to raise it to the level pedagogical practice and develop a methodology for essential inclusion of critical thinking techniques into architecture education programs. Critical thinking occupies a key position in the general structure of architectural professional thinking and must be embedded in architectural curriculum. The fears of the new technologies and the potential for de-humanising the design process through the introduction of machine learning and generative artificial intelligence, are widely published through media, with some headlines prompting this as ‘the end of humanity’ and ‘can architects survive?’ [14, 15] The Time Magazine reported 100 artificial intelligence professors and leading experts signed a one sentence statement “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war” [16]. This virtual hysteria questions the potential challenge to the human race, but broadly within the architectural profession there is a hesitant caution to its acceptance, with it being seen to augment and not replace the designer. An acceptance that AI is here to stay and to seek interventions to embrace and manage the risks.

5 A Case for Integrated Strategies for Architectural Pedagogy It can be argued that most dominant model of teaching in design studio involves a design process which commences manually through sketch and physical modelling. The process generally follows the linear pathway described in Fig. 1. [17] This analog

122

J. Latto

approach can provide a natural and free level of expression unhindered by the varying levels of digital literacy and individual students limitations in managing the complexity of the digital tools. There is an assumption that students are of a digital generation and as such are digital natives, whilst they may demonstrate software operational abilities, the vast majority do not have a deep understanding of computing or digital media principles [18] As a result, their work tends to be inefficient and derivative. The analog process is also however limited to the consciousness of the individual student. These initial ideas are often then transferred to digital, to further explore, optimize and refine the model.

Fig. 1. Design process of architectural practice - Model 2 (Rahbarianyazd-Nia-2019a).

Lescop and Chamel [19] argue that the role of digital tools is a question mostly avoided in pedagogical debates and that in order for this to evolve, we should no longer paradoxically take digital tools as the ‘entry point for reflections that we can transfer to pedagogy’, but educators should take them instead as one of the indicators, or one of drivers of the pedagogy. Their study explored and compared the respective curricula of Ecole Nationale Supérieure d’Architecture in Nantes, France and the FAMU in Tallahassee, USA and particularly the modes of operation which they describe as “ legacy “ which makes no particular distinction between an analog or digital practice and tends to favour analog tools. The other, which they call “emergence” being based on the almost exclusive use of digital tools, by proposing a cybernetic thinking methodology. Cameron Campbell in his paper entitled Digital Design Pedagogy [20] observes that “when digital methodologies are brought into the curriculum, digital skills become an addition to an already formed methodology. He argues that it is valid to have a contrast between mediums, but that it is necessary to provide a digital foundation prior to developing the opposition of digital versus analog.” He goes on to state that students become better informed participants in understanding the differences, benefits, and liabilities of the different mediums (approaches) The intention is to create a digital pedagogy removing the “opposition” with analog, by providing a digital design foundation in parallel to

Exploring an Evolving Architectural Pedagogy

123

analog. Results reporting the impacts and success of the program are yet to be collated, however the immediate benefits were reported through very positive student feedback. Oxman describes digital design in the Theory and Design in the First Machine Age [21] as a unique phenomenon - a new form of design rather than merely than a conventional design achieved through new media. She continues that if true, then perhaps a reevaluation of design theories and methodologies leading to a new framework of thought. Previous theories were focused on analysis and formal modelling of the procedural, behavioral, and cognitive activities of design as described by Professor Bryan Lawson in his publication ‘How Designers Think’ [22]. Oxman further develops her research [23] observing that a new world view develops conceptual structures for design, that may contradict the prevailing logic of design thinking. Rather than the employment of digital technologies, it is these emerging conceptual structures that strongly influence the logic of architecture and its design methods. Oxman goes onto explain that these Conceptual changes become the content of new pedagogical methods of design education. “The awareness of change and conflicts can stimulate the necessary theorization and conceptualization for new approaches to design didactics. Design thinking precedes design learning. The evolution of design thinking now appears to have generated a new paradigm for design. As this paradigm crystallizes we first encounter it as a series of conceptual conflicts between the prevailing and the new values of two design ontologies. New pedagogies can operate within this condition of the evolution and instability of ontologies. However, it can do so only by directly articulating and working with conceptual structures as pedagogical material. While digital design skills are critical for designers, architectural education must also recognize and deliver more than technical proficiency. Working creatively and effectively with computers, digital fabrication machines, and other devices requires a new set of workflows and adaptations to academic and professional behaviours. Boyer’s report [24] makes it clear that one of the key values of an architectural education is developing learning habits. A present gap in student learning is that traditional learning habits have not been updated in response to changes in technology [25] Learning objectives and soft skills for digital design can help to bridge these gaps, supporting the goal of not only working well with technology, but collaboratively.

6 Precedent Studies 6.1 Optimization-Aided Design: Two Approaches for Reflective Exploration of Design Search Space. Wang, L (2022) [26] This study supports performance-based design optimization in early-stage design development so as to mitigate against poor design outcomes and explores two approaches adopting the design tool ‘Evo-Mass’ and facilitates the designers reflection, exploration and decision making. The plug-in tool operates within Rhino-Grasshopper and provides designers with a coding free environment to generate many alternatives to explore building mass. The optimization process allows designers to efficiently adjust the design generation and/or parameters, providing prompt feedback and allowing decisions. In the

124

J. Latto

two approaches explored, the first focuses on comparative building designs with varying characteristics whilst the second considers is a convergent, iterative reflection and modification, narrowing desirable design spaces. The research shows that such optimization can play an important role in supporting designers in loosely defined design phases, to better understand the performance criteria and identify design weaknesses at an early stage and make remedial changes. The study does not however identify how designers critically synthesis the data produced, nevertheless this does illustrate the supportive integration between digital and analog design processes. 6.2 Form Finding and Evaluating Through Machine Learning: The Prediction of Personal Design Preference in Polyhedral Structures Zheng, H (2021) [27] 3DGS is a powerful and convenient method for the architect to find a form in structural design. By subdividing the polyhedral geometries in the force diagrams, a variety of forms can be generated, each of which is quite different and the set of parameters to express a form is unique and distinguishable from each other. Thus, the neural network can also learn the design preference from a specific architect, by learning the result of a preference test taken by the architect. The machine will assist the design process not only in simple repeated work but also in creative work by learning the design examples from human beings. 6.3 AI and Architecture: An Experimental Perspective. Chaillou, S (2021) [28] Chaillou applies AI to floor plan generation and analysis. The approach is threefold: (I) to generate floor plans using generative adversarial network models, i.e., optimize the generation of a large and highly diverse quantity of floor plan designs, (II) to qualify floor plans, i.e., offer a proper classification methodology built on reliable metrics (footprint, orientation, thickness and texture, program, connectivity, and circulation), and (III) to allow users to “browse” through generated design options. Chaillou argues, the machine, once the extension of our pencil, can today be leveraged to map architectural knowledge and trained to assist us in creating viable design options.

7 Conclusions In conclusion, the integration of creative digital design and generative machine learning into architectural pedagogy holds immense potential for enhancing the learning experience, fostering innovation in the field and preparing students for industry. This paper has explored the challenges and benefits of incorporating these technologies, such as increased design exploration, improved efficiency, and enhanced collaboration. However, it is crucial to recognize that while these design methodologies offer valuable support, they should not replace the development of individual critical and divergent thinking skills. As argued by Schön in the 1982 publication “The Reflective Practitioner” [29] the ability to think critically and divergently is fundamental to architectural education, as

Exploring an Evolving Architectural Pedagogy

125

it enables students to question assumptions, challenge conventions, and develop unique design solutions. Therefore, it is essential to strike a balance between utilizing digital tools and nurturing these cognitive abilities. By integrating creative digital design and generative machine learning into architectural pedagogy, educators can provide students with a rich learning environment that encourages both technical proficiency and critical thinking. To retain individual critical and divergent thinking skills, it is necessary to incorporate activities that promote reflection, analysis, and conceptualization alongside the use of digital tools. For instance, encouraging design critiques, research-based assignments, and theoretical discussions can help students develop their own design philosophies and engage in meaningful dialogue with their peers. Additionally, encouraging students to explore alternative design approaches and experiment with unconventional ideas can foster their ability to think divergently. Furthermore, it is important to acknowledge that the integration of creative digital design and generative machine learning in architectural pedagogy requires continuous evaluation and adaptation. As technology evolves rapidly, educators must remain agile and updated with the latest advancements and ensure that their teaching methods align with the changing landscape. Regular feedback from students and industry professionals can provide valuable insights into the effectiveness of these integrations and help refine the pedagogical approach. This consultation with industry will also ensure students can be suitably skilled to enter practice. Finally, the integration of creative digital design and generative machine learning into architectural pedagogy offers numerous benefits, including increased design exploration and improved efficiency. However, it is crucial to retain individual critical and divergent thinking skills by incorporating activities that promote reflection, analysis, and conceptualization. By striking a balance between utilizing digital tools and nurturing cognitive abilities, educators can create a dynamic learning environment that prepares students for the challenges of the architectural profession. Bob Shiel, Director Bartlett School of Architecture, London, states in the book entitled Educating Architects; “A new relationship between teaching, research, industry and practice is emerging, one that is adaptive and collaborative, non-linear and mutually dependent. Underpinning its future is the digital generation, unleashed and not like anything we have seen before” [30].

8 Future Research Potentials The strategies suggested within this paper have necessary further testing and evidence collection, to understand more fully their potential and effectiveness as a learning model and to dispel views of an erosion in the individuals ability for creative critical thinking. • To further assess the influence of digital design methodologies on students creativity, performance and achievements. • To further assess more defined strategies for the use of AI in student design studio projects.

126

J. Latto

References 1. Kirkham, P.: Charles and Ray Eames: Designers of the Twentieth Century, 1st edn. MIT Press, Cambridge (1995) 2. Ivendo, T.O., Alibaba, H.Z.: Computer aided design (CAD) technology versus students’ learning in architectural design pedagogy – a controversial topic review. Int. J. Dev. Res. 05(01), 3152–3158 (2015) 3. Jandric, P.: Digital: the three ages of digital. In: Design Studies, Chapter in Keywords in Radical Philosophy and Education: Common Concepts for Contemporary Movements, pp. 161–176. Brill, Boston (2019) 4. Dutton, T.: Design and studio pedagogy. J. Archit. Educ. 41(1), 16–25 (1987) 5. Dutton, T.: Voices in Architectural Education: Cultural Politics and Pedagogy. Praeger (1991) 6. Crowther, P.: Understanding the signature pedagogy of the design studio and the opportunities for its technological enhancement. J. Learn. Des. 6, 18–28 (2013) 7. Shulman, L.S.: Signature pedagogies in the professions, vol. 134, no. 3, On Professions & Professionals (Summer, 2005), pp. 52–59 (2005) 8. Madonovic, M.: Persisting beaux-arts practices in architectural education: history and theory teaching at Auckland school of architecture, 1927–1969. Interstices J. Archit. Relat. Arts 9–24 (2018) 9. Anay, H., Ozten, U.: The legacy of Bauhaus pedagogy in digital design. Online J. Art Des. 7(3), 139–145 (2019) 10. Gero, J.S.: Creativity, emergence and evolution in design. Knowl.-Based Syst. 9, 435–448 (1996) 11. Dorst, K., Cross, N.: Creativity in the design process: co-evolution of problem–solution. Des. Stud. 22, 425–437 (2001) 12. Cross, N.: Expertise in design an overview. Des. Stud. 25, 427–441 (2004) 13. Tarasova, I.: Critical thinking for architects. In: IOP Conference on Series: Materials Science and Engineering, vol. 463, p. 042046 (2018) 14. Grace, K.: AI is not an Arms Race, Time Magazine (2023). https://time.com/6283609/artifi cial-intelligence-race-existential-threat/. Accessed 14 Aug 2023 15. Wainwright, O.: It’s already way beyond what humans can do: will AI wipe out architects? (2023). https://www.theguardian.com/artanddesign/2023/aug/07/ai-architects-revolu tionising-corbusier-architecture. Accessed 14 Aug 2023 16. Brauner, J., Chan, A.: AI poses Doomsday Risks (2023). https://time.com/6303127/ai-futuredanger-present-harms/. Accessed 14 Aug 2023 17. Rahbarianyazd, R., Nia, H.A.: Aesthetic cognition in architectural education: a methodological approach to develop learning process in design studios. Int. J. Cogn. Res. Sci. Eng. Educ. (IJCRSEE) 7, 61–69 (2019) 18. Senske, N.: Confronting the challenges of computational design instruction. In: Computer Aided Architectural Design and Research in Asia (CAADRIA) Conference, Kyoto, Japan, pp. 821–829 (2014) 19. Lescop, L., Chamel, O.: How digital tools are changing architecture education. In: Conference Design Communication Association DCA2021: “DCA20/20: Perception to Execution”, The Design Communication Association (DCA), Atlanta, USA, pp. 67–72, hal-03523003 (2021) 20. Campbell, C.: Digital Design Pedagogy, ACADIA 2006: Synthetic Landscapes Digital Exchange (2006) 21. Oxman, R.: Theory and design in the first machine age. Des. Stud. 27, 229–265 (2006) 22. Lawson, B.: How Designers Think. Architectural Press, London (1997) 23. Oxman, R.: Digital design thinking: in the new design is the new pedagogy. In: Proceedings of the 11th International Conference on Computer Aided Architectural Design Research in Asia Kumamoto (Japan), pp. 37–46 (2006)

Exploring an Evolving Architectural Pedagogy

127

24. Boyer, E.L., Mitgang, L.D.: Building Community: A New Future for Architecture Education and Practice: A Special Report. Jossey-Bass Inc. (Preface xvi) (1996) 25. Doyle, S.: Between design and digital: bridging the gaps in architectural education. In: Architecture Conference Proceedings and Presentations, p. 83 (2016). https://lib.dr.iastate.edu/ arch_conf/83 26. Wang, L.: Optimization-aided design: two approaches for reflective exploration of design search space (2022) 27. Zheng, H.: Form finding and evaluating through machine learning: the prediction of personal design preference in polyhedral structures. In: Yuan, P.F., Xie, Y.M., Yao, J., Yan, C. (eds.) CDRF 2019, pp. 169–178. Springer, Singapore (2020). https://doi.org/10.1007/978-981-138153-9_15 28. Chaillou, S.: AI and architecture: an experimental perspective. In: The Routledge Companion to Artificial Intelligence in Architecture, 1st edn., pp. 420–441. Routledge (2021) 29. Schon, D.A.: The Reflective Practitioner. Ashgate Publishing, England (1991) 30. Schiel, B.: The digital generation. In: The Book Educating Architects: How Tomorrow’s Practitioners Will Learn Today, pp. 138–145. Thames and Hudson, London (2014)

Exploring 3D Concrete Printing of Lattice Structures on Robotically-Shaped Sand Formwork for Circular Futures Cristina Nan(B)

and Alessio Vigorito

Eindhoven University of Technology, Eindhoven, The Netherlands [email protected] Abstract. This research bypasses the use of wasteful scaffolding and moulds, by exploring 3D concrete printing (3DCP) on reusable sand-based substructures. Non-standard, complex geometries generally require the use of formwork to be produced, leading to wasteful, material-intense manufacturing processes. The research explores optimised material depositing strategies for 3DCP on robotically shaped sand formwork. The need for timber scaffolding, plastic or foam-based moulding for casting concrete is avoided. The fabricated structures are latticebased lenses with varying curve densities, envisioned as screens part of facade systems and spatial dividers. The research addresses the need for circular construction, reduction of raw-material usage for concrete formwork and the development of optimised material depositing strategies to undercut concrete usage. The deployed method falls under the category of sub-additive concrete printing. Firstly, the sand-based formwork for printing is shaped robotically through the use of different customised end-effectors. The robotic sand-forming of the substructure can be repeated multiple times, as almost no material loss is encountered. Subsequently, the robot arm is used for 3DCP along a three-dimensional tool path on the robotically shaped sand formwork. Parameters such as extrusion flow, printing speed and angle were tested and optimised to guarantee high-precision printing and resolution. Keywords: circularity · sub-additive manufacturing · 3D concrete printing · sand forming · robotic fabrication

1 Introduction The construction sector is facing a crisis due to a lack of implementation of sustainable, circular building practices. Companies which dominate this sector move slowly in terms of developing, adapting and implementing innovative construction processes and new materials. The construction industry and architecture play an important role in the environmental equation. The “2019 Global Status Report for Buildings and Construction”, coordinated by the UN Environment Programme, states: “The buildings and construction sector accounted for 36% of final energy use and 39% of energy and process-related carbon dioxide (CO2) emissions in 2018, 11% of which resulted from manufacturing building materials and products such as steel, cement and glass.” (Global Status Report for Buildings and Construction 2019 – Analysis, n.d.) © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 128–135, 2024. https://doi.org/10.1007/978-981-97-0621-1_16

Exploring 3D Concrete Printing of Lattice Structures

129

Concrete is worldwide the most common construction material, its production requiring high inputs of energy, water and embodied energy, particularly due to the use of sand and cement. Additionally, the production of concrete elements and structures involves the use of scaffolding, formwork or moulds. Depending on the complexity of the geometry to be fabricated, the required moulds and scaffolding systems can add to the production of waste, increasing the CO2 footprint even further. Therefore, the efficient and optimised use of concrete through new production methods, such as additive manufacturing but also including considerations relating to used scaffolding and formwork, significantly reduces material inputs, energy and water. Considering the construction sector’s size, the global climate emergency, energy and material crisis, it is imperative to develop innovative digital fabrication strategies which reduce material usage, minimise waste and deploy support systems which can be fully reused.

2 State of the Art Formwork represents the support structure put in place to cast concrete in its fluid state. Standardized formwork systems are predominantly based on modular mass-produced flat panels, which allow for the fabrication of rectilinear elements (Jipa and Dillenburger, 2022). Yet, the largest cost associated with the production of concrete structures and components relates to the formwork (Antony et al. 2014). Therefore, it is highly relevant to present the industry with sustainable yet low-cost options for the fabrication of formwork. Its cost reduction can be achieved by employing low-maintenance, reusable materials. The more complex the concrete geometry to be fabricated is, the more likely it is that the costs of the formwork will increase. Additive manufacturing expands significantly the geometric freedom for concrete allowing for complex forms and can be executed without the need of formwork or substructures. This first approach is defined as direct digital fabrication of concrete (DDFC) (Jipa and Dillenburger, 2022). 3D concrete printing (3DCP), initially coined as under the name of Contour Crafting, is based on a layer-by-layer fabrication logic, which allows for a faster, automated production of complex geometries and for an optimized material use (Khoshnevis, 2004). The most common method for 3DCP is the material extrusion. Departing from a vertical orientation, concrete is extruded in consecutive layers based on the slicing strategy, building up vertically. Although 3DCP based on material extrusion allows for the exploration of non-standard geometries, it still faces specific restrictions. Challenges are the maximum inclination and angle of the overall geometry, overhangs, steep angles, voids within the geometry and achievable height due to deformation under self-weight (R. A. Buswell et al. 2018) (Jipa and Dillenburger, 2022). A second approach to additive manufacturing with concrete includes as well the digital fabrication of formworks (DFF), which allows for a significant expansion of the geometric nonstandard vocabulary of forms (Jipa and Dillenburger, 2022). DFF is divided into three categories: layered extrusion, slip forming, and binder jetting (Wangler et al. 2016) (Jipa and Dillenburger, 2022). This categorisation system omits robotic shaping or subtractive fabrication. In this subtractive fabrication scenario, common materials used for the formwork for 3D concrete printing are foam or aggregate, sand. Other than aggregate or sand, robotic foam milling has the disadvantage that the foam mould is not reusable

130

C. Nan and A. Vigorito

for new sets of geometries. Aggregate and sand allow for an endless reshaping and can be considered nearly zero-waste. The work developed at Cornell University investigates sub-additive 3D printing of optimized double curved lattice structures where gravel is used as a substructure for the production of large-scale concrete arches. Sub-additive concrete printing based on two steps. Firstly, the support material (aggregate) is shaped, followed by a second step of 3D concrete printing on the aggregate-based formwork (Zivkovic and Battaglia, 2018). Research conducted at the University of Stuttgart investigates fully recyclable formwork systems made of water-soluble sand for spatial lattice structures (Kovaleva et al. 2022). At Iowa State University research is undertaken with water-soluble formwork made of PVA and PLA with ground steel reinforcement (Doyle and Hunt, 2019). These approaches not only reduce waste in the production process of concrete components, but allow for the fabrication of complex geometries characterised by larger inclinations, the occurrence of large groups of voids, overhangs to name just a few.

3 Research Setup The focus of this research is set on (1) the use of sand as a fully recyclable formwork, (2) the computational development and fabrication of lattice-based screen components for subsequent interior or exterior use by (3) exploring the ornamental qualities of concrete. The robotic facility of the company Vertico served as the fabrication setup. The described experiments were undertaken with the robot model IRB6640_130–320 placed on a single carriage track IRBT6004 with a travel length of 470 cm. The concrete pump operated with a frequency of 25 Hz and a water flow of 340l/h. The printing experiments used Vertico’s 3DP mortar at a water cement ratio of 0.4 with a variable printing speed of 100 to 250 mm/s. All prototyping tests were executed within the boundaries of timber boxes with interior dimensions of 20 × 75 × 115 cm. The next sections detail the robotic workflows of both subtractive (sand) and additive (concrete) manufacturing, tool path and material depositing strategy. 3.1 Sand Forming—Subtractive Manufacturing The research on the robotic sand shaping was conducted in several phases in order to gain an understanding and expertise of the relationship between the geometry of the end effectors, the robot’s movement speed, the tool path, the resulting surface quality of the formwork and the material behaviour of sand. Initial trials revolved around testing the moulding capacity of dry sand when being robotically shaped as well as the fabrication of different end effectors. The aims of this initial experiments are to fabricate sand formwork with both single and double curvature. For this purpose, three timber-based end effectors were developed and assessed. Early testing showcased a high surface quality of the sand formwork, with smooth transitions and the ability of the untreated sand to maintain its shape after the robotic subtractive manufacturing of the formwork. In subsequent experiments water was sprayed onto the sand prior to the robotic sand shaping. No improvement of the resulting surface quality or resolution of the sand formwork was noticed. All further experiments relating to the sand formwork were conducted solely

Exploring 3D Concrete Printing of Lattice Structures

131

with sand subject a relative humidity of 45–55% at room temperature. For all set-ups with the end-effectors EE1 and EE2 the robot’s movement speed robot was of 150 mm/s with manual adjustments during the process depending on the targeted resolution of the sand shape and the built-up resistance of the sand. The first end effector (EE1) with a straight profile was used for various paths: (1) linear, (2) linear inclined testing various sloping angles, (3) following a triangulated and (4) stacked oval path. These experiments offered an indication as to how the sand is compressed during the shaping process, the resistance that builds up and the required strength of the end effector to withstand it without breaking or bending as well as the obtained resolution of the sand moulds. All experiments resulted in a high surface resolution and quality of the sand formwork. Depending on the geometry, the built-up of displaced sand can cause high pressure on the end effector. This was the case for toolpath 3. In order to avoid an overstressing of the end-effector which can lead to bending or even breaking, the boundaries of the fabrication area have to be extended to account for sufficient space for the spreading of the subtracted sand. This space for the accumulation of the displaced or removed sand has to be factored when establishing the work area for the robotic fabrication process, as it may be of significant size, depending on the formwork geometry. Subsequently, EE1 was deployed for the fabrication of the lens-like sand mould based on half of an ellipsoid with its minor axis of 500 mm and its major axis of 700 mm. In the first iterations, the oval shaped was achieved by following an elliptic curve for the path, at a constant angle. Due to its shape EE1 is moving a large volume of sand at once. This leads to an approximation of a half ellipsoid, not rendering it possible for a refined machining of the sand. Simultaneously, due to its geometry EE1 cannot follow accurately the change in curvature of the ellipsoid toolpath. Whereas EE1 was used to incrementally shape an elliptical geometry, the second end effector (EE2) was developed to test a one-movement shaping strategy for a double curved sand formwork. In order to achieve this, EE2 is based on an arc used to follow a curve-based path and thus generating hyperbolic plane geometries as sand moulds. Based on the above listed findings, a redesign of the end-effector followed (EE3). In the tests conducted with EE1, the Z-axis of the tool centre point (TCP) was parallel to the vector extending from the sixth axis of the robot arm. With the redesign of EE3, the length of the shaping profile is significantly reduced and at an angle. The shortened shaping edge allows EE3 to follow the change in curvature of the toolpath and expands the flexibility of geometric changes. 3.2 3D Concrete Printing—Additive Manufacturing Geometry. One of the objectives of this research is set on exploring the sculptural and spatial potential of additive manufacturing with concrete for building facades and skins. To optimise the printing process and to avoid intersecting layers which generate a lowquality resolution of the print and are often characterised by lateral material overflow, the decision was made to opt for a computational design strategy based on a continuous, non-intersecting curve. As such the computational design of the facade prototypes is based on a differential growth algorithm used to generate a continuous curve within the bounding box of the base surface. Based on this workflow different lattice-like patterns

132

C. Nan and A. Vigorito

were generated, with varying densities and openings in order to examine the ornamental qualities of the printed prototypes and to investigate their sculptural properties. Robotic Toolpath. All printing tests were conducted without the use of accelerators in the cement mix. To simulate on a small scale a mass-customized production line but also to avoid the reach of rotational limits for two of the robotic joints, we opted for a configuration where the robot arm moves along the track dynamically according to the TCP position. For the initial printing sessions, single layer printing is tested with the extrusion nozzle in a locked perpendicular angle to the ground plane. In the first phase, multiple printing tests showcase that the extruded concrete layers do not adhere to the sand formwork on steep angles, particularly surrounding the borderline of the ellipsoid, causing lateral sliding. The cylindrical layer cross section offers a minimal contact surface with the sand. To circumvent this problem, the printing height is calibrated in relation to the inclination of the sand formwork. This ensures that the concrete is pushed into the sand, generating a rectangular cross section of the extruded layer. Thus, sufficient contact surface between the concrete and sand is provided, eliminating sliding and subsequent deformation of the 3DCP geometry. Printing tests show that best printing results are achieved when the extrusion nozzle is perpendicular to the sand surface itself, both for single and multi-layer printing. As a next step, multi-layer printing was tested with an individual layer height of 9 mm with a maximum number of 3 layers. After printing the first layer perpendicular to the sand, the next ones (2–3) maintain the same orientation but are translated along the global Z-axis. Variations of the differential growth-based geometry for the façade lenses were generated, alternating the density of the curves and defining ‘opening areas’ on the façade lenses. The printing speed for the base layer was constant across all multi-layer printing experiments at 115 mm/s, changing only for layers 2 and 3. The values of layer width and height which are defined in mm are translated into speed values, as Vertico has developed a slicer that correlates extrusion volume, with the printing speed of the robot, allowing to print with a varying layer width/height in the same print. As such for a given nozzle diameter and layer height a curve describes the relationship between the obtainable layer width at a certain movement speed of the robot arm. Demoulding. Once the 3D concrete printing is complete, easy manual demoulding can take place after only several days. The sand changes minimally the surface finish of the 3D concrete printed layer with which it was in direct contact. The surface finish showcases minimal irregularities due to the imprint of the sand grains on the concrete.

4 Conclusions 4.1 Results The printed layer width and height vary across the geometries following different material depositing strategies, explained in detail below. All discussed samples use a layer height of 9 mm with a constant speed of 115 mm/s for the first layer (Fig. 1). Sample A. This prototype (see Fig. 2) showcases the objective of fabricating a lens with thinner layers towards its longitudinal axis, which continue to be thinner in the upper

Exploring 3D Concrete Printing of Lattice Structures

133

Fig. 1. Left to right. Sample A. Sample B. Close-up of sample B’s layer resolution and alternating layer thicknesses. Visualization of altered printing speed along the print path.

half of the lens. The vertical thinning of the layers reduces the weight on the lower half the ellipsoid, increasing its stability. Speed values are assigned to the second layer curve based on the following logic: the second layer points are first projected onto an XY plane with the origin in the center of the panel and the Y axis running along the longitudinal direction of the oval. The points get their speed assigned by remapping their distance from the vertical plane YZ against a domain going from 160 mm/s to 115 mm/s. The closest point to the plane will be assigned a speed of 160 mm/s while the farthest will have a speed of 115 mm/s. On top of this logic, the points with a positive Y coordinate get assigned a percentage increment going from 100% to 130% the farthest they are from the center. Layer 3 follows the same logic with the only difference that the first domain goes from 200 mm/s to 115 mm/s. Sample B. The objective of this prototype (see Fig. 2) is to obtain a panel with thinner lines towards its longitudinal axis with thicker ones on the edges and on the bottom of the panel. An overhang of the curve corners of layer 3 accentuates the three-dimensional qualities of the façade screen. This effect was achieved by a locally defined drop in speed. Sample C. Another material depositing strategy tested is based on the radial thickening of the 3DCP layers as shown in Fig. 3. As in this case, several prototypes were fabricated with a continuously printed frame surrounding the lens-like component. This serves the purpose to not only ease handling but to also facilitate the assembly of the components as part of a larger system. The speed logic for 2nd and 3rd design layers is defined by domains expressed according to the distance of a point from vertical XY plane running through the major axis of the ellipsoid (thinner lines along the major axis). Those domains are then increased by a percentage going from 100 to 130% based on Y coordinates (slower speed/thicker lines in the lower part of the lens). Sample D. The printing logic of the prototype overlaps with the one described for sample A (see Fig. 4). This component differs with regards to its underlying lattice design as one larger and one smaller opening within the lattice area were incorporated. This was done in order to test the feasibility of large openings during the panel’s vertical assembly in order to allow for possible visual connections between a viewer standing behind the façade screen and an object of relevance. Discussion. The research on sub-additive 3D printing of optimized double curve concrete lattice structures developed at Ball State University follows an approach of rough path printing, allowing for large tolerances and imprecisions during printing. Subsequently, different post-production tooling methods are used to refine the resolution and

134

C. Nan and A. Vigorito

Fig. 2. Left to right. Sample C. Sample D Visualization of printing speed.

surface quality of the print (Zivkovic and Battaglia, 2018). Differing from this approach, the presented series of prototypes results in a high-resolution surface quality, managing to bypass the need for any post-production methods. This robotic fabrication method has proven to be not only efficient in terms of a near zero-waste process but also highly time-efficient. The average time needed to shape the sand formwork for one component was around 3 min. The maximum printing time required for a 4-layered lattice lens was of 7:30 min. Additionally, this workflow offers high degrees of flexibility for geometric exploration of facade components with expressive spatial articulation for lattice-based elements, other than following a volumetric approach. The use of concrete for primarily ornamental components is unusual. Although proof-of-concept prototypes, the finalised components showcase concrete’s ornamental potential through the produced shadow play of the lenses (see Fig. 3).

Fig. 3. Cured 3D printed concrete, lattice-based lenses and the generated shadow play.

4.2 Next Steps A comparative study is planned to investigate the advantages and disadvantages of using for the same geometry positive and negative sand formwork. In the presented case studies all lattice structures were printed on a positive sand formwork. The same geometry could be 3D printed not on top a positive half-ellipsoid, but within a scrapped out negative mold. Workflows, procedural steps and printing quality will be compared. Part of future research is dedicated to aspects relating to the development of a computational design workflow that optimizes the structural performance of the lightweight lattice screens by even further reducing material usage. In relation to this, we plan to investigate more complex geometries for the sand formwork. Although ellipsoids are three-dimensional shapes with double curvature, in the next research phase a particular focus will be set on geometries with pronounced double curvature. For the produced elements to be viable for mass-customization and their use in construction, the structural behaviour,

Exploring 3D Concrete Printing of Lattice Structures

135

layer delamination and stability of the lens-like façade components has to be tested. Different material mixtures containing synthetic fibres will be explored. Reinforcement will have to be integrated within the printed layers, allowing for a significant scaling up of the individual pieces. Future research will also consider constraints and steps for on-site assembly. Although robust when removed from the sand formwork and handled, customised steel handling frames may be needed to ease the transport and handling of the individual lattice components. Consortium and Funding. The presented work is developed by a consortium of the Architectural Design and Engineering Chair of TU Eindhoven (Prof. Juliette Bekkering, Assist. Prof. Cristina Nan, research assistant Alessio Vigorito), Neutelings Riedijk Architects (founder Michiel Riedijk, architect Chaoyu Huang), the 3DCP Vertico (founder Volker Ruitinga, Kees Leemeijer, Orestis Pavlidis) and Betonhuis (Cindy Vissering). The research is funded by the Dutch Research Council NWO, KIEM-CE grant.

References Antony, F., Grießhammer, R., Speck, T., Speck, O.: Sustainability assessment of a lightweight biomimetic ceiling structure. Bioinspir. Biomim. 9(1), 016013 (2014) Buswell, R.A., Leal de Silva, W.R., Jones, S.Z., Dirrenberger, J.: 3D printing using concrete extrusion: a research roadmap. Cem. Concr. Res. 112, 37–49 (2018) Doyle, S., Hunt, E.L.: Dissolvable 3D printed formwork, pp. 178–187 (2019) Global Status Report for Buildings and Construction 2019 – Analysis. (n.d.). IEA. https://www.iea. org/reports/global-status-report-for-buildings-and-construction-2019. Accessed 5 July 2023 Jipa, A., Dillenburger, B.: 3D printed formwork for concrete: state-of-the-art, opportunities, challenges, and applications. 3D Printing Addit. Manuf. 9(2), 84–107 (2022) Khoshnevis, B.: Automated construction by contour crafting—related robotics and information technologies. Autom. Constr. 13(1), 5–19 (2004) Kovaleva, D., Nistler, M., Verl, A., Blandini, L., Sobek, W.: Zero-waste production of lightweight concrete structures with water-soluble sand formwork. In: Buswell, R., Blanco, A., Cavalaro, S., Kinnell, P. (eds.), Third RILEM International Conference on Concrete and Digital Fabrication, vol. 37, pp. 3–8. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-06116-5_1 Wangler, T., et al.: Digital concrete: opportunities and challenges. RILEM Tech. Lett. 1, 67–75 (2016) Zivkovic, S., Battaglia, C.: Rough Pass Extrusion Tooling. CNC post-processing of 3D-printed sub-additive concrete lattice structures, pp. 302–311 (2018)

Geometric Variability and Viability in Designing and Fabricating Concrete Façade Components–A Systematic Review Deyan Quan1(B)

, Christiane M. Herr2

, and Davide Lombardi1

1 Department of Architecture, Design School, Xi’an Jiaotong-Liverpool University, 111 Ren’ai

Road, Suzhou, China [email protected] 2 School of Design, Southern University of Science and Technology, Xueyuan Dadao 1088, Nanshan, Shenzhen, Guangdong, China

Abstract. This paper explores the integration challenges of advanced parametric modelling techniques and architectural-scale concrete façade fabrication technologies. Despite concrete’s versatility and durability, which make it a preferred material for parametric façade designs, a nuanced comprehensive understanding of balancing its geometric variability and viability is absent. This review investigates the interplay among digital geometric design and fabrication techniques, shedding light on design strategies regarding fabrication constraints and the limitations of current design approaches. By categorising design methods in concrete façade design and production, it underscores the importance of fabrication-informed parametric design strategies. The paper reviews recent research on geometry types and related design approaches. It concludes by highlighting the strength and potentials of balancing geometric variability and viability in the parametric context, which has implications for the future design and construction of concrete architectural elements. Keywords: Geometric Variability · Concrete Façade Element · Parametric Design · Digital Fabrication

1 Introduction Concrete’s widespread use in construction, attributed to its strength and versatility for complex geometries [1], makes it a preferred material for parametric façade design. The primary limitations in freeform concrete façade realisation mostly lie in fabrication technologies rather than structural performance. Moreover, the integration of innovative parametric techniques enabling freeform façade elements with existing architecturalscale fabrication technologies is not always seamless [2]. Innovations like 3D concrete printing and 3D printed formwork have revolutionised concrete component production. Yet, few studies have systematically investigated the geometric constraints of these technologies, or the interplay between fabrication limitations and geometric design. While some research considers fabrication constraints in design methods, most studies focus © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 136–143, 2024. https://doi.org/10.1007/978-981-97-0621-1_17

Geometric Variability and Viability in Designing

137

on optimisation algorithms [3] and software [4] after the geometric design phase, rather than their early integration. The process of balancing geometric variability and viability in designing and fabricating concrete façade elements is under-explored, signalling the need for more comprehensive study. This paper provides a systematic review of current studies in concrete façade element production. It starts by defining parametric variability and geometric viability in fabrication and investigating their interaction with digital modelling tools and fabrication techniques. It categorises design strategies related to fabrication constraints and assesses existing design approach limitations and potentials, drawing from previous research. The paper then reviews state-of-the-art concrete façade production studies and projects, using data from CuminCAD and Elsevier’s Scopus databases. It employs search terms like “Concrete,” “Facade,” “Design,” and “Fabrication” to filter research published between 2013 and 2023, offering a comprehensive overview. The paper concludes by synthesising insights to enhance the understanding of how to achieve a balance between geometric variability and viability. In doing so, this paper underscores the value of a fabrication-conscious design strategy to enable feasible, creative solutions in concrete element design.

2 Geometric Variability and Viability Advancements in digital design have enhanced the exploration of intricate geometries using computational techniques. Parametric design, known for its flexibility, controllability and variability [5], facilitates non-standard design exploration through differences and variations. In this context, ‘parametric variability’ is defined as the alteration of one or more design parameters while others are held constant to observe the impact on a design or system’s output or performance. This concept is crucial in a parametric design process [6]. The process allows a wide exploration of variants within a parametric schema, supporting geometric design strategies involving variations in shape relations. Utilising a dynamic, rule-based approach, it permits the creation of parametric models with internal variables, aligning geometry with design boundaries [7]. Parametric design plays an essential role in bridging design and fabrication processes, enabling an evaluation of ‘geometric viability’, the suitability of a specific geometry for manufacturing [8]. It facilitates the switch between different models in a framework, promoting exploration of new forms while enhancing their geometric viability. The rise of parametric modelling and digital manufacturing has expedited the development of complex façade solutions. Constructing a parametric model builds essential relationships between geometric variations and functions, incorporating geometric viability to streamline the process. The incorporation of ‘parametric variability’ and ‘geometric viability’ plays a critical role in bridging design and fabrication processes, enabling the creation of precise, yet complex architectural designs.

138

D. Quan et al.

3 Geometric Constraints of Concrete Fabrication Techniques While digital fabrication techniques have advanced significantly, they each possess inherent geometric limitations. The exploration of these constraints, specifically within 3D concrete printing and digitally fabricated formwork technologies, is key to identify and address associated challenges, as highlighted in previous research [9]. In 3D concrete printing technology, geometric constraints pose a challenge in printing curved surfaces and overhangs. Some studies recommend an inclination angle of 45° for overhanging parts [10], while others propose that this angle is contingent on the specific features to be printed [11]. Solutions such as additional supporting structures using optimisation algorithms have been explored [12], and a topology optimisation formulation with layer-wise filtering was proposed to eliminate unprintable geometries [13]. Nonetheless, these solutions could limit geometric flexibility and may introduce tensional stresses [14]. Mainstream construction methods typically cast concrete into 3D printed moulds shaped to achieve the desired shapes [9]. However, fabricating accurate moulds for complex double-curved panels and volumetric non-standard shapes remains challenging, and various factors such as integrating reinforcement in concrete construction introduce additional geometric limitations [14]. From another perspective, the 45-degree printing of overhanging inclines is also pertinent in 3D printed formwork production. However, the specific angle limitation is primarily determined by the printing material properties, particularly the drying speed after material extrusion. These challenges underscore the importance of project-specific strategies and the integration of fabrication and geometric constraints into the design process. In the context of other digitally fabricated formwork methods, each has its unique limitations. Aspects like tool accessibility, material constraints, and economic considerations contribute to these limitations [9]. CNC hot-wire cutting works best with ruled surfaces, CNC folding with developable surfaces, fabrics cater to anticlastic surfaces, milling serves undercut-free shapes, and formative processes suit surfaces with limited principal curvatures. Despite their advancements, these methods still confront geometric limitations, such as the challenge of undercuts in CNC milling, and factors related to fabric behaviour. A design process that integrates these considerations can help navigate these constraints, maximising the potential of digitally fabricated formwork and enabling the creation of intricate concrete structures with enhanced precision and efficiency.

4 Design Methods - Accounting for Fabrication Constraints This section reviews design methods that consider fabrication constraints, often referred to as design rationalisation. This critical step in digital fabrication techniques for intricate geometries addresses fabrication and material constraints. Geometry rationalisation, after two decades of exploration, has been further categorised into five specific methods within the general framework of pre-, co- and post- rationalisation by Austern, et al. [15]. Pre-rationalisation includes fabrication-driven design, guided by a specific fabrication technique, and fabrication-aware form-finding, using algorithms for feasible designs. Post-rationalisation methods include optimisation, where design precedes fabrication

Geometric Variability and Viability in Designing

139

constraints and adjustments are later made by algorithms, and translation, which entails converting an initial design into a fabrication-friendly computational medium via the third party. In addition, parametric co-rationalisation provides continuous design feedback as the geometric design and prototyping fabrication are conducted interactively. Austern et al.’s [15] classifications, this paper investigates the appropriate timing and methods for integrating designers’ considerations into these processes (Fig. 1).

Fig. 1. Timing and iterative integration of designers’ considerations in five methods

Within fabrication-aware form-finding and optimisation methodologies, the integration of designers’ considerations is inherently constrained due to the algorithmic nature of these approaches, which precludes real-time feedback from the physical fabrication process. Contrarily, fabrication-driven design allows for an active participation of designers in incorporating fabrication constraints, although the inherent reliance on experiential data might limit the scope for design variability. Parametric co-rationalisation, harnessing the robustness of parametric tools as outlined in Sect. 2, facilitates a continuous interplay of design feedback throughout the process of geometric design and prototyping, thereby fostering an encompassing integration of the designers’ considerations.

5 Balancing Variability and Viability in Concrete Façade Design and Fabrication 5.1 Method of Selecting Studies After detailing design methods that accommodate fabrication constraints and therefore enhance the balance between variability and viability, this section proceeds to review their applications in concrete façade-related studies. This review aims to present the current state of the art in this field, with specific attention to the geometry type, employed fabrication technology, fabrication constraints, and the applied design methods. To this end, the process of selecting case studies for this research involved a comprehensive search using relevant search terms, such as “Concrete,” “Facade,” “Design,” and “Fabrication,” in relevant databases including CuminCAD and Scopus. A variety of combinations of these terms were employed to target research works published between 2013 and 2023. Initially, a total of 32 studies that met the defined selection criteria were identified. The assessment aimed to verify whether the design and fabrication processes were

140

D. Quan et al.

adequately documented, leading to the selection of 12 studies for further investigation. These selected studies concentrated specifically on design methods and investigated strategies adept at achieving a balance between geometric variability and fabrication viability, specifically in the context of concrete design and fabrication. 5.2 State of the Art Research Presented in chronological order in Table 1, the reviewed studies offer an overview of the development of geometrical complexities within recent concrete façade research. A detailed examination of the geometric types within these studies provides an understanding of the intricate interplay between design intention and manufacturing realities. Furthermore, an assessment of fabrication technologies and their inherent constraints illuminates how these factors shape the production of façade elements. This paper also investigates the design methodologies adopted across different categories, as delineated in Sect. 4. Table 1. Details of concrete façade element studies Author(s)

Façade Geometry Type

Fabrication Technology

Fabrication Constraint

Design method

Austern, et al. [16]

Double curved surface

CNC milling formwork

NA

Parametric co-rationalisation

Grobman [17]

Double curved surface

CNC milling and hot-wire cutting formwork

Surface curvature

Fabrication-aware design

Herr, et al. [18]

Freeform volumetric shape

3D printed formwork NA

Parametric co-rationalisation

Mohamed, et al. [19]

2D panel

Concrete 3D printing NA

Optimisation

de Campos, et al. [20]

Freeform volumetric shape

CNC milling formwork

NA

Parametric co-rationalisation

Loh, et al. [21]

Double curved surface

CNC adjustable mould

Surface curvature

Fabrication-driven design

Sitnikov, et al. [22]

Volumetric shape

CNC-milled ice formwork

NA

Parametric co-rationalisation

Sitnikov and Rogers [23]

Volumetric shape

CNC-milled ice formwork

NA

Parametric co-rationalisation

Meibodi, et al. [24]

Thin shell

Binder jet 3D printing

Interface precision

Parametric co-rationalisation

Bedarf, et al. [25]

2D panel

3D printing of mineral foams

Printable height

Translation

Quan, et al. [26] Freeform volumetric shape

3D printed formwork Curvature and overhang

Parametric co-rationalisation

Emami [27]

3D printed formwork Joint placement

Optimisation

2.5D Panel

Table 1 demonstrates that CNC milling is prevalent in the fabrication of doublecurved concrete façade studies, while 3D printing of formwork significantly contributes

Geometric Variability and Viability in Designing

141

to the production of freeform volumetric façade elements. There is also evidence of 3D concrete printing, albeit primarily limited to 2D panels. Regarding fabrication constraints, the curvature of double-curved surfaces emerges as the primary geometric limitation of CNC milling technology, even though some studies have omitted discussions on this aspect. Overhangs and curvature represent the fabrication constraints associated with 3D printing technologies involving formwork. In terms of design methodology, the parametric co-rationalisation approach is broadly employed, with a portion of the research still relying on post-rationalisation, generally confined to translation methods.

6 Discussion The reviewed studies highlight a persistent tension between design variability and viability in freeform concrete façade production. Despite ongoing progress in the development of digital design tools, fabrication viability remains a significant hurdle. While innovative methods like 3D printing and CNC milling are utilised, inherent geometric constraints limit their applicability. Various strategies have been explored to balance geometric variability and fabrication viability. Among them, parametric co-rationalisation is prevalent, integrating fabrication constraints into the earlier geometric design process. The strength of this method not only enables the rapid feedback loop between geometric design and fabrication process, but also improve the design flexibility due to embed more designer’s consideration according to the continuous feedback. However, there is a considerable reliance on post-rationalisation methods, whose sequential nature may inadequately address the intricate interplay between design diversity and fabrication feasibility. Despite limitations, the growing adoption of 3D printed formwork is noticeable. Merging 3D printing with parametric co-rationalisation could represent a significant shift in architectural production, potentially resolving prevalent challenges.

7 Conclusion This paper aimed to explore the balance between geometric variability and viability in concrete façade element design and fabrication. A comprehensive review of existing literature and case studies highlighted that, while strides have been made in the field, there are still critical challenges to be addressed. Parametric design tools and advanced fabrication techniques have indeed revolutionized concrete façade element production. However, the integration of these elements is not always seamless, and geometric constraints inherent to fabrication technologies often limit the creative potential of the design process. Existing design methods, such as parametric co-rationalisation, show promise in integrating fabrication constraints continuously into the design process. Given the increasing trend towards the use of 3D printed technologies, future research might focus on how these technologies can be further optimised to minimise geometric constraints and maximise design variability in a parametric environment. This paper enhances the understanding of contemporary challenges and potential design methodologies in the creation of concrete façades, facilitating the production of parametric concrete façade elements that are not only aesthetically captivating but also structurally sound and efficient to manufacture.

142

D. Quan et al.

References 1. Li, W., Lin, X., Bao, D.W., Xie, Y.M.: A review of formwork systems for modern concrete construction. In: Structures, pp. 52–63. Elsevier (2022) 2. Austern, G., Elber, G., Capeluto, I.G., Grobman, Y.J.: Adapting architectural form to digital fabrication constraints. In: AAG, pp. 10–33. (2018) 3. Koronaki, A., Shepherd, P., Evernden, M.: Fabrication-aware joint clustering in freeform space-frames. Buildings 13, 962 (2023) 4. Mesnil, R., Douthe, C., Baverel, O., Richter, C., Leger, B.: Structural exploration of a fabrication-aware design space with Marionettes Meshes. In: Proceedings of IASS Annual Symposia, pp. 1–10. International Association for Shell and Spatial Structures (IASS) (2016) 5. Fischer, T., Burry, M., Frazer, J.: Triangulation of generative form for parametric design and rapid prototyping. Autom. Constr. 14, 233–240 (2005) 6. Lin, B., Yu, Q., Li, Z., Zhou, X.: Research on parametric design method for energy efficiency of green building in architectural scheme phase. Front. Architectural Res. 2, 11–22 (2013) 7. Shepherd, P., Hudson, R., Hines, D.: Aviva Stadium: A parametric success. Int. J. Archit. Comput. 9, 167–185 (2011) 8. Megahed, N.A.: Digital realm: parametric-enabled paradigm in architectural design process. Int. J. Architecture, Eng. Constr. 4, 175–184 (2015) 9. Austern, G., Capeluto, I.G., Grobman, Y.J.: Real-time fabrication analysis: a method for evaluating fabrication constraints of complex concrete shapes. Archit. Sci. Rev. 65, 421–435 (2022) 10. Kuo, Y.-H., Cheng, C.-C., Lin, Y.-S., San, C.-H.: Support structure design in additive manufacturing based on topology optimization. Struct. Multidiscip. Optim. 57, 183–195 (2018) 11. Gaynor, A.T., Guest, J.K.: Topology optimization considering overhang constraints: Eliminating sacrificial support material in additive manufacturing through design. Struct. Multidiscip. Optim. 54, 1157–1172 (2016) 12. Anton, A., et al.: Concrete choreography: prefabrication of 3D-printed columns. Fabricate 2020: making resilient architecture, pp. 286–293 (2020) 13. Bi, M., Tran, P., Xia, L., Ma, G., Xie, Y.M.: Topology optimization for 3D concrete printing with various manufacturing constraints. Addit. Manuf. 57, 102982 (2022) 14. Raphael, B., Senthilnathan, S., Patel, A., Bhat, S.: A review of concrete 3D printed structural members. Front. Built Environ. 8, 291 (2023) 15. Austern, G., Capeluto, I.G., Grobman, Y.J.: Rationalization methods in computer aided fabrication: a critical review. Autom. Constr. 90, 281–293 (2018) 16. Austern, G., Capeluto, I.G., Grobman, Y.J.: Rationalization and Optimization of Concrete Façade Panels. Comput. Better Tomorrow, 727 (2018) 17. Grobman, Y.J.: Fabrication-Aware Design of Concrete Façade Panels (2018) 18. Herr, C.M., Lombardi, D., Galobardes, I.: Parametric design of sculptural fibre reinforced concrete facade components. In: Learning, Adapting and Prototyping, Proceedings of the 23rd International Conference of the Association for Computer-Aided Architectural Design Research in Asia, pp. 319–328 (2018) 19. Mohamed, B., Elkaftangui, M., Zureikat, R.: Towards rethinking the precast concrete industry in the UAE. In: Proceedings of the 23rd International Conference of the Association for Computer-Aided Architectural Design Research in Asia, pp. 287–296 (2018) 20. de Campos, F.M., Leite, R.M., Prudencio, C.F., Dias, M.S., Celani, G.: Prototyping a facade component mixed technologies applied to fabrication. In: 37th Education and Research in Computer Aided Architectural Design in Europe and XXIII Iberoamerican Society of Digital Graphics, Joint Conference (N. 1), pp. 179–186 (2019)

Geometric Variability and Viability in Designing

143

21. Loh, P., Leggett, D., Prohasky, D.: Robotic fabrication of doubly curved facade system. In: Proceedings of the 24th International Conference of the Association for Computer-Aided Architectural Design Research in Asia, pp. 521–530 (2019) 22. Sitnikov, V., Eigenraam, P., Papanastasis, P., Wassermann-Fry, S.: IceFormwork for cast HPFRC elements: process-oriented design of a light-weight high-performance fiberreinforced concrete (HPFRC) Rain-Screen Façade. In: The 39th Annual Conference of the Association for Computer Aided Design in Architecture (ACADIA), pp. 616–627 (2019) 23. Sitnikov, V., Rogers, P.: Preliminary assessment of environmental performance of ice formwork production method for irregular architectural elements of concrete. Int. J. Space Struct. 36, 78–87 (2021) 24. Meibodi, M.A., Odaglia, P., Dillenburger, B.: Min-max: reusable 3d printed formwork for thin-shell concrete structures: reusable 3d printed formwork for thin-shell concrete structures. In: Proceedings of the 26th International Conference of the Association for Computer-Aided Architectural Design Research in Asia, pp. 743–752 (2021) 25. Bedarf, P., Martinez Schulte, D., Senol Güngör, A., Jeoffroy, E., Dillenburger, B.: Robotic 3D printing of mineral foam for a lightweight composite facade shading panel. In: Proceedings of the 26th International Conference of the Association for Computer-Aided Architectural Design, pp. 603–612. Association for Computer Aided Architectural Design Research in Asia (2021) 26. Quan, D., Herr, C.M., Lombardi, D., Gao, Z., Xia, J.: Prototyping parametrically designed fiber-reinforced concrete façade elements using 3D printed formwork. In: Proceedings of IASS Annual Symposia, pp. 1–11. International Association for Shell and Spatial Structures (IASS) (2022) 27. Emami, N.: Employing additive manufacturing to create reusable TPU formworks for casting topologically optimized facade panels. J. Build. Eng. 106946 (2023)

Augmented LEGO™ An Experiment Utilising Augmented Reality (AR) for Algorithm-Oriented Generative Design and Conceptual Model Assembly Guidance Yang Song1(B) and Wei Zhao2,3 1 School of Architecture, University of Liverpool, Liverpool, UK

[email protected]

2 School of Architecture, Zhengzhou University, Zhengzhou, China 3 Henan International Joint Laboratory of Eco-Community and Innovative Technology,

Zhengzhou, China

Abstract. Using discrete elements, LEGO™ bricks, as the design unit, this paper presents experimental research utilising augmented reality (AR) technology for algorithm-oriented generative design and conceptual model assembly guidance. The research aims to develop a unique pipeline and workflow that allows users to modify, set constraints, preview, generate discrete architectural design outcomes immersively, and assemble the physical conceptual model manually through AR guidance for the initial architectural design draft stages. A sample workflow has been tested as a series of generated conceptual complex discrete structures in the Augmented LEGO™ workshop. We transformed the current design method to an algorithm-oriented way with discrete element generate design plug-in Wasp, and enriched the current generative design, scheme preview, and the handicraft modelmaking methods holographicly with AR immersion plug-in Fologram. As for the physical outcomes, all participants designed and assembled generated discrete LEGO™ structures successfully with the assistance of AR. To conclude, this paper describes the workshop research questions, methods and process in detail, reflects and summarises the findings and limitations of the Augmented LEGO™ experiment at the end. Keywords: Augmented Reality (AR) · algorithm-oriented generative design · discrete element · assembly guidance · conceptual model

1 Introduction Generative design means geometric generation from an idea or concept that starts with the definition of a series of initial mathematical and geometric parameters. It generates some possible solutions allowed by the variability of the original parameters. Generative design can leverage the latest technology trends in Artificial Intelligence (AI), along with human input, to automatically create the best possible design solutions for a given © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 144–151, 2024. https://doi.org/10.1007/978-981-97-0621-1_18

Augmented LEGO™

145

goal, which offers numerous solutions, and new solutions are swiftly generated with changes in input parameters [1]. Moreover, generative design produces complex and sophisticated design solutions. For architecture, visual design is a critical component. Visualising complex objects, especially at scale, is incredibly difficult with existing visualisation tools like computer screens. Architects used to work with physical models made out of cardboard or wood in order to achieve a better understanding of the design spaces. This traditional physical model paradigm does not fit the generative design method and solutions. In addition, the conventional assembly processes often resort to descriptive instructions on paper or in digital format to guide the assembly sequence. These traditional methods require operators to perform the mapping between the instructions and the actions on real objects without any mechanism to provide feedback or assistance during such procedures [2, 3]. The assembly of a physical concept model for a design solution is already incredibly resource-costing and time-consuming for normal designs, let alone generated design outcomes that are more complicated. Augmented Reality (AR) is proposed to be the best visualisation tool with its related technical features for these complex design solutions, with holographic support and spatial tracking that enable accurate visualisation and interactive modifications at scale on-site. AR gives designers a way to directly cognise the virtual concept model in a real environment without restricting their perception [4]. This technology may also significantly reduce costs for the assembly of physical concept models through increased spatial perception and reduced errors, time, and cognitive workload, as shown in previous studies [5]. Through AR, it is possible to display digital information (e.g., text, images, step-by-step instructions, 3D illustrations, or other relevant data) directly in the users’ field of view while performing an assembly procedure [6]. In spite of the significant benefits of AR applications in generative design and the following concept model assembly, there is still little research conducted on the process in architecture design. The main research question of this experiment is to explore whether AR technology can assist the conventional generative design method and optimise the related design process immersively, and whether AR can assist the manual assembly of concept models corresponding to the complex shape generative design outcomes without conventional assembly documents. This experiment verifies the above research questions and the feasibility of the above assumptions through the in-person workshop. Additionally, this paper concludes the role and limitations of AR in generative design and conceptual model assembly by analysing the workshop process, and summarising related findings, limitations, and participants’ experience from the workshop.

2 Methodology The Augmented LEGO™ workshop proposes experimental research utilising AR for algorithm-oriented generative design and conceptual model assembly guidance consisting of two phases: A) AR-assisted generative design, in which customised algorithmoriented design code and constraints will communicate with AR interactive inputs to achieve generative design, preview, and optimise the design outcomes through the immersive environment for the initial draft stage; B) AR-assisted assembly guidance, in which user can manually assemble the conceptual model through AR for the physical display

146

Y. Song and W. Zhao

stage. This research aims to provide architectural practitioners with an immersive generative design experience and an efficient conceptual model assembly method for the preliminary design stages. For this workshop, we enriched and developed the traditional algorithm-oriented generative design code with the immersive and interactive way in Grasshopper, which is familiar to architectural practitioners to understand and manipulate. Similarly, we transformed the conventional assembly process from measurements and documentbased instructions to holographic assembly guidance for complex shape fabrication. The employed generative design and assembly guidance scripts are driven by an instant connection between the development environment (Grasshopper), discrete element generative design plug-in (Wasp), and AR interactive immersion plug-in (Fologram). Wasp is an algorithm-oriented generative design plug-in, which can quickly generate a variety of discrete structures according to the rules and restrictions set by designers. Fologram is an AR plug-in developed by architects, which can translate interactive inputs, such as hand gestures, screen taps, and QR codes, into digital data in the Grasshopper, and preview 3D models as holograms overlapping to reality. The above plug-ins work with their integrated graphical algorithm editor, Grasshopper, a standard tool in architectural fields, and are proposed to be easily integrated into workflow developments and workshop experiments (Fig. 1).

Fig. 1. The flow chart of the Augmented LEGO™ workshop, including Phase A (AR-assisted generative design) and Phase B (AR-assisted assembly guidance) (in blue); the related plug-in for each critical step (in red); and the outcomes of each phase (in green).

The hardware in this workshop includes mobile AR devices (smartphones or tablets), either iOS or Android, with the Fologram App, installed (the mobile client of the Fologram plug-in) for AR immersion. Additionally, workshop participants need their personal laptops for the script developments in Grasshopper, and the back-end data live stream between physical and virtual. We have to ensure that any software and hardware used in this workshop should be free and ubiquitous so that all participants can quickly assess and manipulate the corresponding operations. Moreover, this workshop uses three sizes of the basic LEGO™ brick unit (1 × 1, 1 × 4, and 2 × 4) for the discrete element generative algorithm and the physical conceptual model making.

Augmented LEGO™

147

This in-person workshop was taken by 12 bachelor students, three master students, and two PhD researchers majoring in architectural design and urban planning with their consent. They all have 3D-modeling experience in Rhinoceros 7 and Grasshopper, but none had prior knowledge and skill in generative design and AR technology. Either the Wasp and Fologram plug-in was new to them. The participants witnessed the whole process of script development, AR-assisted generative design, and AR-assisted conceptual model assembly during this two-day workshop and presented their results.

3 Experiment and Findings The workshop was imitated for two days. On the first day, participants received introductory lectures about AR and generative design in the architectural field. After that, we brought the Wasp and Fologram tutorials to the participants to demonstrate the essential generative design logic and interactive AR functions. Finally, we delivered the basic discrete algorithm-oriented generative design script with specific rules and constraints in Grasshopper to inspire the participants for their design developments (Phase A); On the second day, the participants were asked to show and discuss the AR-assisted generative design outcome. After that, participants are required to assemble the conceptual model physically with AR guidance as the final result. The end of the day was dedicated to the final review and extensive discussion about the findings and user experience of the AR-assisted generative design and assembly guidance (Phase B). 3.1 Phase A: AR-Assisted Generative Design Phase A proposes an immersive generative design method in AR for architects. Compared with the conventional method, users can control and interfere with algorithm rules and restrictions through interactive inputs immersively for the generative design outcomes, and preview the results as holograms overlapping in reality on-site through AR. Before accessing Phase A, users need to be familiar with the fundamental interactive function of AR UI, which will be easy and controllable for beginners. The design script was pre-set and run in Grasshopper for a discrete LEGO™ unit structure generative design. Participants only need to set restrictions to the algorithm and play with the interactive input through AR UI. If they are proficient in Wasp and Fologram, they can also try to create customised generative design algorithms for other structure designs. To start with the AR-assisted generative design, first, users are required to specify the generative design algorithms, such as how each discrete element unit is connected. For this experiment, the design algorithm set as each type of LEGO™ unit can be connected randomly up and down on each connection joint, but the connecting angle can only be 0, 90, 180, and 270°. The generative design is based on a rectangle range (25.5*25.5 mm), which meets the requirements of the physical assembly scale in Phase B. Moreover, the related interactive inputs, such as unit numbers, random factors, start calculations, etc., will be extracted from Grasshopper to AR UI for users to interact with (see Fig. 2). Users can also set restriction algorithms to interfere with the generative design rules. For example, users can create different geometric volumes in the above rectangle range to set restrictions, and the LEGO™ units will not be generated in the restricted volumes, which

148

Y. Song and W. Zhao

can increase the design variants and complexities. The above-restricted geometries will be extracted to AR UI for users to move immersively (see Fig. 2). So far, the user has finished the generative design algorithm development. Second, participants are required to use mobile AR devices and connect Fologram to realise the AR-assisted generative design and transform the data between interactive inputs in reality and virtual generated models in design software. The participants can adjust different interactive parameters in AR UI to control LEGO™ structure generation, as well as, move the AR restricted geometries to influence the design shapes. Then, the generative design outcome will be shown as holograms overlapping, in reality, on-site (see Fig. 3). Participants can see and feel the generated results, especially the spatial organisations, immersively at any angle or scale. By previewing, perceiving, and comparing multiple generative design results, users can choose the optimal design outcome for further modifications and the physical assembly process.

Fig. 2. This is a screenshot of Rhinoceros/Grasshopper, which illustrates the generative design algorithms with AR interactive inputs coloured red (on the right). The generative design range is coloured in green, as well as the restriction geometries are coloured in red (on the left).

Fig. 3. The user interacts with the generative design in AR by setting the interactive restriction volumes and AR UI inputs through the mobile device screen. The holographic generative design outcome is shown as holograms on-site in AR overlapping on the surrounding contexts.

Augmented LEGO™

149

The findings of Phase A suggest that, compared to the conventional design method, the AR-assisted way can help designers better understand draft space, and participate in and control generative design in real time through interactive inputs in AR. At the same time, the holographic visual design result display can help designers choose the best result from various generated outcomes based on their own experience and design ideas, significantly improving the efficiency and user experience of generative design. After the basic generative design algorithm and AR interactive function introduction, without any skills, participants can easily start designing within ten minutes on their mobile devices. Moreover, the users’ feedback was good during the AR-assisted generative design phase. All participants, without generative design and AR experiments, indicated they could use AR UI quickly and freely to set rules and restrictions, generate, preview, and optimise the best design outcome. However, due to the relatively large amount of calculations for the generative design, there are often freezes during data transmission. Especially when utilising AR UI for interactive interferences, the results of the generative design are often displayed after a delay of a few seconds. It is believed that through the development and update of AR equipment, this problem will be solved. 3.2 Phase B: AR-Assisted Assembly Guidance Phase B proposes a holographic conceptual model assembly guidance in AR. Compared with the traditional method, which needs measurement and document-based instruction, for the AR-assisted way, users can preview holographic design outcomes overlapping on-site, and the AR assembly guidance layer-by-layer with the specific location of each element that needs to be assembled. This method is proposed to let participants efficiently build physical models with complex shapes on-site with visual guidance instead of reading and translating information from conventional documents. Participants only need to follow the AR-assisted assembly guidance and place the LEGO™ units one by one. To start with the AR-assisted assembly guidance, first, users are required to import the selected generative design outcome from Phase A to the assembly guidance script in Grasshopper. This script is pre-set and developed by the workshop instructor. In the script, each different type of LEGO™ unit has a different colour in AR; at the same time, for straightforward assembly guidance, each LEGO™ unit will be displayed in the form of outlines, users can switch outline or volume display mode through AR UI during the assembly process. Moreover, users need to set a reference plane in Rhinoceros to locate the AR guidance in both digital and reality. This reference plane contains location and orientation information, and it is abbreviated as a QR code for AR devices to recognise and scan. The QR code needs to be printed and pasted accurately into the physical environment after measurements. After scanning the QR through an AR device, the holographic assembly guidance will be displayed and fixed in the assembly location. Second, the participants can assemble the LEGO™ units on the base, layer by layer, according to the AR holographic guide. After accomplishing the assembly of the current layer, the user can activate the assembly guidance of the next layer through AR UI (see Fig. 4). The findings of Phase B suggest that, compared to the conventional physical model assembly method, the AR-assisted way can help designers save time wasted in reading

150

Y. Song and W. Zhao

Fig. 4. The user activates the AR holographic assembly guidance through the mobile device.

assembly documents and measuring the location, and improve assembly efficiency and accuracy, especially for complex shape outcomes from generative design. After the basic AR function introduction, without any skills, participants can quickly assemble the complex LEGO™ structures with AR guidance through their mobile devices. Moreover, the users’ feedback was good during the AR-assisted assembly guidance phase. All participants finished their design physical conceptual model assembly in around an hour or two, which contains almost 600 LEGO™ units for each design (see Fig. 5). However, due to the influence of ambient light and human movements on the assembly site, the AR holographic guidance drifts intermittently and rotates at a slight angle. Therefore, during assembly, the user must repeatedly scan the QR code to correct the AR hologram position. This phenomenon is because mobile AR devices have low accuracy in locating AR holograms through their cameras. This situation is believed to be improved by using multi-sensor head-mounted AR devices (HMD). In addition, for the rotation of AR assembly guidance, since the LEGO™ growth rule can only be built at right angles (0, 90, 180, and 270°), this tolerance can be eliminated during the manual assembly process.

4 Conclusions This paper examines the potential of AR in assisting generative design and manual assembly of physical conceptual models through a workshop. The benefits and limitations of AR in these practices are summarised and concluded. AR-assisted generative design enables designers to see and perceive the generated solutions from any angle and scale immersively for a better understanding of the draft space and to choose the optimal design outcome. AR-assisted assembly projects live holographic guide of complex generative design models to the real environment. The holographic guide streamlines the assembly process as it eliminates document reading and locating processes during the model assembly. AR-assisted generative design and assembly improve the designer’s perception and understanding of design space, which benefits architectural educational practices, and its ability to deal with complex and sophisticated design projects endows itself the potential for building design practices.

Augmented LEGO™

151

Fig. 5. These are some physical conceptual model outcomes designed by the AR-assisted algorithm-oriented generative design and assembled by the AR-assisted holographic assembly guidance through the Augmented LEGO™ workshop.

Acknowledgement. The authors gratefully acknowledge the funding support from the International Cooperation Funding of Henan Province (NO. 231111520800).

References 1. Vicario, P.J., et al. Augmented reality and generative design. a tour through print media for designing and representing architecture. In: Edulearn16 Proceedings. IATED (2016) 2. Khuong, B.M., et al. The effectiveness of an AR-based context-aware assembly support system in object assembly. In: 2014 IEEE Virtual Reality (VR). IEEE (2014) 3. Wu, L.-C., Lin, I.-C., Tsai, M.-H.: Augmented reality instruction for object assembly based on markerless tracking. In: Proceedings of the 20th ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (2016) 4. Wang, J., Hu, Y., Yang, X..: Multi-person collaborative augmented reality assembly process evaluation system based on hololens. In: Chen, J.Y.C., Fragomeni, G. (eds) Virtual, Augmented and Mixed Reality: Applications in Education, Aviation and Industry. HCII 2022. LNCS, vol. 13318, pp 369–380. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-06015-1_25 5. Wang, X., Ong, S.K., Nee, A.Y.: A comprehensive survey of augmented reality assembly research. Adv. Manuf. 4, 1–22 (2016) 6. Alves, J.B., et al., Comparing augmented reality visualisation methods for assembly procedures. Virtual Reality, 1–14 (2022)

Virtual Reality and EEG in Creativity Research: Investigating the Impact of Designed Environments on Creative Performance Fatemeh Taherysayah1,2(B) and Claudia Westermann1 1 Xi’an Jiaotong-Liverpool University, Suzhou, China 2 University of Liverpool, Liverpool, UK

[email protected], [email protected]

Abstract. Creativity is a distinctive feature of human nature that involves various cognitive processes and drives innovation, progress, and personal growth. Recent research in cognitive neuroscience challenges the traditional belief that creativity is a fixed trait. Through investigations of the temporal neural correlations of a person’s creative action, neuroscientists have revealed that creativity is dynamic and based on both explicit and implicit processing to generate and evaluate ideas. In other words, there are distinct steps that lead to a creative action or proposal, such as the analysis of a task, the generation of ideas and their verification, which require different modes of thinking. These different modes of thinking, which correspond to different patterns of brain waves, can be further categorised into divergent, convergent, abstract, and concrete thinking. This paper presents an overview of an ongoing research project that aims to investigate the effect of designed environments on the different stages of a human’s creativity. Tracking the creative performance of individuals in the VR space by using both electroencephalography (EEG) and questionnaires allowed us to gain insights about the ability of spaces with high aesthetic quality to foster creativity. The experimental set-up permitted us also to draw conclusions about the suitability of neuroscience tools in architectural design contexts. This paper provides an overview of the theoretical basis, and a critical review of the employed tools and methods, with a focus on the use of VR technology and EEG in evaluating creativity. Furthermore, the paper outlines potential applications of VR, specifically through VR drawing, in empirical studies of creativity. Keywords: Creativity · designed environments · Virtual reality (VR) · VR drawing · Electroencephalography (EEG)

1 Introduction Research on creativity has been actively pursued for several decades. The advancement of technology and the ability to monitor brain signals has resulted in a notable increase in empirical research on creativity. Recent cutting-edge neuroscience studies have provided researchers with a better understanding of how the brain operates during different stages of the creative process. © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 152–159, 2024. https://doi.org/10.1007/978-981-97-0621-1_19

Virtual Reality and EEG in Creativity Research

153

Researchers have found that creativity encompasses a dynamic process consisting of various crucial stages [1]. Each stage of creativity, including preparation, idea generation (incubation and illumination) and evaluation [2], depends on the functional connectivity between different brain networks [3]. Despite numerous studies indicating that specific environmental stimuli can augment individual creativity, there exist inconsistencies in the findings, and the precise impact of the environment on the temporal dynamics of creativity remains uncertain [4]. This paper aims to outline insights into the ability of designed spaces to stimulate each of the different modes of thinking essential in various steps of the creativity process. It presents an overview of an ongoing research project that explores the impact of virtual reality (VR) representations of environments with high aesthetic value on the different stages of an individual’s creativity, utilising both questionnaires and the brain imaging tool Electroencephalography (EEG).

2 Background Creative thinking is a fundamental aspect of human cognition. As it aims at new and compelling ideas or products [5], it is generally considered the basis of the practice of design [6]. Yet, creativity finds applications in various dimensions of human action, demanding our ability to approach problems from new angles and solve them across multiple domains [7]. According to Pallasmaa, an artifact must transcend what can be logically predicted to be considered creative [8]. Chambers defines creativity as a multifaceted interaction between an individual and their surroundings, resulting in the generation of novel and distinctive outcomes [9]. It is widely agreed that for something to be labelled as creative, it must possess two primary elements: originality (i.e., novelty, uncommonness, or uniqueness) and effectiveness (i.e., practicality, suitability, or task-appropriateness) [7, 10]. Guilford’s theory asserts that creativity depends on divergent thinking [11, 12]. Divergent thinking most reliably predicts an individual’s creative performance [13, 14]. In a broader context, creative thinking relies on striking a proper balance and suitable timing, encompassing a range from divergent and abstract thinking to convergent and concrete thinking [4, 15]. Scholars have proposed various models to describe and classify the creativity process. One of the most well-known and widely applicable models is Wallas’ Art of Thought model (1926), which describes the process of generating a creative idea as a series of four stages. These stages include preparation (identifying a problem), incubation (initiating subconscious attempts), illumination (bursting ideas into conscious awareness), and verification (verifying, evaluating, elaborating, and applying ideas consciously), requiring different cognitive processing [2]. However, empirical studies have not yet fully elucidated the neural correlations associated with different stages of creativity [4]. 2.1 The Role of the Brain in Enacting Creativity Recent advancements in cognitive neuroscience methodologies have enabled researchers to delve into the intricate mechanisms of creativity in the brain and predict patterns of

154

F. Taherysayah and C. Westermann

brain connectivity during creative performance [3]. Studies have consistently demonstrated that the creative process is a dynamic phenomenon involving temporal neural correlations and engaging explicit and implicit cognitive mechanisms for generating and evaluating ideas [1, 4, 16]. Beaty et al. employed sophisticated approaches to identify active neural networks correlated with divergent thinking and discerned brain regions that exhibit varying functional associations with creativity. They discovered that the network responsive to creativity is associated with three specific brain systems and their large-scale participation. Specifically, the default mode network (DMN), the executive control network (ECN), and the salience network (SN) interact dynamically to support creative performance [1, 16]. A noteworthy aspect of these three neural circuits is that they usually work in opposition, yet they concurrently interact within the creative brain [1]. Similarly, Lloyd-Cox et al. exploring the contribution of the ECN and DMN to the different phases of the creative process, observed recorded higher activity of the ECN during idea evaluation while the DMN was more active during idea generation [17]. High-level creativity is the result of the intricate interplay among diverse cognitive processes, encompassing concrete thinking for evaluating ideas and abstract thinking for generating ideas [4, 18]. In particular, the verification phase is assumed to require convergent and concrete thinking and is likely to be associated with ECN neural function. Conversely, the incubation and illumination stages primarily focus on generating ideas and often involve divergent and abstract thinking processes with the assistance of DMN [1, 3, 17]. It is also hypothesised that idea generation is associated with a decrease in alpha strength, while preparation and verification lead to alpha increase [4]. In addition to neuroscience, the field of design has experienced a significant interest in exploring the application of neuroscience tools and methods. This surge can be attributed to remarkable advancements in portable and affordable brain imaging tools, which have opened new avenues for exploring the effect of environment on cognitive actions like creativity by analysing EEG data and investigating the brain waves correlated with this cognitive process. 2.2 Environmental Effect on Creativity Several recent studies have investigated the impact of various environmental features, such as form, light, color, texture, and transparency, on individual creativity using subjective questionnaires [19, 20]. However, research has revealed that study participants often edit, filter, and somehow censor responses that are consciously given, leading to contamination and manipulation of the data provided [21]. To complement these findings, some researchers have turned to neuroscience tools, which offer real-time data on the human neural system. EEG signals, measuring changes in neural activation in the brain, provide clear indications of cognitive activities [22]. Understanding the impact of the environment on creativity relies on key insights into “the temporal dynamics of neural and cognitive functions”[4] (p. 3). Previous studies, which focused on fixed environmental factors and overlooked the dynamic shifts in modes of thinking and cognitive styles that occur during this complex phenomenon, are not sufficient to investigate the complexity of the creative process. There are several hypotheses suggesting that the preparation or verification phases are more effective in

Virtual Reality and EEG in Creativity Research

155

environments that promote attention and concentration. Conversely, abstract-evoking environments may be more conducive to idea generation [4]. However, there is currently a gap in exploring these hypotheses, identifying the specific environmental features involved, and clarifying the effects of the environment on creativity phases in relation to brain waves or brain network functioning. 2.3 Creativity Research in VR Studies have shown that virtual environments simulating the real world can initiate interaction, immersion and trigger the imagination of participants [23]. Virtual environments are increasingly recognised as valuable tools for fostering creativity, particularly in the idea generation phase, by promoting flow, engagement, and immersion [24]. Flow refers to a condition where individuals become completely immersed in their activities and lose track of time [25]. In addition, Ryokai et al. found that design students who utilised VR head-mounted displays (HMDs) exhibited increased empathy towards future users by actively considering their designs from various perspectives, in comparison to a non-VR group of students. The approach enabled them to identify potential issues and effectively cater to the needs of the users they were designing for [26]. Similarly, Toumi et al. emphasised the role of the VR condition in enhancing the designers’ sense of empathy towards their future users through the design process [27]. While the domain of creativity assessment already provides clear evaluation metrics through established methods and questionnaires, the integration of VR in creativity assessment has introduced novel approaches to evaluate creative actions or products. By comparing VR-based approaches with established methods, researchers can explore the effectiveness of these novel techniques, determine their suitability, and assess the potential advantages of utilising VR in creativity assessment. In the realm of virtual environments, there is a novel assessment method for evaluating creativity using 3D drawings as a response to a design task [7, 28]. In their recent publication, Barbot et al. developed a method called VIVA (Virtual Immersive Visual Art) for assessing creativity in VR [7]. This approach involves participants creating 3D drawings in response to specific task, and then employing the Consensual Assessment Technique (CAT) [29] to rate the drawings. While there have been numerous studies examining the influence of VR environments on creative thinking, only a few have utilised brain imaging techniques to investigate this effect [30]. There is also a growing interest in understanding the contribution of aesthetic experience in architecture, leading to an increase in a trend to evaluate theoretical concepts through experimental studies [31, 32]. This paper provides an overview of a project that investigates the impact of environments with confirmed aesthetic qualities in VR on stages of creativity. The project employs EEG and qualitative and quantitative questionnaires as assessment tools.

156

F. Taherysayah and C. Westermann

3 EEG/VR Study on the Impact of Environments of High Aesthetic Value on Creativity Building upon the insights of existing studies, our project “Spaces for Creativity: strategies for integrating empirical research on embodied experience into architectural design processes” has provided us with a new understanding of how well-designed spaces can impact an individual’s creativity. A summary of recent findings in neuroscience research on the dynamic creative process is depicted in Fig. 1.

Creativity steps

Preparation

Incubation

Illumination

Verification

Idea generation Thinking state Processing Brain network Brain waves

Divergent and abstract thinking Attention and focus Spontaneous and unconscious processing DMN Increased alpha

Decreased alpha

Concrete and convergent thinking Attention and focus ECN Increased alpha

Fig. 1. Summary of correlation of neural networks (ECN and DMN) and four conventional phases of creativity.

According to Vessel et al., aesthetic preferences undeniably influence whether an individual feels inspired [33]. Whether an impression of being inspired can be translated into an activity that is creative has not been researched. In contrast, our project provided conditions where individuals’ aesthetic preferences were not directly questioned but their creative output was tested in environments that have been judged as being of high aesthetic quality and suitable for a creative task. The participants’ aesthetic experiences in a VR environment were recorded and analysed using brain imaging tools, specifically EEG. Participants, wearing EEG, were asked to choose the most appealing and suitable environment for a creative design task from three different immersive spaces. Two of these spaces mimic immersive room-sized installations by world-famous artists: Infinity Rooms by Yayoi Kusama and Breathing Light by James Turrell. These spaces have been discussed by visitors and critics over the years as environments of high aesthetic quality. The third is a VR replica of an entry to the Architecture Futures competition held at Xi’an Jiaotong-Liverpool University in 2021, which was awarded the first prize [34]. It should be noted that while VR may not fully replicate the entire atmospheric quality of physical space, it is capable of capturing and transmitting core aspects effectively for evaluation purposes, making it suitable for conducting these types of experiments. 25 participants, including faculty members and undergraduate students from the Design School participated in the first phase of the experiment. According to the questionnaire responses, of the three rooms, the Breathing Light VR replica was experienced

Virtual Reality and EEG in Creativity Research

157

as exhibiting the highest aesthetic appeal. It was also selected as the most supportive space for performing a creative task. The EEG data of the three rooms revealed no significant difference in the theta, alpha, and beta bands. However, the gamma band power was higher in the Breathing Light room compared to the others. Cropley argued that since creativity is multidimensional and influenced by various factors, it is preferable to employ multiple measurements rather than relying solely on one test to comprehensively assess its diverse aspects [35]. Therefore, the second phase employed three tasks to assess creative action in the environment: A) the standard Alternate Uses Test (AUT), which has been widely utilised in creativity studies (51.1%) [36], B) a short VR 3D drawing, and C) Word Association Task (WAT) [37]. In the second phase, conducted in two different VR environments, the Breathing Light room and a typical office space of moderate aesthetic quality, we compared the recorded EEG data across three distinct phases of creativity. As demonstrated in Fig. 2, there is a significant increase in the power of all four frequency bands: theta, alpha, beta, and gamma, during the idea generation phase when compared to the preparation and verification phases. While the study did not reveal any significant differences in the creative actions of the participants in the two different environments, visualization of the four frequency bands showed slightly higher strengths in beta and gamma waves in the Breathing Light room. Given that the pilot phase included a relatively small sample size of 14 participants across two distinct conditions, drawing any robust conclusions necessitates a larger sample size, which will be undertaken in the next phase of this project.

Fig. 2. Frequency band power across various phases of creativity

4 Conclusion Investigating the impact of the environment on each stage of the creativity process, considering various cognitive processes and the emission of different brain waves, holds significant potential for gaining new insights that will assist with the development of spaces designed to foster creativity. Despite inconsistencies in creativity research findings across different fields such as architecture, environmental psychology, and cognitive

158

F. Taherysayah and C. Westermann

neuroscience, advancements in portable brain imaging tools and VR technology offer promising prospects for evaluating design projects during the design process. Investigating individuals’ aesthetic experience and creative performance, the project presents the potential of VR/EEG study in establishing an experimental condition to explore the effect of environmental features on different phases of creativity.

References 1. Beaty, R.E., et al.: Robust prediction of individual creative ability from brain functional connectivity. Proc. Nat. Acad. Sci. 115(5), 1087–1092 (2018) 2. Wallas, G.: The Art of Thought Harcourt. Bruce and Company, New York. (1926) 3. Beaty, R.E., et al.: Network neuroscience of creative cognition: mapping cognitive mechanisms and individual differences in the creative brain. Curr. Opin. Behav. Sci. 27, 22–30 (2019) 4. Daikoku, T., et al.: Importance of environmental settings for the temporal dynamics of creativity. Thinking Skills Creativity 41, 100911 (2021) 5. Kaufman, J.C., Sternberg, R.J.: The International Handbook of Creativity. Cambridge University Press, Cambridge (2006) 6. Cross, N.: Designerly Ways of Knowing. Springer, London (2007) 7. Barbot, B., et al.: Creativity with 6 degrees of freedom: feasibility study of visual creativity assessment in virtual reality. Creativity Res. J. 1–18 (2023) 8. Pallasmaa, J.: Empathic and embodied imagination: intuiting experience and life in architecture. Architecture and Empathy (2015). P. Tidwell, Editor, Tapio Wirkkala-Rut Bryk Foundation 9. Chambers, J.A.: Beginning a multidimensional theory of creativity. Psychol. Rep. 25(3), 779–799 (1969) 10. Runco, M.A., Jaeger, G.J.: The standard definition of creativity. Creativity Res. J. 24(1), 92–96 (2012) 11. Cho, J.Y.: An investigation of design studio performance in relation to creativity, spatial ability, and visual cognitive style. Thinking Skills Creativity. 23, 67–78 (2017) 12. Guilford, J.P.: The Nature of Human Intelligence. McGraw-Hill Series in Psychology, New York (1967) 13. Runco, M.A.: Commentary: divergent thinking is not synonymous with creativity. Psychol. Aesthet. Creat. Arts 2(2), 93–96 (2008) 14. Runco, M.A.: Divergent Thinking. Ablex Publishing Corporation Norwood, NJ (1991) 15. Kim, K.H., Pierce, R.A.: Creativity tests, in encyclopedia of creativity, invention, innovation and entrepreneurship. In: Carayannis, E.G. (ed.) Encyclopedia of Creativity, Invention, Innovation and Entrepreneurship, pp. 472–481. Springer, NeW York (2013). https://doi.org/10. 1007/978-1-4614-3858-8_19 16. Yuan, K., et al.: Core brain networks interactions and cognitive control in internet gaming disorder individuals in late adolescence/early adulthood. Brain Struct. Funct. 221, 1427–1442 (2015) 17. Lloyd-Cox, J., et al.: The time course of creativity: multivariate classification of default and executive network contributions to creative cognition over time. Cortex 156, 90–105 (2022) 18. Cagle, M.: A general abstract-concrete model of creative thinking. J. Creative Behav. (1985) 19. McCoy, J.M., Evans, G.W.: The potential role of the physical environment in fostering creativity. Creativity Res. J. 14(3–4), 409–426 (2002) 20. Thoring, K., et al.: Design principles for creative spaces. In: DS 92: Proceedings of the DESIGN 2018 15th International Design Conference (2018)

Virtual Reality and EEG in Creativity Research

159

21. Ramachandran, V.S., Hirstein, W.: The science of art: a neurological theory of aesthetic experience. J. Conscious. Stud. 6(6–7), 15–51 (1999) 22. Saha, A., Minz, V., Bonela, S., Sreeja, S.R., Chowdhury, R., Samanta, D.: Classification of EEG signals for cognitive load estimation using deep learning architectures. In: Tiwary, U.S. (ed.) IHCI 2018. LNCS, vol. 11278, pp. 59–68. Springer, Cham (2018). https://doi.org/10. 1007/978-3-030-04021-5_6 23. Huang, H.-M., et al.: Investigating learners’ attitudes toward virtual reality learning environments: based on a constructivist approach. Comput. Educ. 55(3), 1171–1182 (2010) 24. Hagedorn, L.J., et al.: Virtual reality and creativity: how do immersive environments stimulate the brain during creative idea generation? In: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (2023) 25. Csikszentmihalyi, M.: Flow: The Psychology of Optimal Performance. Cambridge University Press, Cambridge (1990) 26. Ryokai, K., et al.: Examining children’s design processes, perspective-taking, and collaboration when using VR head-mounted displays. Int. J. Child-Comput. Interact. 33, 100451 (2022) 27. Toumi, K., et al.: Technologies for supporting creativity in design: a view of physical and virtual environments with regard to cognitive and social processes. Creativity. Theories–Res. Appl. 8(1), 189–212 (2021) 28. Yang, X., et al.: Examining creativity through a virtual reality support system. Educ. Technol. Res. Dev. 66(5), 1231–1254 (2018) 29. Amabile, T.M.: Social psychology of creativity: a consensual assessment technique. J. Pers. Soc. Psychol. 43(5), 997 (1982) 30. Taherysayah, F., et al.: Virtual reality and electroencephalography in architectural design: a systematic review of empirical studies. J. Build. Eng. 85, 108611 (2024) 31. Berleant, A., Carlson, A.: The Aesthetics of Human Environments. Broadview press, Peterborough (2007) 32. Redies, C.: Combining universal beauty and cultural context in a unifying model of visual aesthetic experience. Front. Hum. Neurosci. 9, 218 (2015) 33. Vessel, E.A., et al.: Stronger shared taste for natural aesthetic domains than for artifacts of human culture. Cognition 179, 121–131 (2018) 34. Westermann, C. Architecture Futures (2021). https://www.architecture-futures.com/ (2021) Accessed 03 Jan 2023] 35. Cropley, A.J.: Defining and measuring creativity: are creativity tests worth using? Roeper Rev. 23(2), 72–79 (2000) 36. Benedek, M., et al.: Creativity assessment in neuroscience research. Psychol. Aesthet. Creat. Arts 13(2), 218–226 (2019) 37. Merseal, H.M., et al.: Free association ability distinguishes highly creative artists from scientists: findings from the big-C project. Psychol. Aesthetics, Creativity, Arts (2023)

Analysis of Differences in Street Visual Walkability Between Human and Machine Perception: A Case Study of an Anonymous University Campus Yuchen Xie1(B) , Yunqin Li1,2 , Lingshan Huang1 , and Jiaxin Zhang1,2 1 Nanchang University, Nanchang, China

[email protected] 2 Osaka University, Osaka, Japan

Abstract. Recent studies of street visual perception have shown that street visual walk perception can be predicted using deep learning models, but machine-human perceptual differences limit the direct application of deep learning models to decision aids. In previous research, we developed a visual walk perception classification deep multitask learning (VWPCL) model for measuring visual walk perception (VWP). Also, the activation maps generated by the interpretable machine learning method Grad-CAM (Gradient-weighted Class Activation Mapping) were used to generate visual interpretations for the prediction results of the VWPCL model. However, its visualised machine perception results are not validated with real human visual perception data, and therefore designers have difficulty trusting the model’s perceptual prediction results. Based on this issue, this study conducted an experiment based on a desktop eye-tracker on a university campus to analyse the differences between human and deep learning models in street visual perception. The results of the study show that there are some differences between humans and deep learning models in street vision walkability perception. In future work, more quantitative analysis methods based on image comparisons and other sensory data will be incorporated for further in-depth research. Keywords: Visual walkability perception (VWP) · Eye tracking · Deep learning · Gradient-weighted Class Activation Mapping

1 Introduction “Walkability” encapsulates the efficacy, safety, and amenity of pedestrian navigation within an area, rooted in subjective perception [1]. Numerous Street View Imagery (SVI)aided walkability audit methodologies, founded on human subjective scoring data and computer vision techniques, have recently emerged [2]. Yet, these often inadequately replicate human ambulatory perception. To bridge this gap, Grad-CAM, a representative interpretable machine learning technique, could decode perceptual patterns of computer vision models. Nonetheless, studies employing objective human perception data to validate these models are scant. Objective ocular movements, captured via eye-tracking © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 160–168, 2024. https://doi.org/10.1007/978-981-97-0621-1_20

Analysis of Differences in Street Visual Walkability

161

technology, can instinctively evaluate human walking choices and spatial perception [3], augmenting traditional walkability scoring. Our prior research developed a deep learning framework leveraging virtual reality panoramic imagery and interpretable machine learning to quantify and interpret Visual Walkability Perception (VWP) across six dimensions. In this research, one of the visual walkability perception dimensions (comfort) will be selected for auditing, and the differences between human perception and deep learning perception results will be discussed qualitatively and quantitatively by comparing and analysing the subjects’ eye-movement data with the results of the interpretable machine learning Grad-CAM auditing, and combining with the results of the in-depth interviews in order to better optimise the machine learning model.

2 Methodology and Data Sets 2.1 Research Framework In this study, a three-phase research framework was designed based on a previously trained VWPCL model and human eye movement experiments. (1) evaluation, prediction, and interpretation of VWP (‘comfort’); (2) in-depth human visual perception insights derived from eye-tracking experiments and interviews; and (3) quantitative and qualitative comparison of perceptual discrepancies between human and machine. ‘Comfort’, chosen for examination, encapsulates both visual aspects (e.g., width, openness, and greenness) and psychological insights. Firstly, the VWPCL model was engaged to estimate street VWP ratings on comfort, with Grad-CAM employed to visualize comfort-associated elements via an activation map. Secondly, eye-tracking experiments provided insights into human visual perception. Employing a desktop eye-tracker, participants viewed images, performed perceptual scoring, and selected principal influencing factors, followed by interviews for deeper exploration. Lastly, perception results of humans and machines were scrutinised quantitatively and qualitatively. Data for this study were sourced from (1) pre-labeled panoramic SVIs with VWP scores as training data for the VWPCL model; (2) 110 SVIs from an anonymised university campus for VWPCL model predictions; (3) eye-movement data for interpreting human visual perception; and (4) in-depth interview responses for visual interpretation validation (Fig. 1).

162

Y. Xie et al.

Fig. 1. Research framework diagram

2.2 Study Area In this investigation, an anonymous university campus served as the research location. Sampling points were situated every 50m on the streets within the 96.6 ha campus, with 110 panoramic photographs captured by a panoramic camera forming the field dataset. These 110 photographs were subdivided into four categories: primary vehicular, secondary vehicular, pedestrian, and pedestrianised commercial streets. Ten representative panoramic images were selected from this categorisation (Fig. 2).

Fig. 2. Campus Map

2.3 Deep Learning Model Perception Introduction to Interpretable Deep Learning Techniques for VWPCL Models and VWP Results. The DenseNet-inspired Visual Walkability Perception Convolutional Learning (VWPCL) model, a deep learning model, was trained on the VR Visual Walking Perception Rating (VRVWPR) dataset, achieving a high accuracy (90.4%) in the Comfort perceptual dimension [4]. Gradient-weighted Class Activation Mapping

Analysis of Differences in Street Visual Walkability

163

(Grad-CAM), an interpretable deep learning technique, provided visual interpretation of the VWP results [5] (Fig. 3). This process blends gradient information with feature mapping, generating a gradient-weighted class activation map that identifies crucial class discrimination areas. In the output activation heat map, warmer coloured regions indicate more discriminative pixels.

Fig. 3. VWPCL Training and Grad-CAM Workflow

2.4 Human Visual Perception Human Ratings, Eye Movement Experiments and In-depth Interviews. We recruited 11 volunteers (6 female, 5 male) from architecture and urban planning backgrounds for an eye-tracking experiment. Ten representative panoramas were each decomposed into four cubic slices and presented to the subjects. Each volunteer rated the Comfort perception dimension and selected the primary influencing elements for each photograph. This experiment was recorded using a desktop eye-tracker. Post-experiment, in-depth interviews were conducted with the subjects to gain a richer understanding of their experiences and thought processes. Comparison of Grad-CAM Activation Maps with Eye Movement Data. In this experiment, a pixel-by-pixel comparison of Grad-CAM activation maps with eyemovement heat maps is combined with element selection and in-depth interviews to further explore in-depth the differences between machine and human. Comparison of VWP Scores and Correlation Analysis of the Proportion of SVI Subjects with VWP Scores. Correlation analyses were conducted for human and machine scores on the Comfort perception dimension. Firstly, VWP scores are obtained from the VRVWPR dataset, and the semantic segmentation-based DeepLabv3 + model and the Cityscapes dataset are used to compute the percentage of elements in the built environment of a street. Finally, a stepwise multiple linear regression model [Eq. (1)] was used to investigate the relationship between the VWP scores of the comfort perception dimension and physical components. yi = β0 + β1 + xi + . . . + εi , i = 1, . . . , n

(1)

where yi is the response variable; xi represents the regression variables; β1 , β2 ,…, βn are partial regression coefficients; εi is an error term; and the subscript i indexes a particular observation.

164

Y. Xie et al.

3 Experiments and Results 3.1 VWP’s Predicted Results Figure 4 shows the results of the VWPCL model’s prediction of comfort (VWP) scores for 10 panoramic images. These images were predicted to have low, medium, and high scores, and the overall accuracy of the model on these 10 images was 90.8% with high confidence. Among the predicted high-scoring SVIs, larger green areas were a common feature, suggesting that the proportion of greenery may have a positive impact on comfort. Whereas in the low-score SVI, some scenes have more non-motorized vehicles and others have more open skies and a smaller proportion of greenery, implying that open skies and non-motorized vehicles may have a negative impact on comfort.

Fig. 4. Human-machine score difference chart

3.2 Results of Human Visual Perception Scores We recruited 11 volunteers to score the scenes, and the scoring results are shown in Table 1. Among them, there were three, four, and three scenes with high, medium, and low scores, respectively. The first two highest-scoring scenes were Scene 10 and Scene 8, while the lowest-scoring scenes were Scene 4 and Scene 5. Scene 8 and Scene 1 were Table 1. Human Perception Rating Summary Scale Scenes

Average Score

Subject

Ranking

1

2

3

4

5

6

7

8

9

10

11

1

6.36

6

8

4

6

7

6

7

7

6

5

8

4

2

3.55

2

3

6

2

4

3

3

5

4

2

5

7

3

3.55

3

2

3

4

1

2

3

6

3

6

6

7

4

2.64

4

2

6

1

4

1

2

3

2

1

3

10

5

3

2

3

4

2

1

2

2

4

4

4

5

9

6

7

6

7

7

6

8

6

5

8

7

8

9

3

7

6.18

5

8

7

6

5

6

5

8

6

5

7

5

8

7.55

6

8

7

8

7

9

7

8

8

9

6

2

9

4.36

3

6

5

2

4

6

3

5

3

6

5

6

10

8.55

7

9

8

9

9

9

8

9

9

7

1

1

Analysis of Differences in Street Visual Walkability

165

rated low in the VWPCL model but ranked second and fourth in the human ratings, while Scene 7 was rated high in the VWPCL model but in the middle bracket of human ratings, and the remaining scenes had the same human-machine ratings segmentation. 3.3 Comparison of Perceived Differences Between Deep Learning Models and Humans Differences in VWP Scores. Firstly, human perception scores (0–10) were reclassified into three categories: high, medium, and low. Subsequently, the Wilcoxon signed rank test assessed the perceptual scores of machine and human perception on SVI to determine significance level. The results suggested no significant difference (p = 0.414 > 0.1), which indicates that machine perception and human perception are close to each other on the whole. Insights from in-depth interviews and element selection demonstrated in Scene 10, which received highest human perception ratings, the predominant element was appealing greenery, followed by scenic harmony and spatial openness (Fig. 5). Subjects generally observed a harmonious contrast between distant buildings and nearby plants. Conversely, in Scene 8, with substantial rating variation, the influences were scenic harmony, architectural aesthetics, and openness. The open sky view and layered distance perception contributed to comfort for most subjects while standing on the bridge. Scene 7, with lower human perception scores than machine perception scores, was marked by the lake and lush greenery. However, subjects also noted negative elements: dirty ground, bins, and foul odours, resulting in a mid-range rating. Scenarios 4 and 5 were placed at the bottom of both human and machine perception ratings, attributed to dirty ground, poor hygiene, chaotic non-motorised vehicle parking, and deteriorated buildings. Interestingly, subjects acknowledged weather’s influence on perception ratings. Some felt that a sunlit scene without shade could garner a lower score. Diverse perceptions emerged regarding wide streets and sky openness; some subjects expressed potential loneliness walking in a broad street like Scene 1. Moreover, familiarity with certain scenes influenced some subjects’ perceptions. In summary, perceptual judgement discrepancies may exist between humans and machines, notably regarding elements like the sky and road. These differences may persist even in scenes with closely matched human and machine perceptual scores.

Fig. 5. Element selection word frequency map

Differences Between VWP’s Explainable Deep Learning (Grad-CAM) Results and Human Visual Perception. Interpretable Deep Learning (Grad-CAM) Results. Figure 6 showcases a selection of Grad-CAM-generated activation maps, highlighting

166

Y. Xie et al.

image regions significantly influencing the scores. Mirroring the earlier findings, areas activated in high-scoring images included plants, water, and pathways. Notably, in Scene 10, the sky, which the preceding analysis had marked as a negative factor, occupies a significant portion of the scene. Low-scoring images frequently activated the sky and road, implying their potential negative impact on more open streets. Of note, the activation maps revealed some discrepancies with actual perception—for instance, non-activated non-motorised vehicles and bins in low-scoring images, and both high- and low-scoring images activating trees and sky. Differences Between Grad-CAM and Eye Movement Data. We amassed substantial eye-movement data from 11 subjects’ eye-movement experiments (Fig. 6). The data revealed no apparent gender or age bias. A pixel-by-pixel comparison of gaze thermograms and Grad-CAM activation maps showed limited similarity, yet gaze thermograms and trajectories helped to elucidate Grad-CAM’s activation choices. Scene 10, highestrated by human perception, saw Grad-CAM activating the sky, whereas eye movement data concentrated on the greenery. This scene’s top three selected elements were greenery, scenic harmony, and open space. Subjects generally had a positive pre-impression of Scene 10 as spatially open, but humans typically do not scrutinize the sky, explaining the lack of eye movement data there. In the two lowest-rated scenes, Scenes 4 and 5, eye movement data better aligned with subjects’ chosen elements, focusing on dirty grounds, old façades, and disorganised parked non-motorised vehicles. In Scene 5, subjects paid prolonged attention to the rubbish bin and waste, chiefly influencing the low scores. These elements diverged considerably from Grad-CAM’s activated building façades and tree areas. Thus, despite similar human and machine scores in Scenes 4 and 5, there were substantial discrepancies in element selection. In summary, (1) humans are influenced by conspicuous objects or overall scene colour, with particular attention to unique features. Some subjects found Scene 8 colour too monotonous, and the shop sign colour in Scene 4 too jarring. (2) Humans prioritise evaluation criteria harder for elements to reflect, such as environmental cleanliness, tidiness, and overall atmosphere. (3) Grad-CAM may focus more on the proportion of positive and negative elements within the image.

Fig. 6. Schematic diagram of fixation point, fixation time and heat map

Correlation Analysis of the Proportion of SVI Subjects with VWP Scores. Figure 7 illustrates stepwise multiple regression analyses for comfort perception, considering 19 physical components against human and machine perception scores. Here, independent variables are ordered by their impact on the comfort perception dimension, displaying either positive (blue) or negative (orange) effects. The models’ goodness-of-fit (R-squared values) are 0.735 and 0.467.

Analysis of Differences in Street Visual Walkability

167

As depicted in Fig. 8, human and machine perceptions share partial correlation similarities. “Vegetation” and “Terrain” have a positive influence, while “Road” negatively influences both. However, “Truck” and “Sky” diverge, corroborating previous findings that human and machine perceptions of the sky differ. The elements most negatively impacting human perception are “Person” and “Car”, possibly due to indiscriminate vehicle parking and large pedestrian presence in the image. “Motorcycle” also negatively correlates, as humans associate it with noise. Both human and machine perceptions assign a large weight to the “Vegetation” element, aligning with prior studies linking urban greenery to increased pedestrian comfort and street appeal. Cumulatively, these stepwise multiple regression results quantitatively corroborate the comparative analysis findings, underscoring the distinction between human and machine perception.

Fig. 7. Correlation Analysis Chart

4 Discussion and Conclusions While deep learning models have been applied to study street perception, few studies have ventured to visually explore the perceptual divergence between machines and humans using Grad-CAM and eye-tracking data. The framework designed herein enables efficient and straightforward exploration of the likenesses and differences in human and machine perceptions of street visual walkability. Initially, VWPCL model was utilized to predict the scores of 10 Street View Images (SVIs) from a specific campus, under the dimension of comfort perception. These scores were then compared to human perception scores from an eye-tracking experiment involving 11 subjects. The results suggested a closer alignment of perceptual scores between humans and machines, albeit with discernable disparities. Subsequently, an eye-tracking measure was established and juxtaposed with the interpretable deep learning technique Grad-CAM to delve deeper into the similarities and differences. Despite the commonalities in human eye movement data, significant individual differences hindered the establishment of a clear link between gaze tendency and perceptual scores. Combined with the in-depth interviews, it was found that there are some differences between humans and machines in the way of evaluating and structuring the perceptual evaluation of SVI: human perception is more biased towards the overall sensation and observation of details or can be influenced by other previous sensory results, such as the machine model’s inability to recognise open spaces and views coordinated with this sensation, and the more negative elements mentioned, dirty floors, the abrupt colour of the signboards, and old elevations, are also details that could not be noticed. Using a stepwise

168

Y. Xie et al.

regression model, we further probed the correlation between human and machine perception. This affirmed the existence of significant differences, especially in elements difficult to discern through semantic segmentation. This highlights the need for other incorporating sensory and detail elements in future studies. Future work will enrich the eye-tracking experiments and utilize eye-tracking instruments that better reflect the human experience. We will also gather diverse human perception data and incorporate additional quantitative analysis techniques based on image comparison for more thorough research.

References 1. Aghaabbasi, M., Moeinaddini, M., Shah, M.Z., Asadi-Shekari, Z., Kermani, M.A.: Evaluating the capability of walkability audit tools for assessing sidewalks. Sustain. Cities Soc. 37, 475– 484 (2018) 2. Bleˇci´c, I., Cecchini, A., Trunfio, G.A.: Towards automatic assessment of perceived walkability. In: Gervasi, O., Murgante, B., Misra, S., Stankova, E., Torre, C.M., Rocha, A.M.A.C., Taniar, D., Apduhan, B.O., Tarantino, E., Ryu, Y. (eds.) ICCSA 2018. LNCS, vol. 10962, pp. 351–365. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-95168-3_24 3. Chen, Y., Chen, Z., Du, M.: Designing attention—research on landscape experience through eye tracking in Nanjing road pedestrian mall (street) in Shanghai. Landscape Architecture Front. 10(2), 52–70 (2022) 4. Li, Y., Yabuki, N., Fukuda, T.: Measuring visual walkability perception using panoramic street view images, virtual reality, and deep learning[J]. Sustain. Cities Soc. 86, 104140 (2022) 5. Selvaraju, R.R., Cogswell, M., Das, A., et al.: Grad-cam: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 618–626 (2017) 6. Oki, T., Kizawa, S.: Evaluating visual impressions based on gaze analysis and deep learning: a case study of attractiveness evaluation of streets in densely built-up wooden residential area. Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci. 43, 887–894 (2021) 7. Das, A., Agrawal, H., Zitnick, L., et al.: Human attention in visual question answering: do humans and deep networks look at the same regions? Comput. Vis. Image Underst. 163, 90–100 (2017) 8. Bylinskii, Z., Judd, T., Oliva, A., et al.: What do different evaluation metrics tell us about saliency models? IEEE Trans. Pattern Anal. Mach. Intell.Intell. 41(3), 740–757 (2018)

AI and Environmental Session

Exploration of Conceptual Design Generation Based on the Deep Learning Model – Discussing the Application of AI Generator to the Preliminary Architectural Design Process Yuxin Bao

and Changying Xiang(B)

The Hong Kong University of Science and Technology, Kowloon, Hong Kong [email protected]

Abstract. The rapid advancement and heightened awareness of Artificial Intelligence (AI) have stimulated a substantial surge in the growth of deep learning models. This has led to an increasingly prevalent debate surrounding the possibility of AI replacing human architects. Nevertheless, it is foreseeable that architects will explore an AI-aided design process at this stage. Several deep learning models, including NovelAi, DELL-E·2, Midjourney, and Stable Diffusion, have emerged, making it easier to generate images efficiently without the need for multidisciplinary knowledge related to algorithms and programming. During the conceptual design phase of an architectural project, it is crucial for architects to present several massing proposals in various styles within a limited timeframe. It means a huge amount of modelling and drawing works. This paper focuses on Stable Diffusion, Midjourney, and DALL-E 2, as primary examples to discuss the approaches used in the preliminary design process as a smart assistant. These AI platforms are expected to optimize the conceptual design work by reducing the time of transforming hand-drawn sketches into rendering photos and enhancing the visualization of massing diagrams. This article analyzes the impact of AI work-related activities carried out by architects and architectural students based on a designed survey containing various images generated by AI programs. The survey aims to investigate the performance of several popular AI programs and architects’ perspectives towards AI. The findings indicate that AI has the potential to assist human architects to some extent with satisfactory performance. The effective application of AI generators can significantly optimize the design process, allowing architects to explore more creative and aesthetic aspects. Keywords: Machine learning · AI-aided · Architecture · Efficiency · work flow

1 Introduction The integration of AI into architectural design is not a recent concept. The efforts to simulate an architect’s design capabilities using a computer can be traced back to the late 1960’s [1]. With the recent surge in research related to AI programs, nearly all professions are likely to be impacted and potentially disrupted in the foreseeable future, © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 171–178, 2024. https://doi.org/10.1007/978-981-97-0621-1_21

172

Y. Bao and C. Xiang

including the architectural design. Machine Learning is increasingly being used as an assistant tool. Notable progress in the development of AI-aided design has been achieved as the programming and algorithm integrated into the design work [2]. At present, machine learning-based AI generating programs, such as Midjourney, Stable diffusion, and DALL-E 2, are the most widely used. Notably, numerous AIgenerating tools have emerged for architectural applications, including image synthesis, floor plan variation, diagram generation, and 3D model building, among others. While these tools are not yet fully developed for immediate application in the industry, it can be predicted that they will alter the work mode and workflow in a substantially different manner. Architecture sets itself apart from other forms of art in that its creations must harmoniously embody aesthetic appeal, structural stability, and practical functionality [3]. Through all the design stages, the phase of conceptual design is one of the most important parts. At the beginning stage of a design project, the fundamental task is to determine the design style and direction of general massing. Typically, architects commence the work with an abstract concept and an indistinct idea of its form, which provides the basis for proposing a diverse set of options. The initial shape in the concept phase of a building can influence its overall performance and construction costs, energy consumption, layout configuration, solar gain, and other related features [4]. However, the conceptual design of architecture is a complicated process that necessitates professional skills, past experience and inspiration to propose novel designs. It is praiseworthy to learn from the masterpieces and show respect in a new project, while plagiarizing other works or simply combining several designs together cannot be regarded as professional. While, the algorithms of deep learning can be an appropriate approach. Generative deep learning models, such as GANs and VAEs which have been widely studied, have exhibited extraordinary abilities in producing creative design proposals in two or three dimensions. These models have particularly impacted the automated production of building drawings, including massing, architectural plans, interior and elevation design [5]. According to design thinking researchers, generative design tools are perceived as fostering creativity by enabling the generation and exploration of numerous design alternatives during the beginning stages of the design process. This approach maintains possibilities for creative potential in divergent thinking, which is regarded as a critical characteristic of individuals with creative abilities [6]. This article aims to investigate the current popular AI-assisted models and the impact of rapidly developing AI tools on architects and their daily work. This research could serve as a foundation for architects to explore new workflows in the future, paving the way for a novel approach to the profession.

2 Research Questions This paper aims to address three primary research questions: To what extent can deep-learning based AI programs assist in the conceptual design process for architecture? How do architects perceive and evaluate the results generated by AI systems in architectural design? What are the potential implications of AI generators on the role of architects in the design process?

Exploration of Conceptual Design Generation

173

3 Methodology The research methodology employed in this study consisted of four stages. In stage (I), three promising AI programs which are popular currently among designers and artists are introduced as assistant tools for the study. In stage (II), a survey was designed in both Chinese and English to collect information and perspectives from architects. The survey comprised four parts, including basic information, investigation of AI tools, evaluation of AI-generated images, and opinions towards AI. During the third part, several images generated by three AI programs mentioned previously were evaluated by the respondents in the form of an anonymous questionnaire (see Fig. 1).

Fig. 1. Illustration of the survey logic.

In stage (III), the content of the surveys was transcribed and analyzed, and the results were presented in the form of charts displaying the percentage of different choices. Most of the survey questions were multiple-choice, with only a few being open-ended. The Chinese responses were translated into English and combined with those in English. Finally, in stage (IV), the findings from the literature review and survey results were presented, forming the basis for the study’s conclusions and future trends in the architectural field.

4 Results 4.1 Promising Tools In light of the emergence and commercialization of numerous AI tools, there are persistent inquiries regarding the potential comparability of AI-generated outputs with those produced by human designers during the conceptual design phase. Among many AI tools, three of them are most promising, namely Stable Diffusion, Midjourney, and DALL-E 2, which anyone can readily access and operate to generate a series of images within minutes.

174

Y. Bao and C. Xiang

Stable Diffusion (https://stablediffusionweb.com/) Stable Diffusion is developed by Stability AI, Runway and Machine Vision & Learning research group at LMU Munich, released in 2022 [7]. Midjourney (https://www.midjourney.com/) MIdjourney first entered open beta in July, 2022. It was developed as an extension based on a chat application named “Discord”. Every second, more than 20 tasks have been completed [8]. DALL-E 2 (https://openai.com/dall-e-2) DALL·E 2 is founded by OpenAI in April 2022. The number of users has risen to 1.5 million in only 5 months. Averagely, 2 million pictures are generated per day [9]. 4.2 Analysis of Survey Results The survey comprised a total of twenty-one questions, including seventeen multiplechoice and four open-ended questions. The number of questions that could be answered by the respondents varied depending on their selections, ranging from fourteen to twentyone. Therefore, for each person, only finishing no less than fourteen questions was considered as an effective response. A total of sixty-one complete responses were received. After the completion of the questionnaire, the gathered data was entered into a Microsoft Excel spreadsheet. Subsequently, diagrams were generated to facilitate the process of analysis. Investigation of AI Tools. Figure 2 shows the distribution of respondents’ working years and the situation of AI application. The blue bar represents the architect has designed with AI tools while the green one represents not yet using. The results revealed that most of the respondents were young architects, and approximately thirty percent of them reported using AI programs in their work.

Fig. 2. Distribution of working years

Fig. 3. -1. Number of users for four AI generators. -2. Distribution of frequent application phases.

These thirty percent of participants were directed to the second part of the survey, which aimed to investigate their utilization of AI tools. The results indicated that Stable Diffusion and Midjourney were the dominant tools in the field of AI drawing (see Fig. 3.1). For daily working, the majority applied AI programs in concept design phase (see Fig. 3.-2). Survey Relating to the Quality and Reality of Images Generated by AI Programs. Figures 4.-3 and 5.-1 were generated using three different AI programs, namely Stable Diffusion, Midjourney, and DALL-E 2. In addition, a same sketch (see Fig. 4.-1) was

Exploration of Conceptual Design Generation

175

provided as an input in the generation of Fig. 4.-3 to evaluate the style and creativity that AI can perform during the early design phase. The respondents were asked to select the image that best aligned with the standard of an architect from Fig. 4.-3. The results showed that nearly half of the respondents considered the image generated by Midjourney to be the best, while more than a quarter preferred the one generated by Stable Diffusion (see Fig. 4.-2).

Fig. 4. -1. Base input sketch for AI generation. -2. Distribution of preferred AI outcome. -3. Rendering generation results made by Midjourney, Stable Diffusion and DALL-E 2 (from left to right respectively).

Fig. 5.-2 aimed to simulate the situation where architects only have information about the context and function during the brainstorming phase, to assess whether AI can generates varied conceptual hand drawings. There were similar prompts while without any sketch as a reference. The respondents were asked to select the image that they found most useful in expressing design ideas as an architect’s drawing. The results indicated that Midjourney had the most supporters (see Fig. 5.-1).

Fig. 5. 1. Sketch generation results made by DALL-E 2 (A), Midjourney (B) and Stable Diffusion (C). -2. Distribution of preferred AI output.

Overall, Midjourney and Stable Diffusion performed well in generating high-quality images that better meet architects’ requirements. Survey Relating to the Recognition Between Human and AI. In this section of the survey, the goal was to test the recognition of AI-generated images. Several sets of images were presented to the respondents. Part of them, including floor plan drawings and interior perspective renderings were posted below (see Figs. 6.-1 and 7.-1). In each set, two images were generated by AI programs, while one was created by a human architect. The respondents were asked to choose the image drawn by a human. The bar charts in Fig. 6.-2 revealed that most respondents were able to identify the human-drawn image from the three choices presented. However, approximately twenty respondents

176

Y. Bao and C. Xiang

were misled and chose the image generated by Midjourney. Notably, for the interior perspective series, the majority of respondents selected the AI-generated image (see Fig. 7.-2).

Fig. 6. -1. Floor plan generation results made by a human architect (A), Midjourney (B) and Stable Diffusion (C). -2. Selection of the image made by human architect.

Fig. 7. -1. Interior rendering generation results made by a human architect (A), Midjourney (B), and Stable Diffusion (C). -2. Selection of the image made by human architect.

The results imply that AI has the potential as an efficient assistant in generating conceptual images. In certain specific aspects and situations, AI may even confuse architects who possess expertise in this field. Survey Relating to the Learning Ability of Style Transfer. Figure 8.-1 illustrates architectural renderings generated based on the design styles of a world-famous architect, Zaha Hadi. The question comprises four options, with three images generated by the same three AI programs mentioned above, and the fourth option being “unable to decide”. Among the respondents, fifty-one reported having knowledge of Zaha’s designs, and eighty percent of them believed that the image generated by Midjourney exhibited more distinct characteristics of Zaha’s design style (see Fig. 8.-2).

Fig. 8. -1. Zaha Hadid Style rendering generation results made by DALL-E -2 (A), Stable Diffusion (B) and Midjourney (C). 2. Distribution of preferred AI outcome.

Only one person cannot decide within four choices, which means the outcomes are quite clear and recognizable for illustrating the main features of this particular architect. Overall, Midjourney can be considered the most productive AI tool in assisting design work and transforming popular architectural styles. Opinions Towards AI. The final part of the survey was dedicated to gathering opinions of improving aspects and the future of AI in architectural design. Although AI generators

Exploration of Conceptual Design Generation

177

can produce high-quality images within a few minutes, issues remain with the output and modification process. Firstly, the generated images often require partial and precise modifications that blend seamlessly with the surrounding. Second main point is that it would be preferable for AI to inspect and correct the images itself after generation. Lastly, the database needs to be enriched with masterpieces in various styles to enable its application in design work (see Fig. 9).

Fig. 9. Proportion of potential improvements for AI tools. Fig. 10. Opinions distribution about if AI can replace human architects.

Regarding the debate on whether AI will replace human architects, the majority of respondents believed that AI would only serve as an efficient assistant and never replace human architects (see Fig. 10). 4.3 Literature Review One of the challenges involved in this process pertained to formulating precise textual inputs to ensure accurate comprehension by the AI system. [10]. When inputting prompts into an AI platform, users often need to make multiple attempts, modifying the prompts iteratively in order to generate satisfactory and expected results. It is worth noting that the quality of the output images is closely linked to the prompts utilized [11]. In addition, a huge amount of data input is necessary for the learning algorithm to build the AI platforms effectively. While in reality, the fact is that accessible data are rather limited and deviation exists in underlying class distributions. It makes getting sufficient data more complicated [12]. It is commonly worried about whether artificial intelligence will replace the place of human architects. Frey and Osborne made research about the future of employment. They believed that it is not difficult for computer to be novel, but difficult to follow and explain the value, even with the rapid speed development of technology [13]. The future development of AI and its positive impact on the field of architecture are promising. Stanislas believed that AI shows great potential in widespread integration into architect’s daily practice in a short time. Traditional design patterns are facing a transformation if valuable qualities are able to be extracted and imitated by algorithm [14]. In the future, with the further development relating to high level recognition and control, AI platforms could be an effective tool for rendering and auto editing in any device. The technology will help architects or architectural students manage the time arrangement of different projects better, and become a creation inspiration at the same time [11].

178

Y. Bao and C. Xiang

5 Discussion and Conclusion In today’s rapidly developing technological society, it is inevitable to incorporate AI programs into work processes. Various AI-aided tools for architects have already been introduced, and more are likely to be developed in the future with the further innovation of deep learning models. This survey demonstrates that AI programs can successfully generate realistic images or elegant diagrams combining with style, attribute and concept. There is no doubt that this technology has a significant impact on the daily work of architects. Traditional workflows and time arrangement for a project are likely to change soon. While it may not be feasible for AI to completely replace human architects, it can assist in relieving architects from the burdensome tasks of drawing and rendering, providing flexible digital working environment. This will enable architects to explore unlimited imaginative deign, concentrating more on creative, innovative, social sustainability, and humanistic aspects of their work.

References 1. Negroponte, N.: The Architecture machine: toward a more human environment. The MIT Press, Cambridge, Mass (1972) 2. Huang, W., Zheng, H.: Architectural drawings recognition and generation through machine learning. In: Anzalone, P., Del Signore, M., Wit, A.J., Proceedings of the 38th Annual Conference of the Association for Computer Aided Design in Architecture, ACADIA 2018. pp. 156–165. Association for Computer Aided Design in Architecture (ACADIA). Mexico (2018) 3. Song, H., Ghaboussi, J., Kwon, T.H.: Architectural design of apartment buildings using the implicit redundant representation genetic algorithm. Autom. Constr. 72, 166–173 (2016) 4. Agirbas, A.: Façade form-finding with swarm intelligence. Autom. Constr. 99, 140–151 (2019) 5. Baduge, S.K., Thilakarathna, S., Perera, J.S., Arashpour, M., Sharafi, P., Teodosio, B., Shringi, A., Mendis, P.: Artificial intelligence and smart vision for building and construction 4.0: Machine and deep learning methods and applications. Automation Construction. 141, 104440 (2022) 6. Lee, J.H., Ostwald, M.J., Gu, N.: Design Thinking: Creativity, Collaboration and Culture, 1st edn. Springer, Switzerland (2020) 7. Stable Diffusion Launch Announcement, https://stability.ai/blog/stable-diffusion-announ cement 8. Panicker, S.: AI-Inflected Art/Architecture: Who (or rather, what) is the artist/architect? BLUEPRINT SEPTEMBER 2022, 3(2), 15–36(2022) 9. Tiku, N.: AI can now create any image in seconds, bringing wonder and danger. The Washington Post, Sep. 28, 2022 10. Bankar, S.A., Ket, S.: An Analysis of Text-to-Image Synthesis. Proceedings of the International Conference on Smart Data Intelligence (ICSMDI 2021) 11. Beyan, E.V.P., Rossy, A.G.C.: Review of AI image generator: influences, challenges, and future prospects for architectural field. J. Artificial Intell. Architec. 2(1), 53–65 (2023) 12. Navidan, H., et al.: Generative adversarial networks (GANs) in networking: a comprehensive survey and evaluation. Comput. Netw. 194, 108149 (2021) 13. Frey, C.B., Osborne, M.A.: The future of employment: How susceptible are jobs to computerisation? Technol Forecast Soc Change 114, 254–280 (2017) 14. Chaillou, S.: ArchiGAN: Artificial Intelligence x Architecture. Springer Architectural Intelligence, 117–127 (2020)

The Taste of Textures Artificial Intelligence Driven Gastronomic Design and Crossmodal Correspondences in Living Art José Antonio Carrillo Andrada1(B) , José de la Rosa Morón2 and José Luis Oliver Ramírez3

,

1 American University in Dubai, Dubai, UAE

[email protected]

2 Fermented Freelance®, Huelva, Spain

[email protected]

3 University of Alicante, San Vicente del Raspeig, Alicante, Spain

[email protected]

Abstract. This research delves into the application of artificial intelligence (AI) in bio-art and gastronomy. It comprises a chapter within a broader interdisciplinary project that aims to create a dynamic, interactive, and multisensory artwork that seamlessly integrates biology, art, computational design, and gastronomy. The project investigates the interaction of microorganisms with various substrates and eco-friendly materials, the role of AI-assisted computational design, and the potential of flavour engineering to enhance the artistic experience. This paper explicitly examines the use of AI-assisted conceptualisation and computational design in generating unique flavour-related textures. Relying on gastrophysics studies, this research incorporates flavours’ objective and subjective properties into the design process. The findings underscore the effectiveness of interdisciplinary collaboration in producing artistic research and fostering innovation. Future research phases will focus on 3D modelling using parametric design and 3D printing the textures on Petri dishes, conducting experiments with microbial populations to form complex patterns and textures based on the initial AI-generated designs, contributing to the evolving nature of the artwork. Furthermore, the study will explore the audience’s engagement in interactive experiences. This study adds to the body of work on AI-driven design for practical and theoretical applications in developing innovative bio-art and gastronomic experiences. Keywords: Artificial intelligence · texture-taste correspondences · computational design · gastrophysics · bio-art

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 179–189, 2024. https://doi.org/10.1007/978-981-97-0621-1_22

180

J. A. Carrillo Andrada et al.

1 Introduction 1.1 Brief Overview of the Interdisciplinary Project This project represents an exploration of the intersection of biology, gastronomy, art, and artificial intelligence (AI). It aims to create a dynamic, interactive, multisensory artwork seamlessly integrating these fields. The central focus of this project is the innovative use of microorganisms in various substrates and eco-friendly materials, AI-assisted computational design, and the potential of flavour engineering to enhance the artistic experience. This paper uses AI-assisted conceptualisation and computational design to generate unique flavour-related textures. 1.2 Edible Bio-art, Microorganisms Beyond Fermentation. Edible Bio-art represents a novel and exciting frontier in biological art. It pushes the boundaries of what we perceive as food, exploring new textures, forms, and sensory experiences. The role of microorganisms in this process extends beyond their traditional use in fermentation. In this project, we leverage the natural behaviour of microorganisms to create unique, dynamic patterns and textures in edible substrates. Before food is tasted, the pre-oral phase—a stage where sensory expectations are formed—plays a pivotal role in shaping our subsequent taste experiences. By enhancing the visual appeal and the sensory allure of food, bio-art can potentially amplify the pleasure derived from taste. The artistic presentation of food, can evoke certain expectations and prime our taste buds for the forthcoming flavours. These implications extend beyond the art world, challenging and expanding our understanding of food and the gastronomic experience. 1.3 Importance of Food Texture in Gastronomy and Art Food texture plays a crucial role in gastronomy in general and in edible bio-art in particular, contributing significantly to the sensory experience of food consumption and appreciation (Bourne, 2002). In gastronomy, texture influences the perception of taste and palatability, with different textures often associated with specific flavours or cuisines. From a culinary perspective, texture adds a dimension of complexity, allowing chefs to create dishes that engage multiple senses and the overall dining experience (Kilcast, 2004). In the realm of art, food texture provides a tangible medium through which artists can express creativity and evoke sensory responses. Edible bio-art, for instance, leverages the diversity of food textures to create visually and gastronomically appealing artworks (Carruth, 2013). The transformative potential of food texture in gastronomy and art underscores an opportunity initiated with this research work.

2 Bio-art: Definition and State of Art Bio-art is an artistic practice that incorporates living organisms and life processes. It emerges at the intersection of biology, art, and technology, challenging traditional artistic mediums (Andrzejewski, 2018). Bio-art can take many forms, such as genetic bio-art,

The Taste of Textures

181

tissue culture bio-art, and microbial bio-art, depending on the nature of the biological materials and processes used (Noronha, 2016). The current state of the art reveals a rich variety of approaches, from artists who manipulate DNA (Genetic bio-art) or tissues (Tissue culture bio-art) to create new life forms to those who use microbes to paint living pictures (Microbial bio-art) (Noronha, 2011). This paper focuses on the last one, which uses microorganisms as a medium for creating art. It uses edible microorganisms and substrates, such us mould and yeast painting, a form of microbial bio-art that uses these microorganisms to create intricate designs and patterns. This technique involves inoculating a substrate with specific microorganisms and allowing them to grow, change and evolve over time.

3 Techniques in Edible Bio-art Pairing the suitable microorganisms with the chosen substrate is essential in edible bioart. Each microorganism, be it bacteria, yeast, or mould, has specific growth requirements and produces different visual, textural, and flavour outcomes. For example, a particular type of mould might produce a striking blue colour on cheese but not on bread (Mouritsen and Styrbæk, 2017). Each of these microorganisms has its own specific growth requirements and produces different visual, textural, and flavour outcomes. The right combination can help achieve the desired aesthetic and sensory results in the final edible bio-art piece. 3.1 Experiment 1: Edible Mould Painting, Gustav Klimt’s Kiss Using living microorganisms to generate non-preconceived designs might turn the methodology easier due to its abstract and random results. However, the control and delimitation of the growth area turn key if intentional and preconceived results are expected (Noronha, 2016). To ease the match between the mould’s growth pattern and the expected result, Gustav Klimt’s Kiss (see Fig. 1) has been taken as a preconceived design for this experiment. To simplify the pattern and design of the painting, an AI-assisted conceptualization has been used to translate the original Kiss paintings’ geometry into filamentous moulds and microorganisms-like patterns. Midjourney v5.2, a generative AI program that converts natural language text-based prompts into images, was used for this purpose, and the following prompt was inserted: “full perspective, full art, the kiss from Gustav Klimt as it was painted by growing moulds and other microorganisms such this https://s.mj.run/2QcGJD-jhqM, art designed by nature, art designed by filamentous moulds, velvety texture, high definition, 8K, realistic mould painting the kiss of gustav klimt --q 2 --v 5 --s 750”. The output (see Fig. 1) was then black/white printed in a A4 standard sheet and used as a guide for the following steps. Five different moulds were selected for this experiment, each of them manifesting a different colour: albin Aspergillus oryzae (white). Aspergillus sojae (green-ish), Aspergillus luchuensis (yellow/brown-ish), Aspergillus luchuensis var. Awamori (black), Penicillium roqueforti (grey/blue-ish). Inoculation is the process of introducing the mould spores into a culture media so that it reproduces there. Two different substrates were chosen to inoculate these moulds: water and flour for Aspergillus sp. And milk and

182

J. A. Carrillo Andrada et al.

Fig. 1. Left. Gustav Klimt’s Kiss artwork (Belvedere Object ID: 6678); Centre: Cropped study area. Right. AI-generated image of Gustav Klimt’s Kiss artwork translated into filamentous moulds and microorganisms-like patterns (Midjourney v5.2, 2023, by the authors)

flour for Penicillium sp. The abbreviation “sp.” stands for “species”. Both water and milk in combination with flour were chosen due to their capability to retain water while keeping a thick texture full of nutrients for each mould. Aspergillus sp. is traditionally inoculated on cooked grains such as rice, barley, wheat, oat, etc., essentially carbohydrates, plant-based proteins and water that are also covered by the selected wheat + water substrate. On the other hand, Penicillium sp. in this specific case P. roqueforti is usually inoculated on milk-based products for cheese-making purposes, reason why water has been replaced by milk for this mould. Both substrates were cooked to sterilize them and get a thicker paste that can then be spread on the printed sheet and create specific patterns. The thickening of the substrates ensures a higher nutrient content that still moist enough for mould growth, otherwise liquid substrates might get absorbed by the paper sheet avoiding mould growth. To get desirable substrate thickness the water/milk:flour ratio was 5:1. All the ingredients were mixed in a pot and cooked at medium heat for 10 min, then chilled to room temperature. Both Penicillium sp. And Aspergillus sp. Share 25–40 ºC as optimum temperature growth. Once the substrates were chilled, preferably under 40 ºC to avoid the spores getting burnt, they were separated into five batches of 30 g. Each batch was placed in a sterilised glass bowl and then inoculated with 0.3 g of the five mould spores. The spores: substrate ratio was significantly higher than traditional koji or cheese-making applications to ensure a more homogeneous spore distribution at the time of “painting”. 95% ethanol solution was used to sterilise five 2 mm thick paintbrushes. Each paintbrush was intended for a single substrate use to make sure no cross-contamination occurred. Hand-free technique was used to spread the substrate moulds spores on the printed sheet. The different colour decision-making was decided by the differentiation of dark (black mould) and bright areas (white mould). Green-ish, grey/blue-ish and yellow/brown-ish mould were used for blurring/grey areas. The substrates were spread on top of the printed surface with 2–3 mm thickness to ensure a higher nutrient and water content that eased the mould’s growth. Even if they were hands-free spread, a minimum of 1mm separation between defined lines and shapes was established to avoid mould overgrowths and the consequent colour migration. The already “painted” sheet was then incubated for 10 days at 30 ºC, 80% Hr and medium fan mode within a multifunctional ANOVA(r) Precision Oven. Medium fan mode was selected to ensure the correct flow of oxygen that these moulds require. A dark environment is also needed, specially to avoid photosynthetic spoilage. After four days, evidence of moulds

The Taste of Textures

183

growth was found, however, most of the surface got covered after five days. Penicillium roqueforti showed to be slower than Aspergillus sp. And the manifestation of blue/grey colour was weak (see Fig. 2). After day 10 mycelial migration was evident and colour blurring occurred. (see Fig. 2). Even though, defined growth areas were created with isolated lines and shapes.

Fig. 2. From left to right. “Painted” sheet incubation at days 5, 8 and 10 (2023, by the authors).

Overall, the final result notably matches the design created by AI and respects the dark/bright pattern (see Fig. 3). The chosen ingredients set a simple base for adding flavours, aromas and textures that may enrich the pre-intended culinary experience. Further explorations must be done to achieve a more diverse colour exploration, inoculation timings schedule, substrates, textures, flavours, aromas.

Fig. 3. Comparison. Left. AI-generated image of Gustav Klimt’s Kiss artwork translated into filamentous moulds and microorganisms-like patterns (Midjourney v5.2, 2023, by the authors); Right. “Painted” sheet incubation at day 10 (2023, by the authors).

4 Understanding Food Texture 4.1 Definition and Perception of Food Texture Food texture is defined as the sensory and functional manifestation of a food’s structural, mechanical, and surface properties as perceived by the senses of touch, sight, and hearing (Bourne, 2002). These properties include factors such as hardness, cohesiveness, viscosity, and granularity, all of which contribute to the overall perception of texture.

184

J. A. Carrillo Andrada et al.

From a physiological perspective, texture perception is mediated by a variety of sensory receptors in the mouth and hands that respond to mechanical stimuli (Bourne, 2002). Psychological factors, such as previous experiences and expectations, also play a role in texture perception. Furthermore, environmental factors, including the presentation and context of food consumption, can influence the perception of texture (Kilcast, 2004).

5 The Interplay of Food Taste and Texture 5.1 Flavours’ Objective and Subjective Properties Flavour is a multifaceted sensory perception that encompasses both objective and subjective properties. The objective properties of flavour refer to food’s physical and chemical characteristics that elicit sensory responses. These include taste (sweet, salty, sour, bitter, umami), aroma, and trigeminal sensations (such as pungency or astringency) (Mouritsen and Styrbæk, 2017). The subjective properties of flavour refer to the individual’s personal perception and interpretation of these sensory stimuli. 5.2 Cutlery’s Role in Food Taste-Texture Perception Cutlery can significantly influence the perception of food. The material, weight, size, and shape of cutlery can affect the sensory experience of eating, influencing food’s perceived texture and flavour (Harrar and Spence, 2013). The tactile experience of holding and using cutlery affects the overall sensory experience of food consumption (PiquerasFiszman and Spence, 2012). The material of the cutlery can also influence the perceived taste and texture of food. Food tasted with a rough spoon is perceived to be less creamy than when tasted with a smooth spoon (Harrar and Spence, 2013). The shape of the cutlery can also influence the perceived taste and texture of food. Spoons with a more angular shape can enhance the perceived sharpness of cheddar cheese (Mielby et al., 2018). In addition to the factors mentioned above, the temperature of cutlery can also influence the perception of food texture.

6 Artificial Intelligence and Computational Design in Gastronomy 6.1 AI’s Role in Designing Texture Patterns on Cutlery to Enhance Food Sensorial Perception The role of cutlery in food sensorial perception is a largely explored area (Spence, 2022). However, the potential role of AI in comprehending and crafting the interaction between cutlery and food remains largely unexplored. In the realm of edible bio-art, particularly in the research conducted herein, AI steered both the artwork translated into filamentous moulds and microorganisms-like patterns, as well as the generation of unique and dynamic cutlery texture designs.

The Taste of Textures

185

6.2 Experiment 2. Tasting the Klimt’s Kiss In this experiment, AI significantly contributed to the design of surface and texture patterns for five rectangular, seamless modular tiles, which will serve as pieces of cutlery. These cutlery pieces correspond to the five different moulds from Experiment 1. In future phases, the artwork will be meticulously segmented into uniform squares measuring 5x5 cm. For each discrete cell, the predominant colour will be determined and subsequently matched with the tint most reminiscent of a designated mould. Leveraging a reference image, each mould designation within the cells will be replaced with its authentic colour representation, ensuring a precise correlation with the respective mould. The moulds will be cultivated on the new cutlery shaped as 5 × 5 cm discrete cells, which will be created by 3D modelling and 3D printing the AI-assisted tiles design, and later tasted. AI was employed to conceptualise and compute unique designs for surface and texture patterns, which are associated with the colour and flavour of the moulds. Five AI prompts were crafted based on gastrophysics studies, which delve into both the objective and subjective properties of each mould’s flavours. These prompts were then implemented in Midjourney version 5.2 to generate geometric patterns aimed at enhancing the sensory experience of the edible bio-art, thereby making the act of eating more enjoyable. Phase 1. Mould Colour-Taste Correspondences. Food colour is a significant determinant in setting up our expectations for a certain taste. Our brain uses the colour of food as an anticipatory signal to prepare for the food’s taste, leading to a specific expectation. Likewise, in the context of edible bio-art, the colour of the moulds used can shape the consumer’s expectation of taste. The white colour of Albin Aspergillus oryzae may be associated with neutrality or blandness in western cultures, while in other cultures, white could be associated with specific tastes like saltiness (white salt), sweetness (white sugar) or sourness (yogurt). Green colour, as in Aspergillus Sojae, might imply freshness or vegetal flavours, and in some cases, a sour or tangy taste, as many green fruits are sour when unripe. The yellow/brown colour of Aspergillus luchuensis might suggest a more robust, earthy flavour, potentially umami, given the association of such colours with cooked, roasted, or fermented foods. Aspergillus luchuensis var. awamori, with its black colour, might be expected to have a strong, intense, or bitter flavour. This is because, in many cultures, black foods are often associated with intense tastes, like black coffee or dark chocolate. Finally, the grey/blue colour of Penicillium roqueforti may be associated with a strong, pungent, or tangy taste, considering the mould’s use in blue cheese production. Phase 2. Mould Taste-Texture and Tile Pattern Correspondences. Building upon the mould colour-taste associations outlined in the previous section, the correspondences between mould taste and texture were formulated, based on texture attributes: Hardness, Cohesiveness, Viscosity, Elasticity, and Tactile Experience. Subsequently, AI was employed to define hypotheses, derived from gastrophysics, to establish the Tile Pattern Correspondences with Mould Taste-Texture. Following this, the tile properties were outlined: Material, Weight, Size, Shape, and Visual Experience (Colour, Brightness, and Reflectivity), with the aim of refining the alignment between the mould and the cutlery—materialized as tiles in this project—that augments the mould taste, See Table 1.

186

J. A. Carrillo Andrada et al.

Table 1. Mould Taste, Texture and Tile Properties Correspondences, (2023, by the authors). Mould

A. oyzae

A. sojae

A. luchuensis

A. l. var. P. roqueforti Awamori

Ta

28–42 ºC

28–42 ºC

28–42 ºC

28–42 ºC 20–30 ºC

Hr

75–85%

75–85%

75–85%

75–85%

Time

36–48 h

36–48 h

36–48 h

36–48 h

72–96 h

Colour

White

Green

Yellow-brown

Black

Blue

Taste

Sweet salty sour

Tangy sour

Earthy umami

Bitter

Tangy

Soft-Medium

75–85%

Texture Hardness

Medium-High Medium

High

Medium-High

Cohesiveness High

Medium

High

Low

Medium

Viscosity

Medium

Low

High

Low

Medium

Elasticity

Low

Low

Medium

Low

Medium

Touch

Smooth-Creamy Rough or textured

Slightly rough

Rough

Rough

Tile/Cutlery Material

Ceramic or wood

Metal

Metal

Metal

Metal

Weight

Light

Medium

Heavy

Heavy

Heavy

Size

5 × 5 cm

5 × 5 cm

5 × 5 cm 5 × 5 cm 5 × 5 cm

Shape

Rounded

Angular or with edges

Rounded Pointed or with edges

Pointed or angular

Colour

Neutral

Neutral

Bold and Neutral rich

Neutral

Brightness

Matte

Matte/Shiny

Shiny

Shiny

Matte

Midjourney was used to create visual hypotheses and patterns for each mould within the squared 5x5 cm seamless cell units, aiming to contribute towards enhanced sensorial perception, as illustrated in Fig. 4. Repeatability tests and variability tests were employed to ensure the reliability and consistency of the tool. Subsequent are the final prompts for each mould: Albin Aspergillus oryzae, Prompt: “A light matte white parametric membrane pattern with soft and rounded shapes giving a creamy and sweet feeling, attractor effect --tile --v 5.2”. Aspergillus sojae, Prompt: “A matte white attractor effect parametric pattern with crunchy and detailed sharp polygonal edges shapes giving a strong breakable and crunchy feeling --tile --v 5.2”. Aspergillus luchuensis, Prompt: “A matte white parametric cracked earth pattern giving a rich earthy umami flavour and a chewy mouthfeel to meld in the mouth --tile --v 5.2”. Aspergillus luchuensis var.

The Taste of Textures

187

Fig. 4. Tile Pattern Design and Mold Correspondences. From top to bottom: Albin Aspergillus oryzae (white); Aspergillus sojae (green-ish); Aspergillus luchuensis (yellow/brown-ish); Aspergillus luchuensis var. Awamori (black); Penicillium roqueforti (grey/blue-ish). From Left to Right: Seamless Tile Pattern; 5 × 5 cm Unit Tile; Tile Inoculated with Mould. AI-generated images (Midjourney v5.2, 2023, by the authors)

Awamori, Prompt: “A metal ferro spikes white parametric crisp texture pattern giving a strong bitter flavour to burst in the mouth --tile --v 5.2”. Penicillium roqueforti, Prompt: “A matte white parametric seamless branching edible pattern giving a bit of crumble to release strong and tangy flavours and providing a chewy mouthfeel --tile --v 5.2”.

7 Conclusion and Future Work The blending of AI, edible bio-art, and digital design presented in this research has open new possibilities for dynamic and interactive design workflows as well as for sensoryrich artworks conceptualization. Firstly, AI played a pivotal role in output generation

188

J. A. Carrillo Andrada et al.

transforming an intricate artwork into a simpler version, leveraging learned patterns and algorithms. This capability allowed for the creation of novel artworks that retained the essence of the original while introducing a fresh bio-artistic perspective. The generation of innovative artwork interpretations might have been challenging or time-consuming through manual methods. Later, the application of AI in creating connections between colour, taste, texture and cutlery and guiding the growth of microorganisms, provided a method to combine established and new technologies. In the digital fabrication process following the presented research, these detailed characteristics will guide the creation of 3D models from their 2D images for each mould colour-taste-associated texture. The tools to be used will be parametric design software. The rapid development of AI capabilities and the more than feasible emergence of an AI program that converts natural language text-based prompts or images into sophisticated and editable 3D geometry, can complement or change the forthcoming digital design and fabrication process described here. The AI-generated image of Gustav Klimt’s Kiss artwork, translated into filamentous moulds and microorganism-like patterns, will be subdivided into squared 5 × 5 cm pixel units. Each of these units will host seamless cutlery tiles, arranged to correspond with their respective moulds. Instead of utilising the A4 sheet of paper from Experiment 1 as a support, the foundational material will be a 3D-printable seamless tile, serving both as cutlery and a pixel of the edible artwork. Initially, all tiles will be 3D printed using the same neutral colour and material to avoid influencing mould colour perception, allowing for rapid prototyping and preliminary conclusions. After 3D printing the tiles, the different substrates will be applied to them to inoculate the desired microorganisms. Once the desired results are achieved, tiles with microorganisms will be arranged to represent the Klimt’s Kiss layout physically. Furthermore, the study will explore the audience’s engagement in this interactive experience, the results will be analysed and adjustments will be made to achieve better patterns and textures based on the feedback on the initial AI-generated designs. This will allow for optimising the multifaceted sensory perception that encompasses both objective and subjective properties at the tile design and microorganism growth levels, contributing to the evolving nature of the artwork. Subsequently, with the aim of achieving the optimum sensorial experience, the tiles will be fabricated with the materials and properties delineated in Table 1. Future research could further explore the capabilities of AI in creating personalised food experiences, considering not only individual taste preferences but also dietary needs and health considerations.

References 1. 2. 3. 4.

Andrzejewski, A.: How to frame edible art. The Nordic J. Aesthetics. 27, 82–97 (2018) Bourne, M.C.: Food Texture and Viscosity: Concept and Measurement (2002) Carruth, A.: Culturing food: Bioart and in Vitro Meat. Parallax 19, 88–100 (2013) Harrar, V., Spence, C.: The taste of cutlery: how the taste of food is affected by the weight, size, shape, and colour of the cutlery used to eat it (2013) 5. Kilcast, D.: Texture in Food: Solid Foods. Woodhead Publishing (2004) 6. Mielby, L.H., et al.: The role of cutlery in the enjoyment of cheese texture. Food Qual. Prefer. 63, 135–142 (2018)

The Taste of Textures 7. 8. 9. 10.

189

Mouritsen, O.G., Styrbæk, K.: Mouthfeel: How Texture Makes Taste (2017) Noronha, P.: Biopaintings produced by filamentous fungi. Leonardo 49, 14–18 (2016) Noronha, P.: Yeast biopaintings: biofilms as an artistic instrument. Leonardo 44, 38–42 (2011) Piqueras-Fiszman, B., Spence, C.: The weight of the container influences expected satiety, perceived density, and subsequent expected fullness (2012) 11. Spence, C.: The Form of Taste (2022)

AI Machine Learning in Creative Architectural Design Processes Juan Carlos Dall’Asta(B)

and Giancarlo Di Marco

Xi’an Jiaotong—Liverpool University, Suzhou, China {juancarlos.dallasta,giancarlo.dimarco}@xjtlu.edu.cn

We belong to a generation that forged design and creative production into the digital world, and digital design represents an intersection between a classical iconographic legacy and a contemporary digital language. D. Quayola

Abstract. An increasing number of designers, architects, and artists are using new technologies and artificial intelligence as mediums, aware of their contemporary relevance. Architectural design is based on technical knowledge, accumulated experience, and an intuitive “creative” component able to express the designer’s unique talent. Artificial Intelligence provides suggestions and ideas to designers, acting as a collaborative partner in their creative phase. From this perspective, AI has been identified as a good support, able to suggest ideas during conceptualisation, one of the most critical phases of the Design process. This paper explores the integration of AI and Machine Learning style transfer in architectural education to enhance creativity and design inspiration. It comes as the result of three years of experimentation in an academic environment where Master’s students in Architecture have been asked to apply and understand Neural Networks and methods to train them to perform specific given tasks. During the numerous experimentations, an interesting evolution has been observed across the three years when users approached this innovative Design tool in architecture. AI-informed design processes have been a good practice of “learning by doing”, breaking the rule about having just one solution in architectural design processes. Machine Learning applied to creativity requires a correct understanding of the input and a good level of abstraction during the phase of interpreting the results. This paper also aims to inform the audience about the workflow of creative AI applications and the philosophy of “Creating with Artificial Intelligence.” Keywords: AI machine learning · Creativity · Architectural design · Architectural education · Style transfer

© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 190–200, 2024. https://doi.org/10.1007/978-981-97-0621-1_23

AI Machine Learning in Creative Architectural Design Processes

191

1 Introduction Over the past three years, this innovative approach to architectural design and education has provided students with novel sources of inspiration, ultimately enhancing their design imagination and conceptualisation capabilities. In a way, an architect, designer, or any other creative professional should be aware of the mystery behind ideas generation. Where do ideas come from? Understanding where ideas are coming from is surprising, even for creative professionals. At the same time, asking this kind of question signals that one’s creativity is moving forward and could bring innovation. Good practice in the past was visiting libraries, reading specialised magazines, and browsing the internet in the last decades. With the development of AI and Machine Learning, we face a new era and are forced to ask ourselves where creative ideas come from. The paper digs into the implications, benefits, and limitations of incorporating AI and machine learning into architectural education. The paper also points out how these tools can foster creativity, allowing unprecedented explorations and expanding design horizons, thus shaping the future of architectural practice. The methodology involves qualitative analysis of student direct and indirect feedback and design experimentation to assess the transformative impacts of AI on the creative process (Fig. 1).

Fig. 1. Solutions for a Gaudi’s inspired façade, AI Style transfer generated by Wenjing Wang.

The integration of Artificial Intelligence into our daily life will change our lifestyles. As already happened with other fundamental digital technologies [1, 2], the application of AI and Machine Learning in Architecture is marking a paradigmatic transformation in the creative process. One significant innovation is AI and Machine Learning style transfer: because of its visual specifics, it allows architects and designers to explore new dimensions of creativity by experimenting with AI-generated visuals and images and applying them to architectural concepts. More specifically, this paper examines

192

J. C. Dall’Asta and G. Di Marco

the implications of incorporating AI and Machine Learning style transfer into architectural education at Postgraduate levels, with a focus on its impact on creativity, design inspiration, and as a consequence, on the future evolution of the architectural practice. Over the past three years, this approach has been implemented to provide students with unexplored sources of inspiration, allowing them to broaden their design horizons and collaborate as a partner with AI systems [3] (Fig. 2).

Fig. 2. Urban Design strategies, AI Style transfer generated by Asmaa Hemaid.

Inspiration and Creativity in Architectural Design traditionally depend on the Design Culture, the complex mix of the designer’s cultural background, the environment, the local traditions and references. However, integrating AI and Machine Learning introduces a new dimension of inspiration. Students can now explore an array of conceptual variations through AI-generated visuals that they can adapt to their design concepts: this expands the sources of inspiration beyond conventional boundaries, enabling students or future practitioners to engage with ideas they might not have encountered otherwise. It is a significant evolution moving from a traditional inspirational set of tools to a brand-new constellation of inspirational ideas whose consequences in terms of creativity will soon be discovered. From this perspective, AI-driven inspirational tools act as a catalyst for architectural creativity by offering a departure from the norm. Designers and Architects can interact with AI-generated visuals that encourage innovative thinking and experimentation. The juxtaposition of AI-generated ideas with architectural elements encourages students to push boundaries, take risks, and venture beyond their comfort zones. A fundamental step forward that needs to be faced by whoever is involved in the creativity sector. This paradigm shift fosters a culture of exploration and innovation within the architectural field. Architectural education benefits from exposure to diverse design philosophies and approaches in an increasingly interdisciplinary and multi-expertise context. AI and Machine Learning experimentations open the door to an expansive collection of design

AI Machine Learning in Creative Architectural Design Processes

193

possibilities that transcend conventional boundaries. By offering access to a broad spectrum of qualitative visuals, ideas, and concepts, from minimalist to maximalist, historical to futuristic, and from simplicity to unlimited complexity, designers can explore unknown territories of architectural expression [4]. This exposure broadens their design horizons and cultivates a deeper understanding of the intricacies of architectural language and form [5]. After careful observation of the results of three years of experimentation, the integration of AI into architectural education does not open to a future replacement of human designers or architects, as many are concerned about, but rather the augmentation of their capabilities. Human-AI collaboration leverages the strengths of both parties to achieve optimal design outcomes and to go beyond imagination [6]. As students engaged with AI-generated visuals, after a short initial doubtful phase and due to their dominant curiosity, they gained an appreciation for the symbiotic relationship between human creativity and machine-generated possibilities. This explorative and thrilling partnership requires adaptability and fluency in working with AI tools, which is crucial in the evolving landscape of architectural practice [7] (Fig. 3).

Fig. 3. Inspirational concepts for Venice, AI Style transfer generated by Wenya Xue.

2 Methodology Integrating AI and Machine Learning style transfer in architectural education involves qualitative analysis and design experimentation as a research methodology. The qualitative analysis encompasses student direct and indirect feedback gathered through surveys, dedicated interviews, public presentations, exhibitions and focus group discussions. This approach captures the audience and students’ perceptions and concentrates

194

J. C. Dall’Asta and G. Di Marco

on experiences regarding the integration of AI-driven creativity into their design workflows. Simultaneously, design experimentation involves assigning projects that require the integration of AI-generated concepts and ideas. These pilot projects encourage collaboration, fostering the exploration of innovative design tools, new architectural design processes and their synthesis with traditional design principles (Fig. 4).

Fig. 4. Barcelona’s urban network strategies, AI Style transfer generated by Wenjing Wang

As previously mentioned, most of the experimentation has been done through AI and Machine Learning style transfer, referring to different platforms available during the last three years, which involves using deep neural networks to alter the style of an image while preserving its content. Convolutional neural networks (CNNs) are utilised to extract features from content and style images, which are combined to generate a new image that embodies the content of one image and the artistic style of another. Originally developed as an artistic tool, style transfer has found its way into various creative domains, including architecture. By applying this technology to architectural design, architects and students access a unique toolset that supplements traditional design approaches. Rather than asking students to select a specific “artistic style”, they have been given preference to “style transfer tools”, which allow the free choice of images, both the main and the “style” one, with enormous advantages in terms of experimentation and innovation as demonstrated from the promising results. The relevant qualitative analysis and design experimentations collection encompass student direct and indirect feedback gathered from the last Academic Years 2020–2021, 2021–2022, and 2022–2023. The cohorts, represented on average by twelve to fifteen students, were enrolled in the Master of Architectural Design Programme (MArch Des), Master Year two, ARC411 Module, delivered at XJTLU—Xi’an Jiaotong-Liverpool University, Suzhou, China (Fig. 5).

AI Machine Learning in Creative Architectural Design Processes

195

Fig. 5. Gaudi’s inspired façade process A + B = C, AI Style transfer generated by Wenjing Wang

2.1 Phase One: Testing, Analysis, and First Approach. The educational project was organised in subsequential steps. Firstly students were trained on the basic understanding of creating with AI, a general introduction dedicated to a philosophical and practical showcase of collaborative and creative AI, the basic of AI and neural networks, and the basics of Collab and programming. The first experiment pilot experimentation was creating the image through DeepDream and GANs to obtain the conceptual basics of neural networks for 2D style transfer. Students were requested to: • • • •

Use Collab to style transfer images. Upload desired images to Collab (review) Run style transfer example using Collab NST. Get to experiment with the content and “style” different weights

The purpose was to get an understanding of their effect on the output image; they were asked to produce 4–6 image pairs of Architectures and other 2D artworks and experiment relevant to make them understand what makes from an AI perspective a good ‘content’ image and what makes a good ‘style’ image. Since the beginning, students were engaged in an open discussion about what style means, with the intent to overpass this concept. This first approach phase’s expected outcome was clearly understanding the process between input and output images (Figs. 6 and 7).

196

J. C. Dall’Asta and G. Di Marco

Fig. 6. First approach, testing and exploring, AI Deepdream generated by Zihuan Zhang.

Fig. 7. Chinese landscape re-interpreted, AI Deepdream generated by Wenya Xue.

2.2 Phase Two: Understanding AI and Working with It. The real challenge when facing AI tools in the creative design process for the first time is understanding the logic behind it. After three years of experience working with almost 50 students, it is possible to prove that making creative minds ready to accept suggestions from others is a matter of mindset.

AI Machine Learning in Creative Architectural Design Processes

197

As designers, we are aware of some phenomena involving the creative moment, which we can summarise as an individual experience, and this is what often makes designers feel the loneliness of creativity, or in the other hand, the feeling of accomplishment (Fig. 8).

Fig. 8. Learning by doing, AI Style transfer generated by Lokhang Cheung.

Over the years, people can learn to share the creative process and ideas with peers, but working and sharing with “Artificial Intelligence” is a new process, also from the anthropological point of view. The very understanding of two main concepts about creativity is itself a meaningful teaching and learning experience incoming from experimenting with AI tools. The first concept is the need to interpret the interface with AI as an ongoing conversation, meaning users never get the solution directly. Still, building a step-by-step path where both actors learn from each other is necessary. In order to obtain better results, the designer needs to collaborate with the tools: machine learning and AI provide better information only if the user provides more information. It is a learning loop where the “conversation” becomes the most potent tool. The second concept is the possibility of multiple results, potentially opening the conceptualisation phase to unlimited solutions. The creative design process tends typically to focus on one solution as absolutistic. Working with AI tools helps break this limitation, providing the designer with several promising solutions and the opportunity to enhance creative skills as he is forced to leave his comfort zone. 2.3 Phase Three: Experimenting and Specializing. After users gain confidence in working with AI style transfer as a design tool during the creative design process and taking it as a “virtual partner”, the next step is the experimental and specialisation phase, where we can observe the most exciting findings from both the pedagogical and research points of view.

198

J. C. Dall’Asta and G. Di Marco

• By changing from architectural to territorial or “architectural language” scale, we have found that the tool can provide innovative ideas for the same design task at different scales; this is relevant considering a multiscale design approach where AI could help on keeping a sequential and coherent conceptual approach across the whole process, moving from territorial to architectural and even detail scale. • During the experimentation, there has been a clear evolution in observing how the criteria selection of the input images has been moving from “realistic” to abstract ones. The abstraction process allows a broad spectrum of interpretation, which is fundamental for considering a higher number of potential ideas. • Since the beginning and throughout the evolution process consisting of a deeper machine learning incremental practice, the input image selection has been refined to a more precise “design purpose” related to the user’s design strategy. From this perspective, concepts such as dissolving, blending, superimposition and layering have been experimented with as design objectives. AI tools have played a fundamental role in terms of “innovative” and potential creativity. • As per previous descriptions, an additional abstract phase has been introduced after the “realistic” one, where users “translate” the abstract outcome into realistic concepts through 3d modelling. The result is a conceptual “meta-project” or “conceptual design prototype”, ready to be translated by the designer into concrete architectural design according to the selected strategy. • A meaningful evolution in terms of input images has been observed, culminating in the self-production of “ad hoc” drawings and images to feed AI Style transfer, thus reaching the level observed during the last academic year. Physical models were also created ad hoc to achieve the results expected from working with “AI” as an integrated design team member (Figs. 9 and 10).

Fig. 9. Physical models built and processed ad-hoc, AI Style transfer generated by Yifan Wu.

AI Machine Learning in Creative Architectural Design Processes

199

Fig. 10. Digital modeling inspired by AI generated images as a conceptual model ready to be applied to Design, by Lokhang Cheung.

3 Results and Discussion The qualitative analysis of student feedback reveals that integrating AI and Machine Learning style transfer in the design process significantly impacts the creative process in architecture. Students reported increased exposure to unexpected solutions and a heightened sense of innovation. Further than working groups discussions and shared feedback, students were asked to answer the following questions: • How have you used AI and Machine Learning style transfer as a design approach? • How will AI and Machine Learning style transfer be applied to design studios and future practices? 94.73% of the students positively responded about using it as a design approach, and 89.47% see it as an essential tool for their future practice. The ideas suggested by the AI provided a fresh perspective, allowing students to experiment with unconventional design approaches. Collaborative projects further emphasised the value of human-AI partnerships, where students leveraged AI-generated outputs to complement and enhance their design narratives. Design experimentation corroborated these findings. Architectural projects that incorporated AI-generated concepts demonstrated a synthesis of traditional design principles and AI-inspired aesthetics. The projects showcased an enhanced capacity for innovation and a willingness to explore uncharted territories. Students expressed newfound confidence in using AI as a creative tool, highlighting its role in expanding their design horizons.

200

J. C. Dall’Asta and G. Di Marco

4 Future Implications and Conclusion While AI and Machine Learning style transfer offers numerous benefits, ethical considerations must be addressed. Universities are already debating about how the indiscriminate use of AI-generated visuals and concepts without understanding their cultural context or historical significance can lead to cultural appropriation or misrepresentation issues. Educators play a vital role in guiding students to employ AI technology responsibly, preserving design integrity while respecting cultural diversity. The paper emphasises the potential of AI to empower future architects with the ability to push creative boundaries and expand design horizons. Integrating AI and Machine Learning style transfer into architectural education represents a pivotal moment in design pedagogy. As AI technology matures, its impact on architectural practice becomes more profound; as researchers, we already have an agenda of future planning and innovation to be introduced in the next academic year. Architectural firms are increasingly exploring AI integration, making students well-versed in AI-driven creativity and valuable contributors to the field. In conclusion, the past three years have witnessed a transformative journey in architectural education, with AI machine learning style transfer emerging as a potent tool for enhancing design inspiration and creativity. Empowering future architects with the ability to push creative boundaries and collaborate with AI systems, this symbiotic relationship between human ingenuity and AI-generated possibilities shapes the future of architectural design.

References 1. Di Marco, G.: Simplified Complexity – Method for Advanced NURBS Modeling with Rhinoceros, Le Penseur, Italy (2018) 2. Di Marco, G.: A Reasoned approach to the integration of design and fabrication technologies in architecture education. In: 2019 XJTLU International Conference: Architecture across Boundaries, pp. 510–521. KnE Social Sciences (2019). https://doi.org/10.18502/kss.v3i27. 5553 3. Gatys, L.A., Ecker, A.S., Bethge, M.: Image style transfer using convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016) 4. Menges, A., Ahlquist, S. (Eds.): Computational design thinking. John Wiley & Sons (2011) 5. Wang, J., Yuan, C.A., Yang, X., Liu, Y.: Architectural design in the age of AI: A Survey. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 39(9), 3124–3137 (2020) 6. Bigham, M., Sheppard, S., Natarajan, S.: Design and digital creativity: an intersectional exploration of the built environment. In: Design Studies 63, 102–118.(2019) 7. Kolarevic, B. (Ed.): Architecture in the digital age: Design and manufacturing. Taylor & Francis (2003) 8. Harfouche, A., Nasr, W., Melhem, R.: From design ideation to architectural form: an AI-based approach. Design Stud. 56, 55–81 (2018) 9. Olah, C., Mordvintsev, A., Schubert, L.: Feature visualization. Distill. (2017) 10. Savage, N.: How artificial intelligence is changing architecture. In: Architectural Digest (2018)

An Assessment of Thermal Comfort in Urban Quality of Life in Architecture Using Fuzzy Logic in Decision Making: A Case Study of Iran Alireza Gogani1(B) , Faezeh Choobkar2 , and Asli Cekmis1 1 Istanbul Technical University, Maslak, Istanbul, Turkey

[email protected] 2 Iran University of Science and Technology, Resalat, Tehran, Iran

Abstract. Thermal comfort is a notion that has been discussed recently in various studies, in particular architecture and urban planning as a response to many problems facing new cities all over the world and it is one of the factors that plays a very important role in the quality of life in architectural and urban design process. Evaluating thermal comfort based on climatic conditions assist architects and designers in choosing suitable regions to shape cities and buildings. In this context, Iran as a country with different climatic conditions is proposed to be analyzed and visualized as a case study. Fuzzy logic approach is used in this study as the methodology in conjunction with its inputs that are defined as air temperature, air quality, and humidity which are known as indispensable principles in architectural and urban design, and the output is determined as thermal comfort in each province, ranges between 0 and 1 (from not preferable to the most preferable provinces). This paper aims to throw light on the importance of fuzzy logic approach in architecture and urban planning and the process of analyzing thermal comfort of each region for users by means of MATLAB (a fuzzy logic-based software). For this, after an introduction and overview on fuzzy logic method and its application in this study, the results of analysis obtained are collected in tables and visualized as figures. Keywords: Climatic analysis · Fuzzy logic · Thermal comfort · Decision making

1 Introduction The multidimensional character of the concept of quality of life (QOL) and its evolutionary nature raises significant difficulties in its evaluation and monitoring which are seen as an important support to urban planning and management [1, 2]. Urban QOL can be defined as the general well-being of people and societies living in cities and the quality of the environment in which they live [3, 4]. As it is known, the climate parameter has undoubtedly been an effective factor in the basic needs of people such as shelter, nutrition, and settlement for centuries. Many factors have caused increases in air temperature and climatic deterioration. Especially the temperature increases in the cities lead to the formation of heat islands, which is the most striking indicator of the © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 201–208, 2024. https://doi.org/10.1007/978-981-97-0621-1_24

202

A. Gogani et al.

urbanization phenomenon. It has been observed that the heat island affects the thermal conditions of people as well as the air quality in the cities [5]. In a recent comprehensive study [6], Gholami et al. reviewed the most prominent photovoltaic potential assessment in Iran. By studying the related literature, a lack of in-depth potential assessment studies in Iran can be observed. Furthermore, the limited publications usually were emphasized on the energy supply and management [7] and mainly considered the technical points of view while neglecting the climatic and ecoenvironmental criteria or just studied a particular part of the country and not the whole during the architectural design and urban planning process. Although quality of life has been investigated extensively in the past few decades, QOL as a discipline is still embryonic [8] and architects and urban designers’ decision on which regions are more preferable to base cities and building sites in them in order to increase the quality of life are overlooked. To propose a solution to this issue this paper presents a comprehensive assessment of thermal comfort as one of the factors in the quality of life in Iran, focusing on climatic conditions in architecture and employing fuzzy logic in decision-making. In this study, data from 31 provinces are collected and used to generate an output based on the evaluation of factors of each region. Parameters are used in the model to evaluate thermal comfort as inputs are first, air temperature which is considered as one of the main factors in every urban planning and design process. Second one is air quality, which seems as the most prominent factor for a city. The last one is humidity, in city planning and urban design, considering humidity is essential for creating comfortable and sustainable living environments. High humidity can impact the overall quality of life, human health, and the functioning of buildings and infrastructure. The output variable which can refine initial concept and input with modifying them. This further influence the perception of quality of preferred urban spots for designers which fluctuates between 0 and 1. Final results present from a heat map visualization of preferred urban planning and designing pattern in Iran result from MATLAB output.

2 Methods and Frame of Work 2.1 Fuzzy Logic System One of the mathematical models to compute uncertainties is Fuzzy logic. Fuzzy logic and sets were first introduced by Zadeh in 1965 to represent and manipulate imprecise data. Fuzzy logic [9]. This approach has been an alternative to classical Boolean logic for problems where uncertainties in means of imprecision, vagueness, imperfect knowledge exist. In Zadeh’s theory, objects of the sets are represented with their membership degrees to that set and fuzzy sets were first introduced by Zadeh as a generalization of conventional sets theory [12]. Zadeh tells that fuzzy logic is a methodology where words are used in place of numbers for computing. In that sense, fuzzy mathematics could replicate human thinking and reasoning perfectly [11]. Zadeh’s fuzzy logic provided the means of expressing linguistic rules in such a form that they might be combined into a coherent control strategy [12]. Since then, the decision making with linguistic variables has attracted a lot of researchers’ attention [13].

An Assessment of Thermal Comfort in Urban Quality

203

In this paper, the design process of fuzzy inference system is described through the following steps: • Determine the input and output variables and make a correlation between them • Using Fuzzy Logic for data processing • Modification of design process and outputs in order to achieve desired factors in inputs In Fuzzy logic, the truth of any statement becomes a matter of a degree. Fuzzy inference system is used to determine the value of each spot in the library layout plan and in the following generate the efficient preferred spots. This process will determine output by examining each rule and find the average suitable output. Figure 1 provides a conceptual diagram of the design process of control system.

Fig. 1. Conceptual diagram of the design process of control system [14]

2.2 Input and Output Variables Three parameters as inputs used here in the model to evaluate life quality in each province are: 1. Air Temperature range: according to the ranges of the thermal indexes predicted mean vote (PMV) and physiological equivalent temperature (PET) for different grades of thermal perception by human beings and physiological stress on human beings created by Matzarakis and Höppe, it has been revealed that people feel better between 18.0 and 23.0 °C (comfort zone) air temperature values. The increase or decrease in these values causes people living in the city to feel more stressed mentally and to feel a decrease in their desire to work, while physical health problems such as eye burns and nosebleeds occur [15]. 2. Air Quality: air quality significantly affects the quality of life in cities. Clean and healthy air is essential for the well-being of residents, as poor air quality can lead to various health issues and environmental problems. The standard for air quality in cities is often measured using the Air Quality Index (AQI). The AQI is a numerical scale used to communicate how polluted the air is and what associated health effects might be a concern. The AQI typically ranges from 0 to 500, however, an AQI value of 50 or below represents good air quality, while an AQI value over 300 represents hazardous air quality. 3. Level of humidity in the air: outdoor relative humidity levels can range from as low as 10% in arid regions to over 90% in tropical or coastal areas. However,

204

A. Gogani et al.

for most people’s comfort, outdoor humidity levels in cities are considered pleasant and acceptable when they fall within the range of 40% to 70% [15] (Table.1). Table 1. Inputs and output Inputs

Outputs

Air Temperature (too cold—comfortable range—too warm)

Preferred provinces in terms of their life quality based on thermal comfort for users: Range: 0–1 Air Quality (good—moderate—unhealthy—hazardeous) (not preferable to the most preferable) Humidity (dry air—sufficient range—moist air)

2.3 Frame of the Model Fuzzy inference interprets the values in the input vector and, based on user-defined rules, assigns values to the output vector [16]. In this paper, thermal comfort as an output is assigned to provinces of Iran based on three abovementioned parameters as inputs. Inputs and output if required are granulated by fuzzy subsets, therefore a numeric input value gains a linguistic value, which is called fuzzification. By using fuzzy subsets (membership functions: MFs) and logical operators (and, or, not), if-then rules are listed to formulate the conditional statements between inputs and outputs. They are subjective natural arguments contextually composed in language. Fuzzy inference system evaluates those control rules, and calculates an output due to which rules are triggered more by given input values [11]. Using the fuzzy system in the model brings the advantage of a gradual representation of the life quality on each certain area in the provinces of Iran, instead of abrupt changes or binary results of e.g., either good or bad. For example, If the air temperature range is sufficient (20 °C) and air quality is good (AQI: 0–50) and humidity is in comfortable level (40%): then the province and its area are the most preferable (the higher membership function(blue)). Regarding the example, in this paper, each province has a valued quantity between two numbers based on the definition of each input and 36 rules were formulated in this survey, as following: It is obvious that sufficient temperature in each urban region has a benefit for users living in that area, and can be numerically formulated with measurement in fuzzy mathematics. Less temperature or more humidity decline the output value and vice versa. When the input and output is granulated with membership functions (MFs: fuzzy subsets) conditional statements between them are written in the following form: If (input is MF x), then (output is MF y), like: “If the temperature is very low, then the advantage value is very low”. Generally, more than one input is in the operation and they are connected by logical operators–and, or, not–where needed. The parameters and rules

An Assessment of Thermal Comfort in Urban Quality

205

are not fixed and change to suit different layout designs, which prompt the flexibility of the model. The inference process determines an output by evaluating each rule, and the final output of the FIS is the weighted average of all rule outputs. In this case, each given province has a crisp single-valued quantity: 0 ≤ x ≤ 1 at the end, expressing their advantageous value. When the overall calculation process is completed, a list of outputs is produced as a precursor to visualization [16].

3 Case Study The data gathered in a year in the provinces of Iran since 2022 is regulated in FIS and the proposed model is applied on 31 different provinces of Iran as areas in order to design urban areas for settling users as it is shown in the Fig. 2. For this case, Fuzzy Inference System was built in MATLAB as a three-input, one-output, and 36 rules. In order to calculating each point in fuzzy operator, the inputs were transformed to fuzzified known as fuzzification: input to input membership function(mf) then to outputs through evaluation of the fuzzy control rules: input(mf) to output(mf), and finally to an output known as defuzzification: output(mf) to output. For this model, 31 provinces of Iran were chosen which then prepared to be analyzed based on the designer’s preferences on where to set urban sites for the users with determined inputs and output as can be seen in the Fig. 2. In mandani type Fuzzy Inference System, the output that was about the study value of a spot, was also defined by subsets from the lowest membership to the highest membership: mf1: [0–0.25]not preferable, mf2: [>0.25–0.25–0.5] less satisfied, mf3: [0.25–0.5–0.75]acceptable, mf4: [0.5–0.75–0.75 comb -> harmonizer -> cross Synth (with tape, Player 8) -> spectral delay Output: comb, cross Synth, spectral delay (but not harmonizer).

Fig. 3. One possible audio chain routing

Such a high flexibility in the audio chain was designed to render an always surprising and unpredictable response of the system (Fig. 3).

An AI-Mediated VR Sound Installation

231

Machine Learning Control The audio chain and spatialization tools require to control almost 600 parameters at once (although rarely the effects are all turned on at the same time). It would be impossible for anyone to directly control that number of parameters. As shown in the background section, one possible use of machine learning consists in dimensionality reduction. Therefore, the decision to use a multilayer perceptron to perform a regression task capable of “mapping” 7 input parameters to hundreds of output parameters. The chosen neural network is a multi-layer perceptron included in ml.mlp, an object for Max/MSP from the library ml.star 2 . It can be trained with a very low number of training vectors. For this project, 8 vectors (including input and output dataset) were used. The algorithm would then iterate the training on these vectors (choosing randomly among them at each iteration) until either desired error value or the desired maximum number of iterations are reached (Fig. 4).

Fig. 4. Flowchart representing the dataflow in Oracle. In the audio module, the voice input is always fed into the random loop. After that, the audio flow is controlled via a routing function. Any mono effect can feed into any other mono or cloud effect, or can be sent directly to the spatializer. Cloud effects cannot feed in any other effect and go directly to the spatializer. The routing is controlled by the ML algorithm, as well as the control parameters of any audio effect (and of the spatializer). The VR module sends the input vector for the ML control.

The creation of the training set required the creation of a dedicated architecture. First of all, a system able to store the current state of the whole system (the value of each parameter). Then, the creation of a high number of presets, so to facilitate a swift change of state for each tool, as well as to favour troubleshooting and experimentation. One 2 From Benjamin Smith, https://www.benjamindaysmith.com/ml-machine-learning-toolkit-

in-max.

232

G. Santini and Z. Chen

crucial step in making the neural network control work consisted in the normalization and correct identification of range of values allowed for each parameter. As audio chains can be extremely sensitive to parameters outside a workable (or even “safe”) range, value ranges were set for each parameter. Then, the current value of parameter was re-scaled between 0–1. The process allows one to control the whole number of parameters with only 7 input values. 3.2 The VR Module The VR module consists of an application realized with Unity. The main function of the application is to provide the visual rendering, tracking, controller interactions. The application makes use of OpenXR and is therefore swiftly adaptable to numerous different VR headset/controller types. The visual design of the VR environment has been crafted to resemble the connection to ancient Greece while at the same time placing it in a “post-technologic” landscape. There are remains of what could have been the ruins of a Greek temple, but everything seems to happen inside a malfunctioning machine, full of glitches. From a strictly technical points of view, the VR environment has two functions: • Sending information to the audio module (through OSC), according to interaction data • Receiving information from the audio module (through OSC), according to sound spatialization data; consequently, instantiating particle effects in the positions where virtual sound sources are spawned. With the current design, the user can interact with the interface to activate a question and with five levers set on an altar. The five levers send input data for the machine learning algorithm. Anytime one lever is moved, the current state of all levers is sent to the audio module. To make the response of the oracle unpredictable, anytime there is an incoming vector, two more random variables are created and added to the stream of values. This way, the value of 7 input values is reached. Then, the control values for effects and spatialization are produced by the neural network. Spatialization coordinates are translated into “xyz” cartesian coordinates and sent back to the VR module. The audio module also receives the listener’s orientation quaternion from the VR space. When the VR module receives the spatialization data, it spawns particle effects in the position indicated. Although the theoretical mapping of coordinates should be 1:1 between, it felt like a mapping of 2:1 or 3:1 could look more convincing.

4 Discussion In this installation, the concept of questioning is looked at from a specific point of view: the question defines the scope and the limits of the answer. We could say that a question contains an interpretation of the world, a frame in which reality is inscribed. In some way, a question is to our life what an experiment is to the quantum realm: observation changes the result, a question changes reality. In this sense, the question is not only interrogating the world, it also gives information about the subject. Especially true, if

An AI-Mediated VR Sound Installation

233

that question is posed to an oracle. That is a question the subject needs an answer for. An important one. How much does the answer matter? And how important is to understand the reason the question is asked, instead? Maybe this is the meaning of the deceiving answers of the ancient Greece’s oracle. The experience has been designed to portray this artistic suggestion. The continuous processing of the voice follows from an interest in sound itself and is meant to motivate the user to explore sound and sound space. Beyond this purely perceptual layer, the meaning of continuously re-proposing the question under new lights is a symbol of the continuous search of the self, analyzing the way one looks at the world by researching the question, more than the answer. The question is about identity. The work described in this paper is still open to developmental improvements and the authors believe that the extension of a highly transformative environment to VR visuals would create a more engaging experience as well as a stronger representation of the core concept. On the contrary, the simple, almost ascetic interaction design could reduce the cognitive load, allowing the user to focus their attention on listening and observing. However, these assumptions will need to be tested with public audiences. An aspect worth mentioning is the amount of control the user has on the ML algorithm, and the perception of control. As mentioned earlier, the user can only control 5 of the 7 input parameters. The remaining two are generated randomically for any input vector. This will always result in the perception of a pseudo-randomic result. In fact, the same levers configuration in the VR space will not necessarily render the same effect chain configuration in the audio module. The user will perceive they can change the response of the oracle, but they cannot control how it changes. That is wanted by design and derives from the idea of the oracle as an entity whose responses are mysterious and unpredictable. It should not be perceived as an instrument (deterministically controllable), but an entity, with its own will. The audio module developed for this installation (with its ML control system) has also been re-adapted for other uses. For example, it has been applied into Giovanni Santini’s Teatro dei Portali, a series of XR etudes for string trio, XR performer, live-electronics, multiple projectors and 3D spatial audio premiered in Hong Kong in October 2023. The series of etudes collects several different technical solutions. In one piece (LINEAR), the ML tool was fully controlled by using 7 different movable virtual squares shaping a particle fountain. By moving the platforms up and down, the performer could achieve full control over the processing chain. However, the complexity of the series of controls (the fountain was only a part of it) would make it very difficult for the performer to acquire full awareness of the controls. In other pieces (e.g. GesturAR), the ML algorithm would “improvise”, by simply receiving random collections of 7 parameters at randomized times. A way to reduce the sense of complete randomness was to limit the range of each parameter for different sections.

5 Limitations and Future Work The work has not been presented in public yet and therefore this study cannot produce an evaluation of the effectiveness of the work, especially in regard of its capability to inspire both listening habits and reflection in terms of what exposed in the previous paragraph.

234

G. Santini and Z. Chen

A thorough evaluation could help to assess the impact of the work on audiences and better inform future refinements of the design. The installation at the moment does not include HRTF selection for the binaural rendering. At its current stage, the work has very limited capabilities in terms of visual modification of the environment. It is opinion of the authors that the work would necessitate a more balanced relationship between audio and visuals. While the audio engine can produce a large number of different outcomes, the VR module is simply placing particle effects to better render the sense of position of sound sources. The use of interactive shaders, a more varied collection of particle effects and visual algorithms which, similarly to the audio component, necessitate of hundreds of parameters could enhance the expressive capabilities of the work. To proceed with the enhancement of visuals, a necessary step would be a very thorough optimization of the audio module, which can almost saturate CPU usage in most intense workloads. That leads to occasional audio clicks. Optimization strategies will include the porting of some sound effects into the more efficient gen ~ extension for Max/MSP, deactivation of graphical interfaces, useful for debugging but expensive in computational terms, reduction of independent voices performing FFT processing, wider use of multi-threading with a wider adoption of the poly ~ object. One improvement could be made to the pseudo-randomic treatment of the input vector generated from the VR space. In particular, the training could be made in such a way that the 2 random variables yield only minor modifications to the sound processing chain. This way, the user could feel more control over the sound result, although not full control. The final intention would be to extend this work to an XR dimension, where spatial audio and multiple projections could be deployed, and to extend the number of participants to 2, while making the installation also available to audiences not wearing a headset. The audio module with its ML control could also be used for other types of installation or for live-electronics situations, similarly to LINEAR (mentioned in the previous section). However, the XR design should be improved in order to favour a full sense of control over the audio chain (either by limiting the controls to only the 7 parameters of the ML algorithm or by creating some visual referencing for debugging the status of the audio chain).

6 Conclusions This paper has presented the VR/AI installation Oracle, where the user, wearing a headset and a lavalier microphone, is transported in front of a holographic oracle, to which they can pose questions. The reply is not an answer, but rather a continuous repetition of the question, modified through a fluid chain of sound effects controlled by a multilayer perceptron neural network. The response of the oracle can be limitedly controlled by the user by moving levers in the VR space. Technically speaking, the system is composed of two modules: audio and VR. The two modules communicate via OSC protocol. The levers data, along with the listener’s

An AI-Mediated VR Sound Installation

235

orientation, are sent to the multi-layer perceptron neural network in Max/MSP, where the small input vector is exploded into hundreds of parameters controlling audio effects and spatialization. Spatialization data are sent back to the VR module, where particle effects are spawned or positioned accordingly. The visual rendering of the transformative process is very limited at the moment, when compared to the flexibility of the audio output. The authors plan to enhance that part by adding interactive shaders and more articulated particle effects which could more sensitively pick up parameters of the audio chain. To achieve that result, a thorough operation of optimization of the audio process would be needed. The effectiveness and expressivity of design solutions for the work will be assessed in public presentations getting evaluation feedback from audiences. Web link to a short demo: https://www.dropbox.com/scl/fi/l5ultiqwlrm55wxdb7swk/ Oracle.mp4?rlkey=n7zi68cxelkh93q2relurusgi&dl=0.

References 1. Jourdan, T., Caramiaux, B.: Machine learning for musical expression. In: New Interfaces for Musical Expression (NIME). Mexico City (2023) 2. Donahue, C., McAuley, J., Puckette, M.: Adversarial audio synthesis. In: 7th International Conference on Learning Representations, ICLR 2019 (2019) 3. Engel, J., Agrawal, K.K., Chen, S., Gulrajani, I., Donahue, C., Roberts, A.: Gansynth: Adversarial neural audio synthesis. In: 7th International Conference on Learning Representations, ICLR 2019 (2019) 4. Tahiro˘glu, K., Kastemaa, M., Koli, O.: Al-terity: Non-rigid musical instrument with artificial intelligence applied to real-time audio synthesis. In: Proceedings of the International Conference on New Interfaces for Musical Expression (2020) 5. Fiebrink, R.: Wekinator homepage, http://www.wekinator.org 6. Ronchi, G., Benghi, C.: Interactive light and sound installation using artificial intelligence. Int. J. Arts Technol. 7, (2014). https://doi.org/10.1504/IJART.2014.066456 7. Schacher, J.C., Miyama, C., Bisig, D.: Gestural electronic music using machine learning as generative device. In: Proceedings of the International Conference on New Interfaces for Musical Expression (2015) 8. Nyström, E.: Intra-actions: experiments with velocity and position in continuous controllers. In: Proceedings of the International Conference on New Interfaces for Musical Expression (2020) 9. Serafin, S., Erkut, C., Kojs, J., Nilsson, N.C., Nordahl, R.: Virtual reality musical Instruments: state of the art, esign principles, and future directions. Comput. Music. J. (2016). https://doi. org/10.1162/COMJ_a_00372 10. Lee, M.: Entangled: A Multi-modal, Multi-user interactive instrument in virtual 3D space using the smartphone for gesture control. In: Proceedings of the International Conference on New Interfaces for Musical Expression (2021). https://doi.org/10.21428/92fbeb44.eae7c23f 11. Santini, G.: Synesthesizer: Physical modelling and machine learning for a Color-based synthesizer in virtual reality. In: lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (2019). https://doi.org/ 10.1007/978-3-030-21392-3_18 12. Zwicker, E.: Subdivision of the audible frequency range into critical bands (Frequenzgruppen). J. Acoust. Soc. Am. (1961). https://doi.org/10.1121/1.1908630

Space Narrative: Generating Images and 3D Scenes of Chinese Garden from Text Using Deep Learning Jiaxi Shi

and Hao Hua(B)

Southeast University, Nanjing, China [email protected]

Abstract. The consistent mapping from poems to paintings is essential for the research and restoration of traditional Chinese gardens. But the lack of firsthand material is a great challenge to the reconstruction work. In this paper, we propose a method to generate garden paintings based on text descriptions using deep learning method. Our image-text pair dataset consists of more than one thousand Ming Dynasty Garden paintings and their inscriptions and postscripts. A latent textto-image diffusion model learns the mapping from descriptive texts to garden paintings of the Ming Dynasty, and then the text description of Jichang Garden guides the model to generate new garden paintings. The cosine similarity between the guide text and the generated image is the evaluation criterion for the generated images. Our dataset is used to fine-tune the pre-trained diffusion model using Low-Rank Adaptation of Large Language Models (LoRA). We also transformed the generated images into a panorama and created a free-roam scene in Unity 3D. Our post-trained model is capable of generating garden images in the style of Ming Dynasty landscape paintings based on textual descriptions. The generated images are compatible with three-dimensional presentation in Unity 3D. Keywords: Traditional Chinese Garden · Landscape Painting · Deep learning · Diffusion Model Virtual Reality

1 Introduction In the context of ancient Chinese gardens, the narrative subject embodies a complex composition, incorporating multiple roles such as the creator, experiencer, and interpreter [1]. Based on the different identities and forms of works, creative endeavors can be broadly categorized into three types: lyrical poems expressing the sentiments of garden owners, articles detailing visitors’ experiences, and visual depictions that recreate human sensory perceptions. Despite the varied narrative subjects, their creations converge upon the same object, namely, the abstract representation of tangible garden spaces. Hence, we can establish a three-dimensional perception of a specific garden space through the mutual corroboration and supplementation of the content presented from the perspectives of different narrative subjects in their works. Therefore, comparing information in © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 236–243, 2024. https://doi.org/10.1007/978-981-97-0621-1_28

Space Narrative: Generating Images and 3D Scenes

237

garden paintings and poems has become an important method of analyzing and restoring traditional gardens [2]. For example, the materials survived from Ming and Qing dynasties of Jichang Garden (Wuxi, Jiangsu province) provided a reliable basis for the later reconstruction [3]. The traditional research methods highly rely on experience and professional knowledge based on long time research and practice. However, it is difficult to reconstruct the garden landscape that never appeared in any graphic material. In order to solve the problems of data scarcity of single garden restoration, we use a multimodal model to create the text-image correspondence of gardens in a particular period. By learning the correlation between graphic and textual data, the model can deduce garden images based on textual information.

2 Related Work Comparing landscape painting of ancient Chinese literati and poems is an important method in garden verification and restoration. Due to the variety of firsthand materials and the complexity of operation, the traditional method is more suitable for single case rather than batch studies. Since 21th century, the digital technologies such as remote sensing, 3D reconstruction, and virtual reality have been widely employed for landscape heritage conservation [4, 5]. But it is still difficult for these digital technologies to recreate unique historical landscapes in a specific period. The generative machine learning model can generalise the character of training examples and create similar new data. In recent studies, pix2pix model is used to learn the topological relationship between the elements of the classic Chinese gardens and generate garden layout for a given site condition [6]. These methods focued on images and overlooked valuable text materials. Combining cross-referenced images and texts can provide a more reliable basis for studying classical gardens. Machine learning models have been used to develop a method for analyzing Chinese garden design based on textual descriptions since 2003. The first research involved the conversion of garden element images into geometric shapes and the establishment of a relational diagram between garden spatial elements with descriptive text information [7]. Transformer and subsequent research efforts have established connections between images and text since 2017 [8]. With DALL·E model one can generate semantically segmented garden plan according to the text describing classical gardens [9]. Diffusion models leverage iterative processes for more realistic and diverse image synthesis [10]. The pre-training models based on the latent text-to-image diffusion model, such as Stable Diffusion and Midjourney, can produce images which are highly consistent with the text semantics [11]. These works shed light on applying text-to-image models to generate garden landscapes from textual data, which can provide an efficient and easy-to-use method for the large batch of garden landscape research.

3 Method This paper proposes a method of generating garden paintings based on textual descriptions using the latent text-to-image diffusion model. The overall process includes the following steps:

238

J. Shi and H. Hua

• Dataset creation: Collect the Ming Dynasty landscape paintings and related description texts and establish the image-text pair dataset. • Pre-trained model fine-tuning: Input the image-text dataset into the pre-trained model and freeze the original model parameters. Use the LoRA method to fine-tune the attention processor in U-net. • Testing and evaluation: Input the text describing Jichang Garden in the late Ming Dynasty to the fine-tuned model to guide it in generating landscape paintings, and carry out a similarity test on the generated results. • VR representation: Select a group of generated results and create a walkable continuous scene in Unity 3D. 3.1 Text-Image Paired Dataset Creation Data Collection. We first collected about 3000 landscape paintings and corresponding text description information from the Palace Museum, the National Palace Museum, Wuxi Museum, and Nanjing Museum through the Internet. Considering the impact of sample quality on the effect of machine learning, we filtered and modified the dataset according to the following conditions: • The image should have clear architectural element. • The image quality must reach six million pixels. • For images that meet the above conditions but lack relevant descriptive text, the data will be discarded. Based on the above conditions, we ultimately selected 1182 pairs of data and stored them as metadata in a format suitable for Diffusers Pipeline. Image Data Scaling. Considering the pre-training model structure, we scale all images to 512 × 512 pixels. Table 1 below provides sample data 3.2 Fine-Tuning Latent Text-to-Image Diffusion Model with LoRA Latent Text-to-Image Diffusion Model Architecture. The Latent text-to-image diffusion model is a machine learning system that gradually denoises random Gaussian noise in the latent space of lower dimensions until clear data (such as images) is obtained. The model is composed of three parts: the Variational Autoencoder (VAE) used for analyzing and generating images, the U-Net used for calculating noise residuals, and the CLIP text encoder used for converting text prompts into vectors in the latent space. The pre-training model actually used in this paper is stable-diffusion-v1-5 (512 × 512 resolution). In the process of using the text-guided model to generate images, the model first converts each input text information into a 77 × 768 text embedding through the CLIP text encoder. Then, it takes the text embedding as a condition and uses U-Net to denoise the noise vector of the representative image, outputting the predicted noise residual. Afterward, the scheduler algorithm (in this paper, we use the preset PNDM scheduler) is employed to calculate the denoised image vector in the latent space based on the previous noise vector and the predicted noise residual. Finally, the VAE decoder is used to restore the vector representing the image to a full image of 512 pixels.

Space Narrative: Generating Images and 3D Scenes

239

Table 1. Sample Data

Image

Metadata

{"file_name": " 000914N000000000.png", "additional_feature": " A figure reclines on a couch in the pavilion, whilst the figure in the boat plays on a flute and dangles his legs in the water. "}

{"file_name": " 000623N000000000.png", "additional_feature": " Cloud Forest Painting. To be clear and vulgar is to have nothing. The Xuanzai in this picture is accompanied by Dong Ju and Yunlin."}

{"file_name": "000626N000000000.png", "additional_feature": " White clouds skirt the mountain; houses, where gentlemen sit talking leisurely, lie half-hidden by the thick forest. "}

Fine-Tune with LoRA. LoRA is a method to accelerate large-scale models’ training with less memory consumption, which frees the trained model weights and inputs trainable rank decomposition matrix into each layer of the transformer architecture to reduce the number of trainable parameters. The total parameters of these rank decomposition matrices || are far less than the total parameters of the pre-training model |0 |. Parameters used during training  re-express the parameter increment  of the model as  = () and constantly update the parameters in the rank-decomposition weight matrix to make the generated results similar to the data in the training set as much as possible. The optimization objectives are as follows:  |y|   log p0 +() (yt |x, y 20% greenery and > 10% seating. Through counting grid numbers, the environment comfort of the design can be evaluated by the proportion of greenery, trees, seating, and shades (Table 5). Table 5. Environment comfort checklist. Greenery

(✓)

Trees

(✓)

Seating

(✓)

Shades

≥ 11 grids

≥ 6 grids

≥ 7 grids

≥ 4 grids

7–10 grids

4–5 grids

5–6 grids

3 grids

3–6 grids

2–3 grids

2–4 grids

2 grids

1–2 grids

1 grid

1–2 grids

1 grid

(✓)

Adaptability of space can improve the level of inclusion and relates to the amount and size of free spaces. In which, passive surveillance can ensure safety through a straight line-of-sight. An average person has a line-of-sight range of 1.3–1.9m, and there should be no obstruction within this range. It can be standardised into degrees of obstruction: ≥ 50%, 20%, and no occlusion (Fig. 2). The smaller the value, the higher the degree to which the site promotes "eyes on the street". Accessibility and connectivity of a site contributes to wayfinding. To analyse the ease of wayfinding, width and twists & turns of a path can be measured (Fig. 3) (Tables 6 and 7). Table 6. Adaptability & surveillance checklist. Size of free space

Presence (✓)

Straight line of sight: 1.3–1.9m

≥ 7 grids

Unobstructed

4–6 grids

20%

1–3 grids

≥ 50%

Fig. 2. Spatial configuration analysis of surveillance.

Presence (✓)

278

S. Zhu et al. Table 7. Wayfinding checklist.

width of pathway

Presence (✓)

twists & turns of pathway

≥ 3m

0

1.5-3m

1–2

0–1.5m

≥3

Presence (✓)

Fig. 3. Configuration analysis criteria of “wayfinding”

5 Preliminary Testing: Applying the Toolkit A test was held with 15 design students working in four teams in shared VR spaces (Fig. 4). After identifying common design goals, students cooperated through a division of tasks to place elements in VR. Researchers analysed results by comparing hand drawings, thematic content analysis of presentations, and VR outcomes. Based on which, four open space designs were proposed, evaluated by the toolkit (Fig. 5). “Complexity of activities” performed best, scoring an average of 71% from all designs, followed by adaptability (63%), sociability (55%), wayfinding (54%), environment comfort and surveillance (50%). All designs had at least six types of activities except for design D, most of which covered 1–2 grids. When trying to enhance other criteria, activity complexity would decrease; for instance, design D scored highest in “environment comfort” and “sociability” with high percentages of greenery and alfresco seating, resulting in lowest score in “complexity of activities. These mutual-constrain indicators presented players with an exercise of spatial trade-off. Overall, proposal B scored highest with 43/64 (67%), followed by C & D (55%), and A (45%). Especially for adaptivity, surveillance and wayfinding, B designed with fewer elements to support more flexible common spaces, wider and straighter pathways, whereas A had low accessibility with semi-open activity designs. Although B was a better design solution quantitatively, graphically, it can be seen how A facilitates better

Structuring Gamified Participatory Public Space Design Developing

279

environment comfort by putting most activities under shades of adjacent buildings in the west, and better surveillance by putting alfresco in the east next to existing ground floor shops (Fig. 6). The combinatorial evaluation method showed how graphical analysis can complement quantitative scoring to provide a more comprehensive assessment of spatial quality and serve as guidance for players with design-educational significance.

Fig. 4. Students worked in teams within networked VR scenes.

Fig. 5. Quantitative analysis of collaborative design outcomes using the proposed toolkit.

280

S. Zhu et al.

Fig. 6. Graphical analysis of collaborative design outcomes using the proposed toolkit.

6 Conclusions This paper investigated a means to guide and evaluate collective work in public open space design, driven by digital gamified co-design methods. From Hong Kong’s open space development history, it can be learnt how high-density cities are prospective grounds to lead international dialogues on the potential to utilise infrastructural capital in enhancing diversity, equality, and inclusivity of spatial design. Public space is one of such urban infrastructures that support social and civic exchanges in strengthening a city’s resilience and community-building. The idea that public space should provide for basic common needs was the aspiration of the past; today, we have the knowledge to aim much higher on spatial resources design using shared digital tools. Public space theories provided insights to how limited space can still facilitate high-quality spatial designs through criteria of activity complexity, sociability, comfort, safety, adaptability, and accessibility. Spatial design guidelines expanding on such criteria are inevitable innovations to implement such goals. The proposed user-activity-environment based toolkit is a first step to communicate a larger vision on how a graphical means in designing spatial guidelines can better support collective decision-making. The preliminary test demonstrated how quantitative and qualitative spatial configuration analysis may be bridged using a combinatorial assessment method. In this way, the universality of the toolkit can compensate for the gaps between planning and spatial experience design. In guiding collective work in design, mutual-constrain indicators help to signal and limit design opportunities while embedding varied community needs. In the game-based process, the scoring system helped participants to develop an awareness of the need to respect differences and compromise for consensus within a community. Our future work will further evaluate the feasibility of the proposed toolkit by inviting citizens and other experts to join the collaborative design process. Acknowledgements. This project was supported by The Chinese University of Hong Kong’s Seed Funding Support for Thesis Research and ICARE Social Service Project Scheme. We thank

Structuring Gamified Participatory Public Space Design Developing

281

Yuechun LI and Bingge XU for their assistance in the design and workshop. We are grateful for the help of all experiment participants.

References van Ameijde, J., Song, Y.: Data-driven urban porosity: incorporating parameters of public space into a generative urban design Process. In: Learning, Adapting and Prototyping–Proceedings of the 23rd International Conference on Computer-Aided Architectural Design Research in Asia, vol. 1, pp. 173–82 (2018). https://papers.cumincad.org/data/works/att/caadria2018_332. pdf Ampatzidou, C., et al.: All work and no play? facilitating serious games and gamified applications in participatory urban planning and governance. Urban Planning 3(1), 34–46 (2018). https:// doi.org/10.17645/up.v3i1.1261 CABE: Spaceshaper: A user’s guide. London, Commission for Architecture and the Built Environment (2007) Carmona, M.: Public places urban spaces: The dimensions of urban design. Routledge (2021) Chan, C.M.A., Liang, J.S.E.: Active Aging: Policy Framework and Applications to Promote Older Adult Participation in Hong Kong. Ageing Int. 38, 28–42 (2013). https://doi.org/10.1007/s12 126-012-9166-z Chan, E.: Public space as commodity: social production of the Hong Kong waterfront. Proceedings of the Institution of Civil Engineers - Urban Design and Planning 173(4), 146–155 (2020) Cheng, S.-T., Lum, T., Lam, L.C.W., Fung, H.H.: Hong Kong: embracing a fast ageing society with limited welfare. Gerontologist 53(4), 527–533 (2013). https://doi.org/10.1093/geront/gnt017 Civic Exchange: Open Space Handbook – A guide for journalists in Hong Kong. Civic Exchange (2018) Dempsey, N.: Quality of the built environment in urban neighbourhoods. Plan. Pract. Res. 23(2), 249–264 (2008). https://doi.org/10.1080/02697450802327198 Trust, D.: Yi Pei Square Playground opens to the public. Design Trust Futures Studio (2021) Frank, K., Stevens, Q.: Loose Space: Possibility and Diversity in Urban Life. Routledge, New York (2007) Gehl, J.: Livet Mellem Husene: Life between Buildings: Using Public Space (1971) Gehl, J.: Cities for People. Island Press (2010) Govada, S.S., Wong, F., Rodgers, T., Chow, H.Y.C., Busa, K.: E-Engagement in Hong Kong. In: E-Democracy for Smart Cities, pp. 143–180. Springer Singapore (2017). https://doi.org/10. 1007/978-981-10-4035-1_5 HKGBC. (n.d.). Public Rental Housing Development at Anderson Road Sites A and B (on Tai estate). Hong Kong Green Building Council: BEAM Plus Online Exhibition. https://greenbuil ding.hkgbc.org.hk/projects/view/26 HKHA: Planning, Design, and Delivery of Quality Public Housing in the New Millennium. Publication Archive. Hong Kong Housing Authority (2023). https://www.housingauthority. gov.hk/en/common/pdf/about-us/publications-and-statistics/publication-archive/HA_DCD_ Booklet_(%202011)_1.pdf HKHS. (n.d.). Jat Min Chuen. Hong Kong Housing Society (2023). https://www.hkhs.com/en/ housing_archive/id/16 HKPH. (n.d.). On Tai Estate. Federation of Public Housing Estates (2023). http://hkph.org/estates/ ontai/ Houlstan-Hasaerts, R., Tominc, M., Nikšiˇc, M., Goliˇcnik Maruši´c, B.: Civil Society Reclaims Public Space: Cross Perspectives Based On Research. Digital Publication, URBANI IZZIV PUBLIKACIJE (2012). https://www.uirs.si/pub/humancities2012.pdf

282

S. Zhu et al.

Jacobs, J.: The death and life of great American cities. Vintage Books (1961) Lai, C.: Unopened Space: Mapping Equitable Availability of Open Space in HK. Civic Exchange (2017). https://civic-exchange.org/wp-content/uploads/2017/04/20170224P OSreport_FINAL.pdf LCSD: LCSD enhances public engagement on Tsim Sha Tsui waterfront design (2015) LCSD: Victoria Park: Outdoor Facilities (2022). https://www.lcsd.gov.hk/en/parks/vp/facilities/ outdoor.html Mehta, V.: Evaluating public space. J. Urban Des. 19(1), 53–88 (2014). https://doi.org/10.1080/ 13574809.2013.854698 Mehta, V., Bosson, J.K.: Third places and the social life of streets. Environ. Behav. 42(6), 779–805 (2010). https://doi.org/10.1177/0013916509344677 Mitchell, D.: The End of Public Space? People’s Park, Definitions of the Public, and Democracy. Ann. Assoc. Am. Geogr. 85(1), 108–133 (1995). https://doi.org/10.1111/j.1467-8306.1995.tb0 1797.xa Ng, M.K.: Researching Community Engagement in Post-1997 Hong Kong. In: The Routledge Handbook of Planning Research Methods, pp. 157–168. Routledge (2014) Ng, P., Li, Y., Zhu, S., Xu, B., van Ameijde, J.: Digital common (s): the role of digital gamification in participatory design for the planning of high-density housing estates. Frontiers in Virtual Reality 3, 184 (2023). https://doi.org/10.3389/frvir.2022.1062336 Oldenburg, R.: The great good place: cafes, coffee shops, community centers, beauty parlors, general stores, bars, hangouts and how they get you through the day (1997) Planning Department: Hong Kong Planning Standards and Guidelines, chapter 4 (2022) PPS. (n.d.) What makes a successful place? The Project for Public Spaces. https://www.pps.org/ article/grplacefeat Sheikh, W.T., van Ameijde, J.: Promoting livability through urban planning: a comprehensive framework based on the “theory of human needs.” Cities 131, 103972 (2022). https://doi.org/ 10.1016/j.cities.2022.103972 Stafford, L., Baldwin, C.: Planning neighbourhoods for all ages and abilities: a multi-generational perspective. State of Australian Cities Research Network, Australia, 1–16 (2015). https://epr ints.qut.edu.au/93600/ UN-Habitat: SDG Indicator 11.7.1 Training Module: Public Space. United Nations Human Settlement Programme (UN-Habitat), Nairobi (2018) Whyte, W.H.: The social life of small urban spaces. Conservation Foundation (1980) Yung, E.H.K., Conejos, S., Chan, E.H.W.: Public open spaces planning for the elderly: The case of dense urban renewal districts in Hong Kong. Land Use Policy 59, 1–11 (2016). https://doi. org/10.1016/j.landusepol.2016.08.022

Author Index

A Agkathidis, Asterios 9 Albano, Silvia 95

J Jin, Mengfan

K Korae, Eva 25

B Bao, Yuxin 171 Belkouri, Daria 103 Bolojan, Daniel 244 C Carrillo Andrada, José Antonio Cekmis, Asli 201 Chen, Zhonghao 225 Cheung, Lok Hang 9 Choobkar, Faezeh 201 Cimillo, Marco 34

34

179

D Dall’Asta, Juan Carlos 9, 190 de la Rosa Morón, José 179 Di Marco, Giancarlo 1, 9, 190 Dickinson, Susannah 253 Dounas, Theodoros 103 F Feng, Lei 18 Feng, Yiheng 77 G Georgiou, Michail 25 Georgiou, Odysseas 25 Gogani, Alireza 201 Guo, Yongcong 262 H Herr, Christiane M. 42, 136, 217, 262 Holmes, Matthew 111 Hua, Hao 236 Huang, Lingshan 160

L Latto, John 118 Li, Chenxiao 217 Li, Li 77 Li, Na 95 Li, Yunqin 160 Liu, Yiming 42 Lombardi, Davide 1, 136 M McKay, Graham Brenton 209 Melnyk, Virginia Ellyn 50 Meng, Wan 95 Muftee, Muhammad Talha 61 N Nam, Hyunjae 69 Nan, Cristina 128 Ng, Provides 270 O Oliver Ramírez, José Luis 179 P Pei, Lulu 217 Q Quan, Deyan

136

R Reyes, Alejandro Veliz

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2024 G. Di Marco et al. (Eds.): xArch 2023, LNCE 343, pp. 283–284, 2024. https://doi.org/10.1007/978-981-97-0621-1

111

284

Author Index

S Santini, Giovanni 225 Shi, Jiaxi 236 Song, Yang 144

T Taherysayah, Fatemeh Tedjosaputro, Mia 1

X Xiang, Changying 171 Xiao, Jun 217 Xie, Yuchen 160 Xu, Wenruo 95

152

V van Ameijde, Jeroen 270 Vigorito, Alessio 128

W Westermann, Claudia 152

Y Yousif, Shermeen

244

Z Zeshan, Asif Hasan 253 Zhang, Jiaxin 160 Zhang, Yu 217 Zhang, Yuqi 262 Zhao, Wei 144 Zheng, Danni 77 Zhu, Shutong 270 Zhu, Zhelun 85