Design Computing and Cognition’20 3030906248, 9783030906245

Citation preview

John S. Gero   Editor

Design Computing and Cognition’20

Design Computing and Cognition’20

John S. Gero Editor

Design Computing and Cognition’20

123

Editor John S. Gero Department of Computer Science and School of Architecture University of North Carolina Charlotte, NC, USA

ISBN 978-3-030-90624-5 ISBN 978-3-030-90625-2 https://doi.org/10.1007/978-3-030-90625-2

(eBook)

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Preface

Design research is often broken into three categories: researching the designer, researching designing (the design process), and researching the design that is produced. The assumption is that each of these categories can be researched, at least to some extent, independently. Designers are embedded in social, technical, and economic cultures, as are the designs they produce. Cutting across these categories are methodological categories: measuring and modeling. Measuring covers the designer, designing, and the design, although in different ways. Measuring requires instruments, and instruments favor particular outcomes. For example, measuring heart rate variability might tell you about the level of stress felt by a designer at a specific moment, but not about their cognitive behavior. While measuring the words they use might tell you about the scope of ideas they are developing but not about their interactions with their design media. Modeling of a design as it is being developed by designers is currently done computationally. These computational models are agnostic about the design process used to develop the design. There is a large body of knowledge produced by design research, but that body is not coherent, comprehensive, or commensurable. There appear to be various reasons why design research has not produced a body of knowledge that is coherent, comprehensive, and commensurable. Prime among them are the use of a single method when studying designers, designing, and designs, and the lack of coherence between experiment designs and their results. To increase our understanding of designers and designing, we will need multi-method measurement approaches. However, there is little research that makes use of multiple methods whose output provides the basis for correlating the results obtained from the different methods, as most research makes use of a single method only. The previous decades of research using a single method has been successful in providing an evidence-based window into designers and designing as an activity. Results from such research have been used to provide evidence for design theories, foundations for changing design pedagogy, and as the basis for the development of design tools. However, research that utilizes only a single method treats designing as a disaggregated activity, where cognition and its embodiment are not connected and where the brain and mind are not considered as a unified whole. In order to advance the v

vi

Preface

field, it is suggested that design research moves increasingly into multi-method studies that use multiple methods. In other research domains, there are projects that have moved beyond the single-method studies with the development of mobile brain/body imaging (MoBI) systems that research the interactions between mind, body, and brain. The papers in this volume are from the Ninth International Conference on Design Computing and Cognition (DCC’20) held virtually at the Georgia Institute of Technology, Atlanta, USA. They represent the state of the art of research and development in design computing and design cognition including the increasingly active area of design cognitive neuroscience. They are of particular interest to design researchers, developers, and users of advanced computation in designing as well as to design educators. This volume contains knowledge about the cognitive behavior of designers, which is valuable for those who need to gain a better understanding of designing. In these proceedings, the papers are grouped under the following seven headings, describing both advances in theory and application and demonstrating the depth and breadth of design computing, design cognition, and the new research area of design neurocognition: Design Cognition – 1 Design Grammars and Networks Design Generation Design Creativity Design Cognition – 2 Design Neurocognition and Physiology Machine and Human Learning in Design A total of 101 full papers were submitted to the conference, from which 41 were accepted and appear in these proceedings. Each paper was extensively reviewed by at least three reviewers drawn from the international panel of reviewers listed on the following pages. The reviewers’ recommendations were then assessed before the final decision on each paper was taken. The authors improved their contributions based on the advice of this community of reviewers prior to submitting the final manuscript for publication. Thanks go to the reviewers, for the quality of these papers depends on their efforts. John Gero

Organization

List of Reviewers Henri Achten, Czech Technical University, Czech Republic Katerina Alexiou, Open University, UK Janet Allen, University of Oklahoma, USA Jose Beirao, University of Lisbon, Portugal Lucienne Blessing, SUTD, Singapore Jean-Francois Boujut, Grenoble INP, France Frances Brazier, TU Delft, The Netherlands Ross Brisco, University of Strathclyde, UK David Brown, WPI, USA Janet Burge, Wesleyan University, USA Jonathan Cagan, Carnegie Mellon University, USA Hernan Casakin, Ariel University Center of Samaria, Israel Gaetano Cascini, Politecnico di Milano, Italy Philip Cash, Technical University of Denmark, Denmark Alexandros Charidis, MIT, USA John Clarkson, University of Cambridge, UK Nathan Crilly, University of Cambridge, UK Nasrin Debhozorgi, University of North Carolina at Charlotte, USA Andy Dong, Oregon State University, USA Alex Duffy, Strathclyde University, UK Chris Earl, Open University, UK Claudia Eckert, Open University, UK Athanassios Economou, Georgia Institute of Technology, USA Benoit Eynard, Université de Technologie de Compiègne, France Katherine Fu, Georgia Institute of Technology, USA John Gero, University of North Carolina at Charlotte, USA Gabriela Goldschmidt, Technion, Israel Kosa Goucher-Lambert, CMU, USA

vii

viii

Ewa Grabska, Jagiellonian University, Poland Kazjon Grace, University of Sydney, Australia Joshua Gyory, CMU, USA Tracey Hammond, Texas A&M, USA Sean Hanna, University College London, UK Laura Hay, University of Strathclyde, UK Ben Hicks, University of Bristol, UK Yan Jin, University of Southern California, USA Jana Kainerstorfer, CMU, USA Yehuda Kalay, University of California, Berkeley, USA Jeff Kan, Hong Kong Udo Kannengiesser, Johannes Kepler University-Linz, Austria Pegah Karimi, University of North Carolina at Charlotte, USA Mi Jeong Kim, Kyung Hee University, Korea Terry Knight, Massachusetts Institute of Technology, USA Lauri Koskela, University of Huddersfield, UK Ramesh Krishnamurti, CMU, USA Ehud Kroll, ORT Braude College, Israel Djordje Krstic, Signalife, USA Julie Linsey Georgia Institute of Technology, USA Ade Mabogunje, Stanford University, USA Mary Lou Maher, University of North Carolina at Charlotte, USA Ray McCall, University of Colorado, USA Chris McComb, Pennsylvania State University, USA Chris McMahon, University of Bristol, UK Jessica Menold, Pennsylvania State University, USA Scarlett Miller, Pennsylvania State University, USA Julie Milovanovic, CRENAU, France Farrokh Mistree, University of Oklahoma, USA Jeff Nickerson, Stevens Institute of Technology, USA Mine Ozgar, Istanbul Technical University, Turkey Panos Papalambros, University of Michigan, USA Pieter Pauwels, Eindhoven University of Technology, The Netherlands Yoram Reich, Tel Aviv University, Israel Duska Rosenberg, University of London, UK Stephan Rudolph, University of Stuttgart, Germany Fil Salustri, Ryerson University, Canada Li Shu, Toronto University, Canada Tripp Shealy, Virginia Tech, USA Vishal Singh, Indian Institute of Science, India Steven Smith, Texas A&M University, USA Ricardo Sosa, Auckland University of Technology, New Zealand Martin Stacey, De Montfort University, UK George Stiny, Massachusetts Institute of Technology, USA Mario Storga, University of Zagreb, Croatia

Organization

Organization

Rudi Stouffs, National University of Singapore, Singapore Joshua Summers, Clemson University, USA Brian Sylcott, East Carolina University, USA Christine Toh, University of Nebraska-Omaha, USA Barbara Tversky, Columbia and Stanford, USA Pedro Veloso, CMU, USA Sonia Vieira, Polimi, Italy Ian Whitfield, University of Strathclyde, UK Kristin Wood, SUTD, Singapore Robert Woodbury, Simon Fraser University, Canada Bernard Yannou, Ecole Centrale Paris, France Rongrong Yu, Griffiths University, Australia Theodore Zemenopoulos, Open University, UK Yong Zeng, Concordia University, Canada

ix

Contents

Design Cognition – 1 Empirically Understanding the Impact of Item Constraints on Designer Ideation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Leah Chong, Kosa Goucher-Lambert, Kenneth Kotovsky, and Jonathan Cagan

3

Exploring the Use of Digital Tools to Support Design Studio Pedagogy Through Studying Collaboration and Cognition . . . . . . . . . . . . . . . . . . Julie Milovanovic and John S. Gero

21

Modelling the Dynamics of Influence on Individual Thinking During Idea Generation in Co-design Teams . . . . . . . . . . . . . . . . . . . . . . . . . . . Harshika Singh, Christopher McComb, and Gaetano Cascini

41

The Psychological Links Between Systems Thinking and Sequential Decision Making in Engineering Design . . . . . . . . . . . . . . . . . . . . . . . . . John Z. Clay, Molla Hazifur Rahman, Darya L. Zabelina, Charles Xie, and Zhenghui Sha

63

Patterns of Silence and Communication Between Paired Designers in Collaborative Computer-Aided Design . . . . . . . . . . . . . . . . . . . . . . . . Meaghan Vella, Alison Olechowski, and Vrushank Phadnis

81

This Is How I Design: Discussing Design Principles in Small Multidisciplinary Teams of Design Professionals . . . . . . . . . . . . . . . . . . Nicole B. Damen and Christine A. Toh

93

Design Grammars and Networks Design Without Rigid Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Sotirios D. Kotsopoulos

xi

xii

Contents

Deriving the Production Rules of Shape-Shifting Grammars for Adaptive Façades: The Case of Hygromorphic Thermo-Bimetal Composites (HMTM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 Rana El-Dabaa and Sherif Abdelmohsen Spatial and Nonspatial in Calculations with Shapes . . . . . . . . . . . . . . . . 151 Djordje Krstic Tracing Hybridity in Local Adaptations of Modern Architecture: The Case of A. William Hajjar’s Single-Family Architecture . . . . . . . . . 171 Mahyar Hadighi and Jose Duarte Five Criteria for Shape Grammar Interpreters . . . . . . . . . . . . . . . . . . . 191 Tzu-Chieh Kurt Hong and Athanassios Economou Contributions and Challenges of Pearl’s Causal Networks to Causal Analysis in Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209 Raymond McCall and Janet E. Burge Design Generation SAPPhIRE: A Multistep Representation for Abductive Reasoning in Design Synthesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 Apoorv Naresh Bhatt and Amaresh Chakrabarti A Mathematical Approach for Generating a Floor Plan for Given Adjacency Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247 Krishnendra Shekhawat Interior Layout Generation Based on Scene Graph and Graph Generation Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267 Xia Su, Chenglin Wu, Wen Gao, and Weixin Huang Less Is More? In Patents, Design Transformations that Add Occur More Often Than Those that Subtract . . . . . . . . . . . . . . . . . . . . . . . . . . 283 Katelyn Stenger, Clara Na, and Leidy Klotz Voxel Synthesis for Architectural Design . . . . . . . . . . . . . . . . . . . . . . . . 297 Immanuel Koh Reverse Algorithmic Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317 António Leitão and Sara Garcia Design Creativity Metaphorical Concepts and Framework for Designing Novel Approaches to Interactive Buildings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331 Jingoog Kim and Mary Lou Maher

Contents

xiii

Design for Cybersecurity (DfC) Cards: A Creativity-Based Approach to Support Designers’ Consideration of Cybersecurity . . . . . . . . . . . . . . 351 Vivek Rao, Euiyoung Kim, Hyun Jie Jung, Kosa Goucher-Lambert, and Alice M. Agogino Towards a Virtual Librarian for Biologically Inspired Design . . . . . . . . 369 Ashok Goel, Kaylin Hagopian, Shimin Zhang, and Spencer Rugaber An Investigation into the Cognitive and Spatial Markers of Creativity and Efficiency in Architectural Design . . . . . . . . . . . . . . . . . . . . . . . . . . 387 Kinda Al Sayed and Alan Penn Conversational Co-creativity with Deep Reinforcement Learning Agent in Kitchen Layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399 Poyen Hsieh, Deborah Benros, and Timur Dogan Assessing the Novelty of Design Outcomes: Using a Perceptual Kernel in a Crowd-Sourced Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 411 Dongwook Hwang and Kristin Lee Wood Serendipitous Explorations in Distributed Work in Parametric Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423 Livanur Erbil Altintas, Altug Kasali, and Fehmi Dogan Design Cognition – 2 Does Empathy Beget Creativity? Investigating the Role of Trait Empathy in Idea Generation and Selection . . . . . . . . . . . . . . . . . . . . . . 437 Mohammad Alsager Alzayed, Scarlett R. Miller, and Christopher McComb How Do Designers Think in Systems? – Empirical Insights from Protocol Studies of Experienced Practitioners Designing ProductService Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 455 Abhijna Neramballi, Tomohiko Sakao, and John S. Gero Towards Modelling Interpretation of Structure as a Situated Activity: A Case Study of Japanese Rock Garden Designs . . . . . . . . . . . . . . . . . . 473 Yuval Kahlon and Haruyuki Fujii Interactive Visualization for Design Dialog . . . . . . . . . . . . . . . . . . . . . . 491 Arefin Mohiuddin and Robert Woodbury Composing Diverse Design Teams: A Simulation-Based Investigation on the Role of Personality Traits and Risk-Taking Attitudes on Team Empathy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 509 Mohammad Alsager Alzayed, Scarlett R. Miller, and Christopher McComb

xiv

Contents

Emergence of Engineering Design Self-Efficacy in Engineering Identity Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 521 John Jongho Park, Elizabeth Starkey, Nathan Hyungsok Choe, and Christopher McComb Design Neurocognition and Physiology Designing-Related Neural Processes: Higher Alpha, Theta and Beta Bands’ Key Roles in Distinguishing Designing from Problem-Solving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535 S. Vieira, J. S. Gero, V. Gattol, J. Delmoral, S. Li, G. Cascini, and A. Fernandes Verifying Design Through Generative Visualization of Neural Activity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555 Pan Wang, Danlin Peng, Simiao Yu, Chao Wu, Xiaoyi Wang, Peter Childs, Yike Guo, and Ling Li NeuroDesignScience: Systematic Literature Review of Current Research on Design Using Neuroscience Techniques . . . . . . . . . . . . . . . 575 Takumi Ohashi, Jan Auernhammer, Wei Liu, Wenjie Pan, and Larry Leifer Exploring Spatial Design Cognition Using VR and Eye-tracking . . . . . . 593 Nayeon Kim and Hyunsoo Lee Machine and Human Learning in Design A Machine Learning Approach for Mining the Multidimensional Impact of Urban Form on Community Scale Energy Consumption in Cities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 607 Mina Rahimian, Guido Cervone, Jose P. Duarte, and Lisa D. Iulo Data Mining a Design Repository to Generate Linear Functional Chains: A Step Toward Automating Functional Modeling . . . . . . . . . . . 625 Katherine Edmonds, Alex Mikes, Bryony DuPont, and Robert B. Stone Variational Deep Embedding Mines Concepts from Comprehensive Optimal Designs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643 Kazuki Minowa, Kikuo Fujita, Yutaka Nomaguchi, Shintaro Yamasaki, and Kentaro Yaji Learning Comes from Experience: The Effects on Human Learning and Performance of a Virtual Assistant for Design Space Exploration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 655 Antoni Viros i Martin and Daniel Selva

Contents

xv

Designing Self-assembly Systems with Deep Multiagent Reinforcement Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 667 Hao Ji and Yan Jin The Order Effect of Associative and Rule-Based Reasoning Strategies in Design Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 681 Batuhan Taneri and Fehmi Dogan

Design Cognition – 1

Empirically Understanding the Impact of Item Constraints on Designer Ideation Leah Chong, Kosa Goucher-Lambert, Kenneth Kotovsky, and Jonathan Cagan

Abstract Despite the potential of design constraints to benefit creative processes like ideation, the relationship between constraints and ideation outcome is still not sufficiently understood in order for constraints to be implemented as a design tool. This study aims to explore the impact of specifically item constraints on the effectiveness of ideation by investigating how the presence of item constraints affects three different measures of ideation outcome: the quantity, variety, and commonness of design solutions. A cognitive human experiment was run with both unconstrained (control) and constrained conditions. The results indicate that when constrained, participants generate ideas that are more common, while the quantity and variety of the generated ideas remain the same. Additionally, item constraints motivate ideas that are unique to the constrained condition. Overall, this research presents opportunities to exploit item constraints to direct the search of the design space to desirable ideation outcome.

1 Introduction 1.1

Ideation in Design

Ideation, or concept generation, is one of the early stages of the design process, typically following problem definition [1]. Effective ideation results in ‘good’ design ideas that are then explored further during detailed design [2]. In order to L. Chong
K. Kotovsky
J. Cagan (&) Carnegie Mellon University, Pittsburgh, USA e-mail: [email protected] L. Chong e-mail: [email protected] K. Kotovsky e-mail: [email protected] K. Goucher-Lambert University of California, Berkeley, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. S. Gero (ed.), Design Computing and Cognition'20, https://doi.org/10.1007/978-3-030-90625-2_1

3

4

L. Chong et al.

improve the effectiveness of ideation, many studies have been conducted to understand the cognitive processes during ideation and to develop different ideation techniques, such as design-by-analogy, brainstorming, and mind-mapping [3–8]. With the discovery of a significant correlation between the percentage of the design space explored during ideation and the quality of the final design [2], ideation is considered effective when the designer expands and explores the design space, which is an abstract space encompassing all possible solutions to a given problem [9, 10].

1.2

Duality of Constraints

Every real-world design task involves a number of constraints, meaning limitations or restrictions for what can or cannot be done and for what criteria the end result should fulfill [11]. Intuitively, as the term suggests, constraints can be expected to limit the expansion and exploration of the design space, therefore, discouraging effective ideation. However, many theoretical and empirical works have demonstrated the dual effect of constraints on creativity; both restraining and directing the search for creative solutions, they often lead to ideas that are new, surprising, and valuable [12–15]. While limiting the availability of some solutions, constraints can reduce the complexity of a design task [16]. Not only this, constraints can lead to the exploration of alternative solutions that are possibly more creative by suppressing people’s tendency to generate those ideas that are easiest to generate [13, 17]. The potential benefit of constraints on creative processes prompts a hypothesis that design constraints may help yield a desirable ideation outcome. With the understanding of the impact of constraints on the ideation outcome, constraints could potentially be utilized as a design tool, in which they are systematically added and/or manipulated, in order to reach a desired outcome. However, research thus far has been limited to the effect of constraints on creativity [12–15, 17, 18], where the findings are too broad, mixed, and inconclusive to effectively implement design constraints as a tool. Furthermore, many research studies have investigated the cognitive processes during creative activities but failed to discuss their implications specifically to ideation with constraints [3, 5, 17]. Therefore, this study zooms in to understand the effect of a specific type of design constraint (item constraints) on a specific creative process (ideation). A cognitive experiment was designed to investigate how the effectiveness of ideation is affected by the presence of item constraints or the limited availability of the items that can be used to construct solutions. This was compared against a control condition in which participants could generate solutions without a restriction on item availability. In this study, the effectiveness of ideation was measured by the quantity, variety, and commonness of generated design solutions.

Empirically Understanding the Impact of Item Constraints …

5

2 Experimental Methods 2.1

Experiment Overview

The experiment was conducted among two groups, with and without item constraints, and compared their ideation outcomes using three different measures: quantity, variety, and commonness. The independent variable was the presence or absence of item constraints in the design task. The dependent variables were quantity, variety, and commonness of design solutions generated within a fixed amount of time. The experiment controlled for the academic background (mechanical engineering) and level (senior undergraduate and graduate) of expertise of the participants, the duration of ideation, the instructions provided, the tools and facilities provided, and the compensation for participation.

2.2

Participants

63 senior undergraduate and graduate Mechanical Engineering students at Carnegie Mellon University were recruited for (and completed) the experiment in accordance with a protocol approved by the University’s Institutional Review Board. All participants were enrolled in one of two capstone design courses required for the mechanical engineering degree for the seniors and optional for the graduate students. Based on this, the levels of expertise and experience in Mechanical Engineering and in engineering design were controlled for.

2.3

Condition Groups

There were two different condition groups: the unconstrained group (control) and the constrained group. Participants were randomly assigned to the two groups. There were 30 participants in the unconstrained group and 33 in the constrained group. The two conditions were given the same design problem and environment but different item constraints (constraints or no constraints). The design problem chosen for this study was adapted from Tseng et al. (2008) where it was used to study the role of timing and analogical similarity in idea generation [19]. The problem was to design a time measurement system. This problem was chosen for its level of abstractness, its relevance to mechanical engineering concepts, and its flexibility for modifications. The specific wording and the constraints in the design task were modified to fit the needs of this study and are shown in Fig. 1. The constrained and unconstrained conditions were identical with the exception of the presence of the item constraints (the list of specific household items). For the constrained condition the participants were restricted to design with those items

6

L. Chong et al.

Fig. 1 Constrained group design task

only, while those in the unconstrained condition were not given any specific items but were told to use any items that might commonly be found around the household. The design task in the constrained condition is shown in Fig. 1. The design task in the unconstrained condition merely omitted the list of household items in the box and the references to them.

2.4

Materials

The experiment was conducted within classrooms during regular class periods. Each participant received a consent form, a question form (only in the second course section), a design task, a pencil, and blank pages with numbered boxes per page. An online timer was displayed on a screen at the front of the classroom. The use of the question form is explained further in the Procedure section.

2.5

Procedure

The experiment was conducted in two separate class periods, with 28 participants in the first course section and 35 in the second course section. In each class, the

Empirically Understanding the Impact of Item Constraints …

7

Fig. 2 Example page of the packet including participant solutions

participants were randomly assigned to the two condition groups. The participants in the first course section were divided into 14 in the unconstrained condition and 14 in the constrained condition. The participants in the second course section were divided into 16 in the unconstrained condition and 19 in the constrained condition. The difference in the number of participants in the two conditions was caused by the unexpected absence of students who had planned to participate. The participants received visibly similar envelopes containing all study materials. Each envelope was labeled either U (unconstrained) or C (constrained) to distinguish between the two condition groups, although the participants were not informed of this information. The participants were first asked to open the envelopes at the same time and only take out and sign the consent forms placed at the top of each package. They were instructed to put the consent forms back into the envelope when they were done signing. In the second course section, the participants were also asked to answer a question form that asked, “Have you heard about anything in regard to this study before? If you have, what have you heard?” to ensure that information about the experiment was not leaked between students during the time between the first and the second class experiments, which was about 6 weeks (there were no students who needed to be excluded for this reason). When all participants were done, they were asked to take out the rest of the sheets in the packet, which included the design task, a pencil, and numbered blank pages. Figure 2 shows a typical page of the packet populated with design content from the experiment. Then, participants were

8

L. Chong et al.

given two minutes to individually read the design task. With the start of the timer, the 30-min idea generation session began. Every five minutes, the participants were reminded to mark the number of the solution box that they were working on at that moment to allow for tracking the quantity of solutions over time.

3 Analysis 3.1

Data Cleansing

The experimental data containing 61 of the 63 participants were evaluated during the analysis: 29 in the unconstrained group and 32 in the constrained group. Across both conditions, a total of 551 design concepts were generated. A participant was excluded for analysis from the unconstrained group because they were an outlier, producing a number of design solutions that was 5 standard deviations away from the mean. All other participants were well-within 3 standard deviations. Additionally, a participant was excluded from analysis from the constrained group because they failed to follow the instructions of the experiment. For consistent analysis, 19 of the total 551 solutions were filtered out of the data. These 19 solutions included those that violated the constraints, those that had incomplete or incomprehensible descriptions, and all but one copy of those that were replicated by the same participant.

3.2

Functional Categories

To conduct analysis at a functional level, all solutions were classified into 37 functional categories, depicted in Table 1. The functional categories were adopted from Tseng et al. (2008) and modified according to the needs of this study. Unlike Tseng’s experiment, this study included an unconstrained condition, which required additional functional categories to be added to include solutions with other household items. Each solution was categorized into one or more functional categories identified in its description. Solutions that included multiple functional categories were placed in all relevant categories in fraction.

Empirically Understanding the Impact of Item Constraints …

9

Table 1 Functional categories 1

2

Periodic oscillation of a pendulum and its rate of decay Rate of change in the position of the sun

13

Rate of evaporation

25

Speed of a firefighter

14

Rate of water freezing or ice melting

26

Rate of repetition of an automated machine's job Flow rate through an hourglass Rate of charging or discharging a battery

27

Rate of diffusion or soaking or drying Rate of a liquid drying on a surface Rate of an automated machine performing a continuous job Rate of dominos being knocked over Calling to ask for time

30

Rate of a radioactive particle release Speed of a mouse trap Rate of an object sinking Rate of fermentation or ripening of food Rate of dissolving Speed of flight of a jet Rate of change in the position of the stars Rate of IC relay switching Rate of mix or separation Rate of tape losing its strength Rate of pressure change Rate of electrolysis

3

Rate of drip of a liquid

15

4

Flow rate of a liquid

16

5

Rate of burning

17

6

Rate of rotation

18

7

Rate of temperature change Rate of an object free falling or rolling down an incline Rate of repetition of a person’s action Heart rate

19

Rate of a person getting drunk or becoming sober Rate of a scale finding its equilibrium

23

Rate of magnifying sunlight

35

24

Rate of spring bounce

36

8

9 10 11

12

20

21 22

28 29

31 32

33 34

37

3.3

Ideation Effectiveness Metrics

A key factor in analysis of empirical studies about ideation is the definition and measurement of ideation effectiveness. Ideation is often defined to be effective when the designer expands and explores the design space [2]. Consistent with this definition, this paper assessed the ideation effectiveness with three measures: quantity, variety, and commonness [2, 20].

10

3.3.1

L. Chong et al.

Quantity

Quantity is the total number of ideas generated by a group or an individual during a designated amount of time [2]. This measure is crucial because generating more ideas increases the chance of better ideas [21, 22]. The total quantity was computed for each participant by counting the total number of ideas generated during the 30-min ideation period. The number of ideas generated per 5-min interval was also recorded and analyzed. The individual time steps were compared to identify when during the ideation process the ideation rate was affected by the item constraints.

3.3.2

Variety

Variety is a measure of the explored design space during ideation [2]. Variety counterbalances the quantity measure which can produce merely a large number of ideas that differ from each other in minor ways, and thus, does not always indicate effective ideation alone [2]. Variety is distinguished from commonness as it measures the breadth of the explored design space, while commonness assesses the frequency of ideas. Variety was evaluated on a functional level by mapping the conceptual distances of the 37 functional categories on a 2-D design space. In order to find a reliable quantitative measure of the explored design space, the generalized non-metric multidimensional scaling (GNMDS) method was used [23]. This method needed human response data to run and map the conceptual distances between the functional categories onto a design space. Therefore, a human subject study was conducted to collect responses to triplet comparisons of the functional categories. This method was chosen for its advantage of capturing the similarity distances to a good accuracy only using non-metric data, reducing the inconsistency of responses. Furthermore, it does not require comparison data for every possible pair. GNMDS uses an algorithm based on convex optimization techniques to assign Euclidean coordinates that map the conceptual relatedness of a set of objects, given a subset of non-metric triplet relations [23]. The non-metric triplet comparison data were collected from 50 human subjects who were each asked 120 questions in the form of “Is A conceptually closer to B or C?”, where A, B and C were different functional categories. The triplets were selected randomly from the 37 functional categories, and none of these triplets were repeated. About a quarter, 23,310 of them in this case, of all possible triplets were queried as Agarwal et al. (2007) showed that GNMDS performed to an acceptable precision with 15,000 triplet comparisons of 78,000 possible triplets, resulting in fewer than a quarter of the triplets being needed for evaluation [23]. The comparison data were input into the convex optimization problem shown in Eq. (1) [23]:

Empirically Understanding the Impact of Item Constraints …

11

ð1Þ

nijkl is a slack variable for each inequality constraint to allow for inequality violations. The subscripts i, j, k, and l represent the functional categories in set S. k is a positive scalar that controls the trade-off between the violations and the complexity of the model. K is a symmetric matrix with elements k that hold information about the distances between the data points (functional categories). Finally, the subscripts a and b are the indices of all the elements of matrix K. Further explanation of this optimization problem can be found in Agarwal et al. (2007). Using the result of this optimization problem, the matrix X with all the x and y coordinates of the data points were deduced. This was possible because of the following relationship in Eq. (2) [23]: kxi  xj k22 ¼ k ii  2kij þ k jj :

ð2Þ

Each comparison datum was implemented as a constraint with a slack variable, and the constraints on the elements of matrix K centered the embedding at the origin. Therefore, finding the matrix K that minimized the objective function with all the slack variables resulted in the matrix X that best satisfied the constraints. Once the 2-D Euclidean coordinates were determined and mapped for the 37 functional categories, the mean of the conceptual distances (Eq. (3)) between ideas was calculated for each participant and compared between the unconstrained and constrained groups. The relative distance between each functional category explored in this work was defined as the Euclidean distance as shown in Eq. (3) [23]:

2 Conceptual distance ¼
xi  xj
2 ; i; j ¼ 1; . . .; 37:

3.3.3

ð3Þ

Commonness

Commonness is a measure of how common an idea is amongst the entire set of generated ideas [20]. This measure assesses the production frequency of each idea during the ideation period. Commonness of ideas was assessed on a functional level using the aforementioned functional categories. First, for each condition, the frequencies of the functional categories were fit to a normal distribution as shown in Fig. 3. Then, a corresponding z-score (number of standard deviations away from the mean) was

12

L. Chong et al.

Fig. 3 Normal distribution of frequency of functional categories

assigned to each functional category. This z-score was chosen to represent the commonness score of the functional categories in this study. The average commonness score was calculated for each participant according to Eq. (4): P zi ci i Average commonness score ¼ P ; i ¼ 1; . . .; 37 ð4Þ ci i

where zi = z-score of functional category i, ci = number of ideas in functional category i. Then, as in the quantity analysis, the average commonness scores in the unconstrained and constrained groups were compared for the entire course of the ideation process and for each 5-min time step.

4 Results The three measures, total quantity, variety, and commonness, of the solutions generated in the constrained and unconstrained conditions were compared to examine the impact of the item constraints on ideation effectiveness. For quantity and commonness measures, the results from every 5-min time step were also evaluated in order to gain insight into the changing effect of the item constraints over the course of ideation.

Empirically Understanding the Impact of Item Constraints …

4.1

13

Quantity

The unconstrained and constrained conditions did not exhibit a significant difference in the total number of ideas generated during ideation (two-sample t-test, p = 0.48) as shown in Fig. 4. Likewise, there was no significant difference in the number of ideas per time step (Wilcoxon rank sum test, p = 0.54, 0.36, 0.18, 0.23, 0.99, and 0.53 respectively) as shown in Fig. 5. These results demonstrate that item constraints do not affect the number of ideas generated over the entire course of the ideation session, nor within smaller time partitions.

Fig. 4 Number of ideas generated. The error bars indicate the standard error

Fig. 5 Number of ideas generated per time step. The error bars indicate the standard error

14

L. Chong et al.

Fig. 6 Conceptual similarity 2-D map of the functional categories as a result of GNMDS

4.2

Variety

Figure 6 shows the GNMDS mapping of the functional categories on a 2-D design space (described in Sect. 3.3.2). The design space is dimensionless, meaning that each dimension on the map does not hold any meaning. Therefore, only the relative distances between the functional categories, not the absolute distances, have significance on the map. Additionally, the numbering in Fig. 6 corresponds to the functional categories in Table 1. The unconstrained and constrained groups showed no significant difference in the mean of the conceptual distances between the ideas (Wilcoxon rank sum test, p = 0.96) as shown in Fig. 7. This result illustrates that item constraints do not affect the variety or breadth of exploration of the design space.

4.3

Commonness

Overall, the constrained group came up with ideas that were more common than the unconstrained group (two-sample t-test, p = 2.0E−6), as shown in Fig. 8. Considering that the quantity and variety of the ideation outcome remained unaffected in the constrained search, this result suggests that item constraints merely affect the commonness of ideas generated. Looking into the commonness score per time step, the increase in the commonness of ideas was only statistically significant in Steps 3, 4 and 5 (two-sample t-test and Wilcoxon rank sum test, p = 0.68, 0.21, 9.7E−4, 0.031, 8.5E−4, and

Empirically Understanding the Impact of Item Constraints …

15

Fig. 7 Mean of conceptual distances between ideas. The error bars indicate the standard error

Fig. 8 Average commonness score. The error bars indicate the standard error

0.26) as demonstrated in Fig. 9. Additionally, the unconstrained group demonstrated a general decline of the commonness score over time, while the constrained group did not. These results together show that item constraints increase the commonness of ideas generated, particularly in the intermediate and late steps of ideation.

16

L. Chong et al.

Fig. 9 Average commonness score per time step. The error bars indicate the standard error

5 Discussion This work aims to explore the practicality of using design constraints to prompt creative processes by demonstrating the impact of item constraints on the ideation outcome. The quantity results in this current study indicate that participants provided with the item constraints generated the same number of ideas during the experiment as those without the constraints (see Fig. 4). In short, item constraints do not restrain the overall ideation quantity. Interestingly, the constraints also did not affect the ideation quantity per 5-min interval, following the same trend of decline in the quantity of ideas after the peak at the first interval (see Fig. 5). It is notable that although item constraints significantly reduce the size of the possible solution space, they do not hurt participants’ ideation rate. Instead, item constraints help to achieve a more focused search and potentially alleviate the cognitive burden caused by a large design space. This result is consistent with prior research regarding the usefulness of constraints in bringing about more creative solutions; while constraints may limit the availability of some solutions, they can direct designers to alternative, sometimes more creative, solutions [13]. Furthermore, the results demonstrate that item constraints promote neither more nor less variety in the ideas generated. The mean of the conceptual distances between generated ideas is unaffected by item constraints (see Fig. 7). Combined with the quantity result, this outcome indicates that item constraints do not change the breadth of exploration, meaning that a constrained design space is not necessarily unfavorable for yielding a diverse set of ideas. Similarly, the availability of a large possible design space does not imply that it will be utilized or is advantageous for a wider exploration of the design space. This result further supports the duality and the possibility of constraints leading to a better ideation outcome, achieving a more directed search by limiting the size of the design space but yielding equally diverse solutions [13–15].

Empirically Understanding the Impact of Item Constraints …

17

Unlike the low impact that item constraints have on the quantity and variety of design concepts, there is a significant difference between the commonness of the design solutions generated with and without item constraints. Participants in the constrained condition generated ideas that were more common, especially in the intermediate and late steps of ideation (see Figs. 8 and 9). Although it seems obvious that a smaller possible design space leads certain ideas to be generated more frequently, it is important to note that the rate and breadth of search during ideation remained constant, while the commonness of ideas increased with the item constraints. This means that the ideas generated with the provided set of items were not more frequently produced in one condition or the other, but were universally more common ideas. In addition, there was a significant overlap in the common concepts (i.e. functional categories with a commonness score greater than 0.4) in the two conditions. The common functional categories in the unconstrained condition were Categories 1, 2, 4, 5, 7, 8, 13, and 15, and those in the constrained condition were Categories 1, 2, 4, 5, 7, 8, and 9. This overlap further confirms that the ideas generated in the constrained search were not more common simply because they were produced frequently in that condition but because they were universally more common. For example, the overlapping Category 1, “periodic oscillation of a pendulum and its rate of decay”, was a commonly generated concept for this specific design problem regardless of the presence of the constraints. Even though no items were provided to the unconstrained participants, they still envisioned many of the same items as the ones provided in the constrained condition. Therefore, it is an unexpected and insightful finding that item constraints increase the commonness of ideas, while producing the same quantity and variety of ideas. Overall, this suggests that the commonness of ideas can be altered without changing the rate and breadth of search. In addition, the evident differences in commonness in the intermediate and late steps of ideation show that adding item constraints increases the commonness of ideas only in those steps and has a trivial impact on the earlier steps of ideation. Although it is confirmed that item constraints impact the commonness of ideas generated during ideation, while keeping their quantity and variety constant, there seems to be no advantage of using them if they merely direct the search to more universally common ideas. However, there was only a small overlap in the uncommon concepts (i.e., functional categories with a commonness score less than 0.4) in the two conditions. The uncommon functional categories in the unconstrained condition were Categories 21, 22, 23, and 24, and those in the constrained condition were Categories 16, 18, 23, 24, 25, 28, 34, 35, 36, and 37. This indicates that item constraints stimulated uncommon and new ideas that were not generated in the unconstrained condition. For example, Category 16, “flow rate through an hourglass”, was uniquely generated in the constrained condition. Stokes’s proposition is empirically confirmed by the results of this study. While constraints limit the availability of some solutions, they also lead to a well-directed search where alternative and potentially more creative solutions are explored [13], and even better, without negatively impacting the rate and breadth of search.

18

L. Chong et al.

Based on the results of this study, item constraints show great potential to be utilized as an ideation tool. More importantly, the results also provide some insight into how they may be utilized: although the overall commonness of the ideas might be higher with constraints, adding and manipulating item constraints can point the search to new ideas while keeping the ideation rate and variety of ideas in parity. This could be relevant to a company that is seeking new uses of their current products, for example as components or sub-systems within a larger product. Their search can be directed to creative alternate solutions by introducing appropriate item constraints to the products’ design settings (limiting the possible design space). It is important, however, to note some limitations of this study. First, the results of the experiment could be dependent on the specific list of item constraints. For example, a common set of household items might affect the quantity, variety, and commonness of ideas generated differently from the way an uncommon set does. Secondly, the generated ideas were classified to the functional categories by one person, which could have caused a bias error in this study. Repeated classification by multiple experts could have detected this bias and inconsistencies in the classification process. Lastly, the mapping of the functional categories assumed the design space to be two dimensional. However, this assumption of the space could have limited the accurate mapping of distances between the concepts. While this paper confirms and specifies the positive effect of item constraints on the ideation outcome, future work is necessary to successfully implement constraints as a support tool for effective ideation. Future work includes conducting similar variations of the current experiment with different sets of item constraints in order to examine whether different sets stimulate different new ideas by quantity, variety, and commonness in a consistent manner. Understanding the relationship between provided items and the ideas generated will allow a detailed and successful manipulation of constraints to yield a desirable outcome. Furthermore, this study can be extended to different types of constraints to discover the benefits of each type as a design tool.

6 Conclusion This work examines the impact of item constraints on ideation effectiveness in a cognitive human experiment, in order to determine the feasibility of and gain practical insights into using item constraints as an ideation support tool. Results confirm the possibility and reveal that when designers are provided with item constraints, the commonness of generated ideas increases and new ideas are stimulated, while the quantity and variety of design solutions remain unaffected. Further work is needed to reach a more comprehensive understanding of the relationship between constraints and the effectiveness of ideation and, eventually, to successfully implement different types of constraints to aid the ideation process.

Empirically Understanding the Impact of Item Constraints …

19

Acknowledgements This work was supported by the Air Force Office of Scientific Research (AFOSR) under grant FA9550-18-0088. Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the sponsors.

References 1. Dieter GE, Schmidt LC (2013) Engineering design. McGraw-Hill Education, New York 2. Shah JJ, Smith SM, Vargas-Hernandez N (2003) Metrics for measuring ideation effectiveness. Des Stud 24:111–134 3. Ward TB (2007) Creative cognition as a window on creativity. Methods 42(1):28–37 4. Schmid K (1996) Making AI systems more creative: the IPC-model. Knowl-Based Syst 9 (6):385–397 5. Martindale C (1995) Creativity and connectionism. In: The creative cognition approach. The MIT Press 6. Linsey J, Markman AB, Wood KL (2008) WordTrees: a method for design-by-analogy. In: Proceedings of the 2008 ASEE Annual Conference 7. Dugosh KL, Paulus PB, Roland EJ, Yang HC (2000) Cognitive stimulation in brainstorming. J Pers Soc Psychol 79(5):722 8. Goucher-Lambert K, Cagan J (2019) Crowdsourcing inspiration: using crowd generated inspirational stimuli to support designer ideation. Des Stud 61:1–29 9. Dylla N (1991) Thinking methods and procedures in mechanical design. Ph.D. dissertation, Technical University of Munich 10. Ullman DG (2010) The mechanical design process, 4th edn. 11. Gross MD (1986) Design as exploring constraints. Ph.D. dissertation, Massachusetts Institute of Technology 12. Boden MA (1994) Dimensions of creativity (Chapter 4). In: Dimensions of creativity 13. Stokes PD (2007) Using constraints to generate and sustain novelty. Psychol Aesthetics Creat Arts 1:107 14. Stokes PD (2008) Creativity from constraints: what con we learn from Motherwell? From Modran? From Klee? J Creat Behav 42(4):223–236 15. Wiltschnig S, Onarheim B (2010) Opening and constraining: constraints and their role in creative processes. In: Proceedings of the 1st DESIRE Network Conference on Creativity and Innovation in Design. Desire Network 16. Caniëls MCJ, Rietzschel EF (2015) Organizing creativity: creativity and innovation under constraints. Creat Innov Manag 24(2):184–196 17. Smith SM, Ward TB, Finke RA (1995) Creative processes in creative contexts. In: The creative cognition approach. The MIT Press 18. Marsh RL, Landau JD, Hicks JL (1996) How examples may (and may not) constrain creativity. Mem Cogn 24:669–680 19. Tseng I, Moss J, Cagan J, Kotovsky K (2008) The role of timing and analogical similarity in the stimulation of idea generation in design. Des Stud 29:203–221 20. Chan J, Fu K, Schunn C, Cagan J, Wood K, Kotovsky K (2011) On the benefits and pitfalls of analogies for innovative design: ideation performance based on analogical distance, commonness, and modality of examples. J Mech Des 133:081004 21. Basadur M, Thompson R (1986) Usefulness of the ideation principle of extended effort in real world professional and managerial creative problem solving. J Creat Behav 20:23–34 22. Parnes SJ (1961) Effects of extended effort in creative problem solving. J Educ Psychol 52:117 23. Agarwal S, Wills J, Cayton L, Lanckriet G, Kriegman DJ, Belongie S (2007) Generalized non-metric multidimensional scaling. In: International Conference on Artificial Intelligence and Statistics

Exploring the Use of Digital Tools to Support Design Studio Pedagogy Through Studying Collaboration and Cognition Julie Milovanovic and John S. Gero

Abstract This paper explores the effect of the use of digital design representation tools to support design studio pedagogy. We present the results of a case study of three types of architectural design critiques also called design reviews. The first one is a traditional desk critique where common design representations (plans, section, mock-ups) were used by tutors and students. The second case study investigates the use of a social Virtual Reality device, the Hyve-3D, that supports design collaboration through an immersive 3D sketch interface. The third case study involves the use of a digital desk utilizing the Sketsha interface to support remote design studio critiques. We used a video protocol analysis to study two characteristics of the design critiques: design collaboration and participants’ interactions with design representations. Results highlight behavioral trends for each type of critique and provide insights on the potential of digital design representations to support design studio pedagogy.

1 Introduction Designing and design representations are influenced and shaped by factors such as the evolution of digital technologies. It changes our design processes, the tools and ways to represent the design process and its result, the design artefact. The emergence of digital design tools and alternative, immersive and interactive design representations raises many questions about the integration of these tools into the pedagogical framework of design education. The design studio is an essential part of design education in many design domains as it aims at teaching students how to design by doing design. We consider design as a reflective practice [1] that relies on a set of implicit cognitive processes.

J. Milovanovic (&)
J. S. Gero University of North Carolina at Charlotte, Charlotte, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. S. Gero (ed.), Design Computing and Cognition'20, https://doi.org/10.1007/978-3-030-90625-2_2

21

22

J. Milovanovic and J. S. Gero

Students build design knowledge and skills as they learn by doing design, in a trial-and-error process, while being mentored by design studio tutors. During design critiques, also called design reviews, taking place in the studio, students present their designs to tutors and get feedback on their design in order to advance in their design process. Design representations used during the critiques are important as they support the interactions between the participants. Common design representations vary from diagrams, sketches and drawings, to plans, sections, and perspectives, and include physical mock-ups, digital models, sometimes animated or immersive. All these design representations support communication between students and tutors and serve as an environment to discuss students’ designs. Design representations have a triple purpose during design critiques. They provide a medium for students to express their design intentions and concepts to their tutors. They support collaboration as the participants in the design critique can negotiate, explain concepts, find solutions through the co-construction of an idea and reasoning with the help of design representations. Finally, they are used as a design tool: tutors and students will be able to propose a whole or partial solution to an unresolved design problem, by manipulating design representation. In this paper, we explore the use of two different digital tools to assist design studio critiques. The first one, the Hyve-3D, is a social Virtual Reality device, that provides an immersive 3D representation of a design and an interface for 3D sketching and navigation. The second one is the remote Collaborative Design Studio (CDS) that uses an augmented tabletop with the SketSha software to organize remote design critiques between two European universities. SketSha supports 2D drawing on documents shared between the two sites. We compared those two types of design critiques with a traditional desk critique in order examine the effect of the use of these digital tools during design critiques by: • exploring the behavior of tutors and students during the critiques: what are their roles in term of designing? • studying how the digital tool is exploited during the critiques: what are the actions of the tutors and students on the design representations? In the next section we build on references from the literature to develop the notions of collaboration and learning in design, and we discuss the importance of design representations, including digital ones, during the design critiques. Then, we present the methodology used to study our cases. The results will be described through two criteria: the role of each participant in the critiques’ reflective practice and the use of design representations. The last sections of the paper discuss the results in the light of previous results found in similar studies and proposes directions for future work.

Exploring the Use of Digital Tools …

23

2 Background 2.1

Learning Design by Doing Design

Design critiques punctuate the temporality of the studio and the progress of the student’s design. The format of design critique varies from one-on-one desk critiques involving a tutor and a student, to group reviews, peer discussions, pin-ups and juries [2]. One-on-one desk critiques provide, on a regular basis, a moment where students can present their design and get feedback from an expert, seek advices when faced with a specific design problem or are stuck in their design process [3]. The objectives of the critique are to evaluate the student’s work, while providing constructive feedback on the design development. Design problems can be addressed during design critiques or simply pointed out to students so that they can reflect on them after the critique and adapt their design accordingly. Exploring, suggesting and proposing solutions can be considered as designing, where verbal and graphic formalization are intertwined. In The Design Studio, Schön identifies four types of actions in design critiques: telling (tutor) and listening (student); demonstrating (tutor) and imitating (student) [4]. The first set corresponds to the explicit formulation of design knowledge, such as specific instructions to be followed, design theories, requirements concerning the format of representations or design references; and the second refers to a design situation through the tutor’s demonstration [4–6].

2.2

Importance of Design Representations During Design Critiques to Support Collaboration

Communication modalities and the relationship between tutors and students anchor design critiques in a social situation. The feeling of trust between tutors and students will allow them to feel comfortable to explain their design. Communication and collaboration appear as two important factors in order for the critique to be beneficial in terms of learning. In design critique situations, the concept of mutual responsibility for collaborative conversation applies between tutors and students. Everyone agrees that their interlocutor has a sufficient understanding of what they have just formulated before continuing talking [7]. The tutor/student team must understand what the other is referring to in order to co-construct the critique and the reflection on the design. Communication is essential for students and tutors to cognitively synchronize their own mental model of students’ designs. The objective is to build a common design reference or common ground [8]. This first step of cognitive synchronization is important in collaborative design situations in order to integrate the point of view and reflection processes of each team member to make a collective decision [9].

24

J. Milovanovic and J. S. Gero

The collaborative interaction between students and tutors is verbal, graphical and gestural and is channeled through design representations used during the critique. All the external design representations such as sketches, diagrams, plans, sections, physical models, digital models, simulations and animations form a representational ecosystem [10, 11] that acts as a support for communication, for an evaluation of students’ designs and for an exploration of design proposals for inherent design problems. During design critiques, these activities are similar to a co-design activity between tutor and student. Indeed, the studio’s pedagogical approach, project-based and by experience, implies that the design activity, which is the learning objective, is also the central activity during design critiques. The externalization of design representations in a collaborative design framework serves to: leave a trace of the designer’s mental effort in an external representation, represent elements that can give feedbacks (reflective conversation with the representations), and create an environment for criticism and negotiation [12]. In the situation of design critiques in architectural design studios, the pedagogical challenge of building design knowledge adds to the function of the representational ecosystem to support design and communication.

2.3

Design Representations to Support Design Processes

The production of drawings during designing, generating shapes and the relationship between these shapes, allows the designer to enrich their exploration space. Sketches are related to reasoning and reflecting during the design activity, where external and internal representations interact in a form of reflective conversation [1] or dialectic of sketching [13]. Designers externalize the concept of their design and explore new concepts by redrawing based on their design knowledge. If an idea appears in the representational ecosystem, it can be developed, revised and tested [14]. New design actions, anticipated or unexpected, may follow, which can be associated with the effect of surprise and creativity in the design activity. Sketching is often considered essential in the design activity, although some studies have shown little difference between designing with or without sketching [15]. Goldschmidt in [13] identified two modes of reasoning related to the way designer see their designs: “seeing as” (seeing as something else) and “seeing that” (seeing the element itself). A form of rationalization or generalization of decisions made in “seeing as” appears in the “seeing that” reasoning. For architects, sketching facilitates the interaction between design representations and the cognitive process of interpreting the concept. Ideas are transposed into sketches and can then be analyzed. In their study, Suwa & Tversky [16] use the concept of “focus shift” and “continuing segment” to study architecture students and professional architects only using sketching as a design tool. The “focus shift” pattern refers to Goel’s lateral transformation [17] and is associated with the proposal of a new space, an emerging element in the design. In this study, it appears that sketching is not only

Exploring the Use of Digital Tools …

25

used to establish spatial relationships between the elements but also to support abstract reasoning. Sketching isn’t the only action on design representations that accompanies the design activity. Gestures are known to connect to thinking and reflecting [18, 19] and designing [20]. For instance, gestures in co-design can serve the purpose of communicating 3D and dynamic elements [21] or support interpretation and information actions [22].

2.4

Using a Digital Representation Ecosystem to Support Design Studio Pedagogy

Student/tutor interaction during design critiques are situated within the design representational ecosystem. The immersive characteristic of design representations potentially has an effect on designers due to the exploitation of external design representation as a thinking tool. The manipulation of virtual environments during the design process helps designers to better perceive space, for example its fluidity and functionality, without using 2D representations [23]. VR is widely used, from design itself to construction and project communication to collaborative decision-making [24, 25]. The use of VR in the studio can promote spatial understanding of the architectural design and improve students’ self-assessment of their work [26], support the construction of design knowledge [27] and favor students’ engagement in a co-design processes during critiques [28]. Other uses of VR in an educational context aim to enhance students’ understanding of the structural parameters of their project [29], to enrich the modalities of representation [30] or to encourage remote collaboration between students [23], to name a few. The use of augmented tabletops is an alternative use to VR that provides a way to reduce the cognitive load of students during the design process by bringing together different type of representation related to specific the design steps and represent rich environmental information such as wind flow, shadows, or traffic [31, 32]. We have emphasized the importance of collaboration between tutors and students during design studio critiques in order to support design learning by doing design. We have explained why the representational ecosystem is important during design critiques as it supports communication, design and teaching design. Design is the learning objective of the studio and it is also the main pedagogical strategy embedded in the learning by doing approach of studio teaching. We defined a design activity as an iterative reflective process of constructing mental and external design representations, where the designer navigates between different types of external design representations included in the representational ecosystem. We also highlighted that actions on design representations relate to specific design processes and type of reasoning. Digital tools like VR and augmented tabletops provides an alternative type of representational ecosystem to support design studio pedagogy and can have an effect on its users’ design processes and interactions.

26

J. Milovanovic and J. S. Gero

3 Methodology 3.1

Description of Case Study

This study aimed at exploring the effect of using digital representations in architectural design studio critiques. It specifically focuses on exploring the behavior of tutors and students during the critiques and how they interact with design representations. In order to address the research questions, a case study of three different type of design critiques is presented: desk critiques, Hyve-3D critiques and CDS critiques (Fig. 1). Observations were made in vivo, with no modification of the studio organization, design briefs, critiques’ settings or timings. The first case study is a traditional desk critique where students and tutors used printed plans and sections, as well as physical mock-ups during the critique. Students were master architecture students at the Graduate School of Architecture Nantes (France). The observations took place during the 2018 Spring semester. The requirements were to integrate public equipment into a housing complex, and to develop high environmental quality designs. The concept was developed by students individually using a series of conceptual mock-ups. The sessions observed took place following the selection of an architectural concept. During these critiques, students use concept mock-ups they had previously developed as a representational ecosystem as well as a set of other representations, plans, sections, perspectives drawings. Three students were observed during three critiques in a row. Each critique lasted between 30 and 60 min.

Fig. 1 Example each of the three critique type: a desk critique (up, left), a CDS critique (up, right) and a Hyve-3D critique (down)

Exploring the Use of Digital Tools …

27

The other two cases, Hyve-3D and the Collaborative Design Studio (CDS), offer entirely digital representational ecosystems. The Hyve-3D provides both a 3D drawing interface and an immersion in the design virtual environment. The critiques in the Hyve-3D were observed during the fall semester of 2017 in an architecture master level studio at the Graduate School of Architecture Nantes (France). Students from this studio worked on one of the two proposed briefs. The first brief is the development of a hotel on the theme of Jacques Tati’s movies. The second brief proposes the development and production of a scenography inspired by Tati’s work, which will then be used to shoot a short film and stage plays. Design critiques for this studio often take place with CAD models or with immersive representation devices (cardboard or immersive screen). For one of the critiques, a group of students participated in a Hyve-3D workshop. On the first day of the workshop, the students were trained in the use of the Hyve-3D. In the afternoon, students worked on their design, the hotel or the scenography, on a 45-min timeframe where they could go back and forth between CAD software (SketchUp) and Hyve-3D. The next day, each student individually presented the progress of their design to the studio tutor. The critique took place in the first half of the semester, i.e. in the conceptual exploration phase. Three of the critiques were analyzed. These critiques are quite short as they varied between 10 and 20 min. This timeframe is partly due to the format of the workshop. For this case study, a bias is due to the learning effects of the use of the Hyve-3D. Students had on a short amount of time to learn how to use this tool, and this probably had an impact on the way they presented their design. In addition, the tutor also spent a short amount of time to manipulate the Hyve-3D, which could lead to frustration during these design critiques. The challenge of the CDS is to set up a remote design studio, integrating tools that support collaborative design [33]. Design critiques in the CDS differ from the others because the participants are spread between two sites. The representational ecosystem used is an interactive tabletop where users can draw in 2D on documents (plans, sections, perspectives). These documents, sketches and annotations appear simultaneously on both sites using the SketSha software [34]. The CDS is a master’s studio proposing a group project including architecture/engineering students from the University of Liège (Belgium) and master of architecture students from the Graduate School of Architecture Nancy (France). This remote collaborative studio has been running since 2007 to support collaboration between both universities. Our observation took place during the fall semester of 2017. This multimodal remote collaboration environment operates with a verbal communication interface (Skype) and a drawing interface, SketSha. Both tutors and students were highly trained in using the digital tools. Three groups were observed, each composed of four to five students, two in University of Liège (Belgium) and two or three in Graduate School of Architecture Nancy (France) and two tutors, one at each of the sites. For this studio, students worked on the development of a community center including a boarding school, common rooms, an auditorium, a restaurant and a sailing club. In between studio critiques, students also used SketSha to work collaboratively. These critiques took place in the final phase of the studio and lasted around 40 min each.

28

J. Milovanovic and J. S. Gero

Table 1 Information on cases observed in vivo Duration of critiques Number of participants Design brief Design phase

Desk

Hyve 3D

CDS

30 to 60 min

10 to 20 min 1 student 1 tutor Hotel or decor Concept

41 min

1 student 1 tutor Housing complex and public equipment Advanced concept

4 to 5 students 2 tutors Community center and boarding school Final concept

In vivo observations provide a rich ensemble of design critique situations that carry a set of limitations. Each of the design studios is led by a different pedagogic team, with a different design brief. For one studio, within the same studio, design briefs vary. We also highlighted the differences in the number of participants in the critique, the different observation moments in the studios and the differences in critique length, from 10 min for the shortest to 60 min for the longest, Table 1. These observations were constrained by the real-life context of the studio: students wishing to withdraw from the study, students absent for a critique, or tutors not respecting the time defined for the critique. All these limitations should be taken into account when interpreting our results. Despite the limitations pointed out, the methodological tools used provide a unique framework to highlight similarities and differences between cases, as explored in other research using a similar methodology [35–37].

3.2

Methodological Tools

The protocol analysis methodology [38] is used to analyze each of the critiques as it aims at inferring a cognitive activity based on encoded collected data. The study explored design cognitive processes and designers’ interactions with design representations. Therefore, the protocols, the video of design critiques, were coded with two coding schemes. The first one, dealing with design processes, is based on the Function Behavior Structure ontology [39], and the second, focusing on the manipulation of design representation, includes actions such as pointing to a representation or sketching. We used the Atlas.ti software to code our video protocols. Each protocol is coded twice and then arbitrated by the same researcher who is an experienced FBS coder, with 10 days between codings and between the second coding and arbitration, to obtain more reliable encoded data, on which the analysis is based.

Exploring the Use of Digital Tools …

3.3

29

Using the Protocol Analysis with FBS Ontology

The FBS ontology provides a description of design knowledge and design processes during a design activity [39]. This ontology represents six design issues and eight design processes at the ontological level: Requirement (R) include the design brief, client or regulation requirements; Function (F) is the design object teleology, i.e. what the design object is for; Behaviors represent how the design object performs, it can be an expected behavior (Be) or a behavior derived from the structure of the design object (Bs); Structure (S) is the description of elements or groups of elements of the design object and their relationships; and Description (D) represents externalizations representing the design object (Fig. 2). Eight transformations from one issue to another describe design processes as shown in Fig. 2. Formulation expresses a transformation of a requirement (R) into a function (F) or a function (F) into an expected behavior (Be). Synthesis is the transformation of an expected behavior (Be) into a structure (S). Analysis is the transformation of a structure into a behavior that is derived from it (Bs). Evaluation is the comparison between an expected behavior (Be) and a behavior derived from structure (Bs), and inversely. Documentation is the transformation of structure (S) or less often function or behavior into a description (D), which is the production of any external representation. Reformulation processes always start from a structure (S) that will redefine some variables in the design space. Reformulation 1 is a redefinition of a structure variable (S). Reformulation 2 is the redefinition of expected behavior variables (Be). Reformulation 3 is the revision of function variables (F). The FBS ontology is relevant to explore design cognitive process as its descriptions of function, behavior and structure do not require any additional ontological concepts to describe design issues. Moreover, it has been used extensively to study diverse design situations [40–43].

Fig. 2 FBS ontology based on [39]

30

3.4

J. Milovanovic and J. S. Gero

Analyzing Participants’ Actions on the Design Representations

The second coding scheme corresponds to the actions of the participants (tutors and students) on the representational ecosystem. References [14–23] and studio observations were used to define five categories of interaction with the representational ecosystem: point to a representation, represent a design element with a gesture, draw/ sketch a design element, navigate in a representation and model a design element

Fig. 3 Five types of interactions with design representations: (a) pointing, (b) gesture to represent a design element, (c) sketching, (d) navigating and (e) modeling

Exploring the Use of Digital Tools …

31

(with a mock-up) (Fig. 3). Two of our categories are a type of gestures as gestures support design and collaboration, particularly in the communication of 3D and dynamic elements [21]. We have identified two types of gestures in our videos: a deictic gesture, and an iconic gesture [44]. The deictic gesture is assimilated to the notion of pointing a representation and the iconic gesture aims to represent by a gesture an element of the project. Sketching and drawing are often used at the premises of the design process [45]. Those actions appeared in all the critiques. Navigation in a design representation implies the use of a 2D plan or a 3D model. This applies both to modeling with physical mock-ups and 3D digital models.

4 Results 4.1

Design Collaboration and Role of Participants

How tutors and students interact during the critique, how they co-design and what their roles are during the critique were initially analyzed. Each FBS design processes can be considered individual or collaborative based on the participants who formulated them. Four possibilities appear regarding the construction of processes: the student formulates FBS design processes individually (S > S), the tutor formulates FBS design processes individually (T > T), the tutor formulates the first element of the FBS design process and the student the second (T > S), and inversely (S > T). As mentioned above, some critiques involve several students or tutors, which have been grouped under two participant categories, student and tutor. In each critique, the FBS design processes formulated by the tutor dominated, Table 2. The tutor dominates the critique by verbalizing individual design processes. For critiques in the Hyve-3D, the dominance is the highest (M = 61.3%, SD = 9.8). The distribution of those processes decreases slightly for traditional critiques (M = 52.1%, SD = 12.5) and CDS critiques (M = 52.5%, SD = 16.9).

Table 2 Normalized distribution of design processes per interactions

Mean desk critique SD desk critique Mean Hyve 3D SD Hyve 3D Mean CDS SD CDS

Individual S>S

Co-design S>T

Co-design T>S

Individual T>T

26.1 10.7 14.8 9.6 27.2 11.7

11.2 3.1 9.8 2.1 10.2 3.9

10.6 2.8 14.1 1.9 10.1 2.8

52.1 12.5 61.3 9.8 52.5 16.9

32

J. Milovanovic and J. S. Gero

The distribution of student > tutor co-design processes oscillates around 10% for all cases. The number tutor > student co-design processes is relatively higher in the Hyve-3D case than in the other cases with an average of 14.1% (SD = 1.9) compared to 10.6% (SD = 2.8) for traditional critiques, 9.9% (SD = 2.6) and 10.1% (SD = 2.8) for CDS critiques. It seems that in Hyve-3D critiques, students are more responsive to the tutor’s verbalizations than in other critiques.

4.2

Actions on Design Representation

Between 40 and 70% of the verbalization of design critiques, for all types of ecosystems combined, are accompanied by an action on a representation or by the production of a representation, Table 3. For all design critiques except Hyve-3D ones, tutors are always more active in terms of actions on representations. Tutors dominate design critiques in all cases, which may explain why they are the most active in interacting with representations. In the Hyve-3D, students use sketching and navigation actions more frequently than the tutor. The distribution of gestures to represent an element produced by the tutor increases in Hyve-3D (M = 16.3%, SD = 2.5) compared to their distribution in the desk critiques (M = 3.7%, SD = 3.4) and CDS (M = 4.6%, SD = 2.5). Pointing at a representation is more frequent, for both tutors and students, in the desk critiques than in the Hyve-3D and CDS. For students in desk critiques and CDS, the dominant type of action is to point at a representation. For students in Hyve-3D critiques, the use of navigation in the 3D model is dominant, and interaction through sketching is important (M = 4.6%, SD = 8.0) compared to other representational ecosystems (Traditional M = 0,3% and CDS M = 1,1%). Table 3 Standardized distribution (%) of participants’ actions during design critiques

Traditional Mean SD Gesture tutor 3.7 Gesture student 2.2 Point tutor 27.3 Point student 20.7 Navigate tutor * Navigate * student Model tutor 1.6 Model student 0.6 Sketch tutor 3.7 Sketch student 0.3 No actions 39.9 * Action not possible

Hyve-3D Mean SD

CDS Mean

SD

3.4 1.3 11.0 5.2 * *

16.3 3.5 13.3 2.6 3.6 9.2

2.5 2.5 9.6 2.6 3.7 6.1

4.6 1.3 13.7 13.4 * *

2.5 2.5 9.6 2.6 * *

2.6 1.3 4.6 0.6 9.5

* * 1.7 4.6 45.2

* * 3.0 8.0 25.8

* * 6.9 1.1 59.0

* * 3.0 8.0 25.8

Exploring the Use of Digital Tools …

4.3

33

Connection Between Actions and Design Processes

We pointed out that design representations support multiple types of design processes. We explored how actions on design representation relate to specific design processes. In order to develop a qualitative representation of the associations between design processes and actions on the representations, correspondence analysis is used to represent relative relationship between an action and a design process. We synthesize all the results from the correspondence analysis in Table 4. We only looked at three types of actions since they are the only ones that occurred in all of our dataset. The action of pointing in the desk critiques and Hyve-3D ecosystem is associated with the evaluation processes while in the CDS, this deictic gesture is associated with design description and analysis. For the desk critiques, Hyve-3D and CDS cases, sketching is associated with the processes of reformulating design intentions, which reinforces the importance of this tool for the design critiques. Sketching is also associated with Synthesis for the Hyve-3D and CDS critiques. The use of the gesture to represent an element is associated with different processes depending on the representation ecosystems used: Reformulation 1 for the traditional ecosystem, Synthesis and Reformulation 2 for the mock-up ecosystem, Analysis and Evaluation for the Hyve-3D ecosystem and Synthesis for the CDS ecosystem.

Table 4 Summary of the connection between design processes and actions on design representations Desk critique

Hyve-3D

SDC

Processes link to Evaluation Evaluation Analysis pointing reformulation 1 Pointing •••• • ••• Processes link to Reformulation Synthesis Synthesis sketching 2 reformulation 2 reformulation 2 Sketching • • •• Processes link to a Reformulation Analysis evaluation Synthesis gesture 1 Gesture • •• • The symbol • represents the connection of design processes for each action: • low connection; •• medium connection; ••• high connection

34

J. Milovanovic and J. S. Gero

5 Discussion This case study explored tutor/student interactions during design critiques when using different type of digital design representations. We saw how tutors and students engage in co-design processes, interact with design representations and use gestures and sketching to accompany cognitive design processes. These preliminary results are limited and cannot be generalized due to nature of the case study (small sample size, in vivo observations of studios in different universities, length of each critique varied, and different design briefs). However, these initial findings validate the usability and relevance of the methodology, and provide a base to develop larger and more representative studies in future work. In the following, the findings from this study are articulated and discussed in relation to findings from other studies.

5.1

Effect on Engagement in the Critique and Collaboration

The role of tutors and students in CDS critiques and traditional critiques is similar, while in Hyve-3D critiques, participants engage more easily in co-designing processes. The representational ecosystem used during design critiques can influence collaboration among participants. All participants should be able to communicate in a designerly way through the representational ecosystem to support design collaboration. In all the critiques we observed, students and tutors were able to engage in the critique. We saw in this study that the distribution of collaborative processes tends to be higher for traditional desk critiques and Hyve-3D critiques. Students in the Hyve-3D engage in responding to their tutor’s questions more than the other representations. From this case result we develop the hypothesis that the immersive screen creates a design space that encourages collaboration between participants. The collaborative and rich dialogue between tutors and students during design critiques enhance the development of students’ conceptual knowledge about their design [6]. This strengthens the potential of the use of immersive environment to support design studio critiques in order to enrich students’ learning experience.

5.2

Effect on Interactions with Design Representations

Designing integrates the proposal of a spatial design organization while including a projection of a sensitive spatial experience or felt-paths [46]. In their study, Elsen and Heylighen [47] highlight the relevance of sketching and perspective representations with an egocentric view to communicate the sensory experience, which echoes the notion of felt-paths. Sketching, beyond its ability to provide a representation that communicates a sensitive experience, also supports the concept’s

Exploring the Use of Digital Tools …

35

exploration [45, 48]. The Hyve-3D integrates an immersive 3D sketching interface than can enhance ideation [11] and the communication of the sensory experience. In our observations in the Hyve-3D, the action of sketching is not that frequent compared to the use of gestural actions (pointing and representing by a gesture). For a design session, the use of sketching tends to be more frequent [34]. The pedagogical dimension of the design critiques may be one reason for this difference in the use of sketching because the objective of the critique is to learn how to design and not to design per se. We saw in our case study that tutors tend to sketch more frequently in the traditional desk critique and CDS ecosystem (2D sketching), unlike the students who exploit sketches more frequently in the Hyve-3D (3D immersive sketching). According to Détienne, Visser and Tabary [22], the action of sketching tends to be associated with solution-generation activities while the action of showing (pointing) corresponds to interpretation or information actions. In a study on the relationship between the design process and the manipulation of external representations, Cardella, Atman and Adams [49] showed that designers use sketching to frame the problem and to reformulate it as well. Sketching is used in the observed critiques to reformulate design intentions (Reformulation 2) for traditional desk critiques, Hyve-3D and CDS critiques, which is consistent with the study presented in [49]. For Hyve-3D and CDS ecosystems, where sketching is more widely used, this action is also associated with proposal processes (Synthesis), which are in line with the study presented in [22]. The importance of graphic representations, their manipulation and the use of gestures to communicate and design have been highlighted in many research studies [16, 19, 50, 51]. Gestures are important to support design and collaboration, particularly in the communication of 3D and dynamic design elements [21]. A link is suggested between the action of showing (pointing) and interpretation or information actions [22]. Spatial gesture actions are more frequent in the Hyve-3D design critiques and it tends to be associated with the Evaluation and Analysis processes. For the Hyve-3D, the action of pointing to refer to a design element is not dominant, which can be explained by the immersion of the participants in the design, and the possibility of navigating in the design virtual space. These two features of the Hyve-3D can reduce the ambiguity related to the object being discussed. In summary, we have seen that the representational ecosystems studied here all support collaboration between the participants and provides an environment for participants to communicate in a designerly way in their reflective practice. Nevertheless, we observed differences in participants’ behavior in each case, related to the prevalence of some actions over others and the function of these actions in the mentored reflective practice. The participants in the design critiques interact with the representational ecosystems with similar actions, such as the gesture of pointing or representing an element and sketching. Differences appear in the design function associated with these actions and in the distribution of their use.

36

J. Milovanovic and J. S. Gero

6 Perspectives This study explored several elements that have an impact on students’ experience of the critique such as participants’ engagement in co-design processes and the use of design representations during the critique. In our case study, different connections between types of gestures and specific design processes were found. Sketching tended to be associated with the synthesis of design concept or their reformulation. In this study, students in the Hyve-3D were more engaged in sketching during the design critique than in the other environments and were also more engaged in co-designing. Using a design representation environment that can support this behavior during the critique can promote co-ideation to enhance students’ experience [28], and potentially augment their learning design skills. Sketching is often the focus of studies of design activities but the analysis of gestures should not be discarded as it is an important part of the designing process [20]. During design critiques, communication and collaboration are essential for learning to take place, and gestures can support it. In this study, the use of gesture during the critiques varied in frequency and in the design process associated with this action. A deeper analysis of types of gestures and related processes will enrich the understanding of gestures’ significance concerning students’ learning experience during the critique. Digital technologies such as VR and AR offer a potential to enrich students’ learning experiences in the studios. To understand these potentials and how to exploit it, tools need to be assessed and refined to better support design learning pedagogy. This exploratory study is a first step in that direction and allowed us to test our methodology and tools used to support our analysis: protocol analysis and quantitative analysis. In future work, we will focus on increasing our sample size to statistically confirm the trends found in this study and provide reliable insights on the use of VR and AR digital tools to support design studio pedagogy.

References 1. Schön D (1992) Designing as reflective conversation with the materials of a design situation. Res Eng Design 3:131–147. https://doi.org/10.1007/BF01580516 2. Oh Y, Ishizaki S, Gross MD, Do Y-LE (2013) A theoretical framework of design critiquing in architecture studios. Des Stud 34:302–325 3. Sachs A (1999) “Stuckness” in the design studio. Des Stud 20:195–209 4. Schön DA (1985) The Design Studio. RIBA, London 5. Goldschmidt G, Hochman H, Dafni I (2010) The design studio “crit”: teacher–student communication. Artif Intell Eng Des Anal Manuf 24:285–302. https://doi.org/10.1017/ S089006041000020X 6. Heylighen A, Bouwen JE, Neuckermans H (1999) Walking on a thin line—between passive knowledge and active knowing of components and concepts in architectural design: between passive knowledge and active knowing of components and concepts in architectural design. Des Stud 20:211–235

Exploring the Use of Digital Tools …

37

7. Clark HH, Wilkes-Gibbs D (1986) Referring as a collaborative process. Cognition 22:1–39 8. Clark HH, Brennan SE (1991) Grounding in communication. In: Resnick LB, Levine JM, Teasley SD (eds) Perspectives on socially shared cognition. American Psychology Association, Washington, pp 127–149 9. Darses F (2009) Résolution collective des problèmes de conception. Le Travail Humain 72:43. https://doi.org/10.3917/th.721.0043 10. Dorta T, Kinayoglu G (2014) Towards a new representational ecosystem for the design studio. In: Proceedings of the 19th International Conference on Computer Aided Architectural Design Research, Hong Kong, pp 699–708 11. Dorta T, Kinayoglu G, Boudhraa S (2016) A new representational ecosystem for design teaching in the studio. Des Stud 47:164–186. https://doi.org/10.1016/j.destud.2016.09.003 12. Arias E, Eden H, Fischer G, Gorman A, Scharff E (2000) Transcending the individual human mind—creating shared understanding through collaborative design. ACM Trans Comput-Hum Interact 7:84–113. https://doi.org/10.1145/344949.345015 13. Goldschmidt G (1991) The dialectics of sketching. Creat Res J 4:123–143. https://doi.org/10. 1080/10400419109534381 14. Lawson B, Loke SM (1997) Computers, words and pictures. Des Stud 18:171–183 15. Bilda Z, Gero JS, Purcell T (2006) To sketch or not to sketch? That is the question. Des Stud 27:587–613. https://doi.org/10.1016/j.destud.2006.02.002 16. Suwa M, Tversky B (1997) What do architects and students perceive in their design sketches? A protocol analysis. Des Stud 18:385–403 17. Goel V (1995) Sketches of thought. MIT Press, Cambridge 18. Arnheim R, McNeill D (1994) Hand and mind: what gestures reveal about thought. Leonardo 27:358. https://doi.org/10.2307/1576015 19. Kang S, Tversky B (2016) From hands to minds: gestures promote understanding. Cogn Res: Princ Implic 1:1–15. https://doi.org/10.1186/s41235-016-0004-9 20. Visser W, Maher ML (2011) The role of gesture in designing. Artif Intell Eng Des Anal Manuf 25:213–220. https://doi.org/10.1017/S0890060411000047 21. Visser W (2009) The function of gesture in an architectural design meeting. In: McDonnell JT, Lloyd P (eds) About designing: analysing design meetings. CRC Press, Boca Raton, pp 269–284 22. Détienne F, Visser W, Tabary R (2006) Articulation des dimensions graphico-gestuelle et verbale dans l’analyse de la conception collaborative. In: Psychologie de l’interaction, L’Harmattan, pp 283–307 23. Schnabel MA, Kvan T, Kruijff E, Donath D (2001) The first virtual environment design studio. In: 9th eCAADe Conference Proceedings, Helsinki, Finland, pp 394–400 24. Bertol D, Foell D (1997) Designing digital space: an architect’s guide to virtual reality. Wiley, New York 25. Milovanovic J, Siret D, Moreau G, Miguet F (2017) Enhancing design representational environment to support design learning in the studios. In: Envisioning Architecture: Space, Time, Meaning. 13th Biennial International Conference of the European Architectural Envisioning Association, Glasgow, United Kingdom 26. Angulo A (2015) Rediscovering virtual reality in the education of architectural design: the immersive simulation of spatial experiences. Ambiances Environnement sensible, architecture et espace urbain. https://doi.org/10.4000/ambiances.594 27. Sopher H, Kalay YE, Fisher-Gewirtzman D (2017) Why immersive? Using an immersive virtual environment in architectural education. In: Fioravanti, A et al (eds) ShoCK! - Sharing Computational Knowledge! Proceedings of the 35th eCAADe Conference - Volume 1, Sapienza University of Rome, Rome, Italy, pp 313–322 28. Boudhraa S, Dorta T, Milovanovic J, Pierini D (2019) Co-ideation critique unfolded: an exploratory study of a co-design studio ‘crit’ based on the students’ experience. CoDesign 15. https://doi.org/10.1080/15710882.2019.1572765

38

J. Milovanovic and J. S. Gero

29. Garcia AR, Marquez J, Valverde Vildosola M (2001) Qualitative contribution of a VR-system to architectural design: why we failed? In: Gero JS, Chase S, Rosenman MA (eds) Proceedings of the Sixth Conference on Computer-Aided Architectural Design Research in Asia. Sydney, Australia, pp 423–428 30. Kalisperis L, Otto G, Muramoto K, Gundrum J, Masters R, Orland B (2002) An affordable immersive environment in beginning design studio education. In: Proceedings of the 2002 Annual Conference of the Association for Computer Aided Design in Architecture, Pomona, California, pp 47–54 31. Schubert G, Anthes C, Kranzlmuller D, Petzold F (2012) From physical to virtual: real-time immersive visualisations from an architect’s working model. In: CONVR, pp 417–426 32. Ishii H, Ben-Joseph E (2002) Augmented urban planning workbench: overlaying drawings, physical models and digital simulation. In: Proceedings of IEEE & ACM ISMAR, pp 203– 211 33. Safin S, Leclercq P (2009) Studio Digital Collaboratif: un environnement de conception collaborative à distance. In: Actes Colloque Interaction Homme-Machine. Grenoble, France, p 3 34. Elsen C, Leclercq P (2008) “SketSha” – the sketch power to support collaborative design. In: Luo Y (ed) Cooperative design, visualization, and engineering. Springer, Heidelberg, pp 20–27 35. Kan JW, Gero JS (2017) Quantitative methods for studying design protocols. Springer, Dordrecht 36. Gero J, Jiang H (2014) Exploring the design cognition of concept design reviews using the FBS-based protocol analysis. In: Adams RS, Siddiqui JA (eds), Analyzing design review conversations. Purdue University Press. https://doi.org/10.5703/1288284315931 37. Gero JS, Kannengiesser U, Pourmohamadi M (2014) Commonalities across designing: empirical results. In: Gero JS (ed) Design Computing and Cognition’12. Springer, pp 285–302 38. Ericsson KA, Simon AH (1984) Protocol Analysis: Verbal Reports as Data. MIT Press, Cambridge 39. Gero JS (1990) Design prototypes: a knowledge representation schema for design. AI Mag 11:26–36. https://doi.org/10.1609/aimag.v11i4.854 40. Bott M, Mesmer B (2019) Determination of function-behavior-structure model transition probabilities from real-world data. In: AIAA Scitech 2019 Forum, San Diego, California. https://doi.org/10.2514/6.2019-1030 41. Neramballi A, Tomohiko Sakao T, Gero JS (2019) What do experienced practitioners discuss when designing Product/Service Systems? In: Gero JS (ed) Design Computing and Cognition’18. Springer, Cham, pp 361–380 42. Hamraz B, Clarkson PJ (2015) Industrial evaluation of FBS Linkage – a method to support engineering change management. J Eng Des 26:24–47. https://doi.org/10.1080/09544828. 2015.1015783 43. Yu R, Gero JS (2016) An empirical basis for the use of design patterns by architects in parametric design. Int J Archit Comput 14:289–302. https://doi.org/10.1177/ 1478077116663351 44. Kida T (2011) Nouvelle méthode de constitution d’un corpus pour transcrire gestes et intonations. Corpus 10:41–60 45. Purcell AT, Gero JS (1998) Drawings and the design process. Des Stud 19:389–430. https:// doi.org/10.1016/S0142-694X(98)00015-5 46. Schön DA (1981) Learning a language, learning to design. In: Porter WL, Kilbridge MD (eds) Architecture education study, Volume 1: The papers. Andrew W. Mellon Foundation 47. Elsen C, Heylighen A (2014) Representations of sensory experiences in the early phases of architectural design: there is more than meets the eye. J Des Res 12:239. https://doi.org/10. 1504/JDR.2014.065846 48. Schön DA, Wiggins G (1992) Kinds of seeing and their functions in designing. Des Stud 13:135–156 49. Cardella ME, Atman CJ, Adams RS (2006) Mapping between design activities and external representations for engineering student designers. Des Stud 27:5–24. https://doi.org/10.1016/ j.destud.2005.05.001

Exploring the Use of Digital Tools …

39

50. Heiser J, Tversky B, Silverman M (2004) Sketches for and from collaboration. In: Gero JS, Tversky B, Knight T (eds) Visual and spatial reasoning in design III. Key Centre of Design Computing and Cognition, University of Sydney, Sydney, pp 69–78 51. Tang HH, Gero JS (2001) Sketches as affordances of meanings in the design process. In: Gero JS, Tversky B, Purcell T (eds) Visual and spatial reasoning in design II. Key Centre of Design Computing and Cognition, University of Sydney, Sydney, pp 271–282

Modelling the Dynamics of Influence on Individual Thinking During Idea Generation in Co-design Teams Harshika Singh, Christopher McComb, and Gaetano Cascini

Abstract Social influence is not evenly distributed in teams. Some individuals, referred to here as influencers, become more influential than others. Consequentially, these influencers play a significant role in shaping project performance. The current work simulates the presence of influencers during idea generation in co-design teams to better understand emergent socio-cognitive phenomena. Besides providing, a novel approach for modelling learning in concept generation the model highlights the results related to individual cognition during idea generation. Idea quality and exploration of design space are affected by the presence of influencers in design teams. Teams with no well-defined influencers produce solutions with high general exploration but less quality. In contrast, the agents in the teams with only one influencer produce solutions high quality than those teams with no influencers.

1 Introduction Co-design is beneficial for innovation because it employs teams of individuals with different skills and ideas [1]. More specifically, co-design is a process used for creating products and services by involving different viewpoints and stakeholders. Implementing co-design has become a trend over traditional practices of design [1]. Research has been conducted to study the factors that hinder or facilitate these collaborative team activities [2]. This has been done either by studying the design outcome or the design process. Currently, more emphasis is being given to explore the factors from individual to project level [2], which influence design process, and consecutively affect final performance. Moreover, examining the design process is a potentially time-consuming practice [3]. Therefore, this research aims at providing a H. Singh (&)
G. Cascini Politecnico di Milano, Milan, Italy e-mail: [email protected] C. McComb Carnegie Mellon University, Pittsburgh, USA © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. S. Gero (ed.), Design Computing and Cognition'20, https://doi.org/10.1007/978-3-030-90625-2_3

41

42

H. Singh et al.

quicker approach to study co-design team dynamics while keeping in mind the relevant socio-cognitive features. The work presented in this paper deals with an agent-based model approach for simulating idea-generation in co-design teams. Besides providing a novel approach to model learning abilities in agents, it also investigates the effect of the influencers on individual thinking during idea generation. The effect of influencers during idea generation is potentially significant but has been unaccounted in the literature. The early results obtained in this work show how solution quality and exploration of the design space are affected by changing the number of influencers in a team during idea generation. The wider focus of the research work is to simulate the entire co-design process based on the framework provided by [4] for assisting future researchers and practitioners. Specifically, the model would provide them with a faster digital approach to understand how people interact in design teams by varying the input parameters to fulfil the purpose of their work. The structure of the paper is as follows; the following section is about the past literature in this domain. It also identifies a research gap and highlights the main contribution of the paper. This section is followed by a detailed methodology on which the model is built and some early results of the simulation are mentioned. In the end, a conclusion provides a summary of the paper along with the limitations and the future goals.

2 Background Agent-based modelling is a relatively new computational approach to model a dynamic phenomenon. It is a quicker, convenient approach for modelling individual behavior and heterogeneous interactions, where individuals (agents) exhibit characteristics such as memory, learning and adaptation [5]. These agents behave according to a set of rules assigned to them to fulfil the purpose of the model [5]. Agent-based modelling has been used in many different domains, from social sciences to medical fields [6]. Here, it is being used to model co-design team dynamics. Simulating a co-design activity involves many parameters [7] and it is unfeasible and often difficult to consider all the parameters in the model. Some authors have investigated problem-solving in design teams [8, 9], while others have proposed models based on team expertise, team experience on task performance [10, 11]. Individual attributes, such as the choice of partners or cognitive style [12, 13] and social attributes have also been modelled in the past [14]. Various former models were focused on simulating project-based design teams [10]; thus, a phenomenon of the same designers working on a certain task for a long period is still underexplored. The above-mentioned trend is most common in small and medium enterprises that have the same set of individuals working on similar problems for years. As a result, the model in this paper is based on this scenario where the same designers work on a complex routine design task as the solutions vary from person to person based on their competence [15].

Modelling the Dynamics of Influence on Individual Thinking …

43

The type of design task alters teams’ performance; hence, it is crucial to define a task fitting the purpose of the model. Design tasks in some computational models mentioned above have been demonstrated as binary functions [16] or have been decomposed into sub-design tasks to serve their purpose [14]. Often these design task representations have extreme solution values, (i.e. immediately next to the best solution, there is the worst solution) which in real world is an inaccurate representation of a more stable design task with robust solutions i.e. they have less variations or a gradual slope (of intermediate values) between the best and the worst solution. Agent-based modeling often involves imbuing agents with the ability to learn about their environment. Some researchers have modelled social learning by means of mental models [15] or learning from doing the task [13, 17]. While simulating learning, it is often assumed that agents know the design solution space and therefore pursue optimal solutions. However, in the proposed model the agents do not directly know any value of the solutions in the design space, but at the same time, they are aware of the design variables and the boundary conditions. This is similar to real situations where the design solution space is not completely known to the team of individuals working on a design problem, and they learn from their previous solution results and from others in the team. Learning from doing or experience is modelled by considering several individual characteristics (see the section on Model description). The rationale behind why and how learning from others in the team or social learning is explained below. Whenever there is human interaction, there is social influence. Social influence may cause individuals to modify their opinions, attitudes and behavior to be similar to the others they are interacting with [18]. This imitation is embedded in human nature and is referred to here as social learning [19]. However; this social influence is not equally distributed in the team. Some individuals are more influential; the capacity to persuade others is not necessarily spread evenly among team members [20]. There may be some individuals who are regarded as more influential in their views and judgements than others [20]. In this work, these relatively more influential individuals are referred as influencers [21]. These influencers play a significant role in shaping the project performance. Many researchers have tried to study the traits, attitudes and behavior makes someone more influential than others. As proposed by Baker (2015), an individual’s personality, skills and communication could result in such phenomenon. In this work, individuals’ self-efficacy is taken as one of the traits that could affect their personality, skills and communication [22]. Self-efficacy, itself depends on intrinsic and extrinsic motivation [24]. Furthermore, how well the two individuals have known each other previously or the ‘strength of ties’ between them [23], is one of the factors accountable for trust is also considered in this model for determining the degree of influence. Trust depends on the interacting individual’s familiarity and reputation [25, 26] (see Fig. 9). Social dynamics within a group can influence individual performance, hence affecting team performance [41]. Thus, the main significance of the paper lies in its contribution towards providing (1) a novel approach to simulate learning ability in agents when the solution space is unknown, and (2) simulating the ‘effect of

44

H. Singh et al.

influencers’ during idea generation. Overall, the research aims to improve the understanding of design teams and intervene in the team dynamics as necessary for better project outcomes. The objective of the work presented in the paper is to assess the effect of the influencers on individual thinking during idea generation in design teams. Hence, based on this research objective, the paper describes a suitable methodology that forms the foundation of a simulation along with some results showing the functioning model.

3 Model Description As mentioned above, the wider purpose of the research is to generate a computational model for an entire co-design activity, the framework of which can be seen from Fig. 1. The co-design activity consists of a project on which teams work. Usually, a project consists of a number of design sessions of idea generation and selection before the final project outcome is reached. In each idea generation activity, an agent takes a certain number of steps (explores multiple solutions) before being ready with one which it shares with the team. The work presented in this paper only simulates idea generation sessions where individuals work on a robust design task without considering the gaps between the sessions. These gaps are the pauses or breaks between the two ideation sessions, essential for the incubation effect (not considered in the scope of this paper). The results stated are related to individual thinking during brainstorming in idea generation where agents at this point generate, explore, evaluate and select solutions [27]. The collective teamwork on idea selection is out of the scope of this paper.

Session 1

Gap

Session 2

Gap

Session 3

Session n

Project

Fig. 1 The focus of the study presented in the paper

Modelling the Dynamics of Influence on Individual Thinking …

3.1

45

The Design Task

Like any co-design activity, the simulation starts with a design task. The controller agent (real-world equivalent to a project leader, project manager or a professor) gives the agents (individual humans who work in a team) a task and the agents have to find solutions to the design problem. In computational terms, the design problem is n-dimensional that consists of a landscape function f(x) (given below in Eq. 1) constructed from the pre-defined best solution points, and agents aim to find the best solutions. For initiation, simplification and visualization purposes, a 2D function is representing the design problem; however, it could be extended to multiple dimensions for the upcoming articles. The 2 dimensions in a design space represent two notional design variables to explore and the values perpendicular to the design variables define the quality of the solutions. The values of f(x) represent the solution space, which has a maximum value of 1 (lightest hue) and minimum 0 (darkest hue), as shown in an example in Fig. 2 with several local maxima and minima. f ð xÞ ¼ ð1 þ e

1   ð p1ffiffi d2Þ N

ð1Þ Þ

where N is the number (size) given to represent the solution space in 2D matrix. In this case, N = 100, such that the solution space was represented as a 100-by-100 matrix. Here, d represents the distance between the random point (x,y) and the nearest best solutions. While mathematically, representing the solution space it was taken into consideration, the real-life design problem-solving, where there is a low probability of having immediate extreme values. This means that there is a low chance of Fig. 2 An example of a design solution space with a side bar showing solution values

46

H. Singh et al.

having the best and the worst solutions placed next to each other. The design problem resembles a case where its solutions have less variation among its immediate surrounding solutions. In other words, the agents are solving a design problem where in most cases the solution generated by them will not extremely vary from the values of its neighboring cells. Likewise, the solution space with this landscape function f(x), is modelled such that there is a gradual decrease in the hue around the best solutions. Some noise is added to the design space so that the probability of having maximum and minimum solution values right next to each other is not completely eliminated. Like any other design problem, this design problem representation also has multiple best solutions.

3.2

Agent Generating Solutions

After the design task is given, the agents start generating solutions. Similar to brainstorming, the agents first individually generate solutions then communicate with the team to further build on them. The paper is only focusing on reporting the method and results related to the state of agents when they are individually thinking during ideation (while the entire process described in Fig. 1 is operating in the backend). An agent generates solutions based on the characteristics given below. These are related to general human behavior related to thinking in idea generation. Certainly, there are many cognitive and social factors influencing individual brainstorming that are complicated to mimic, however, as identified in a review of the literature, these are most salient. • • • • •

Their way to explore solution space Memory to store past experiences Recall capability Ability to learn from failure and successful past experience Influence of the influencer(s) (as explained in the Background section above)

3.3

Agent’s Desire to Explore Solution Space

Exploration of the design solution space is based on the fact that individuals during the initial ideation phase are slower in exploring the solutions as they get warmed up in by triggering memory search. This is followed by more exploration by recalling input from their memory. However, at some point, this recalling process becomes tiring, and exploring the solution space drops towards the end of the session [28] (see Fig. 3). This behavior of individuals is modelled in the agents as in Eq. 2 below. Changing the shape parameter of the curve (r), makes it possible to generate different exploration styles, assignable to different agents.

Modelling the Dynamics of Influence on Individual Thinking …

47

Fig. 3 An example plot of the curve O(x′)

O ð x0 Þ ¼

1 pffiffiffiffiffiffi e x0 r 2p





lnðx0 Þ 2r2



þc

ð2Þ

In the given equation, x′ is x/7 (x  0). The value of r lies between 0 < r  1, it represents the shape parameter which affects the overall shape of the curve. c is the value of exploration when the session starts and it varies from agent to agent.

3.4

Memory

The simulation setup as defined at the beginning of the paper imitates situations found in many small and medium enterprises, where the agents work on a similar design problem for many sessions. Accordingly, an agent has a different memory capacity and stores results from the sessions in the past experience from working on these design problems. These experiences are in the form of success or failure encountered in the past when doing a design task. From the memory of an agent, the stored element is forgotten when it is not recalled for a long time. This forgetting is based on the Decay Theory of Forgetting where it suggests that if there was no attempt to recall an event, the greater the time since the event the more likely it would be to forget the event. Thus, the agents exhibit the behaviour that suggests that memories are not permanent [29].

3.5

Recall Capability

Recalling, on the other hand, refers to the act of bringing a past event back to one’s mind. In real situations, an individual might not be able to recall any similar experience from the past while approaching a problem. Similarly, in the model, an agent has its successes and failures in its memory but it might not be able to recall while solving the problem. In addition, if an agent recalls events from the past that might alter the way it approaches the solution. The model takes into account free recall, where individuals can recall events in any order [30]. The recalling power is different from agent to agent and depends on the intensity of the solution value and the time of recall as explained by [31]. In the real world case, where individuals recall their worst and best events results more clearly than their mediocre outcomes.

48

H. Singh et al.

The events are also recalled based on the recency and primacy effect [30] which means that the recent and the first events are recalled more frequently than the middle ones. In real world, the mediocre results of events that happened a long time ago are more difficult to recall than recalling the best/worst result that happened at the same time. Moreover, recalling the recent best/worst events is easier than past events with similar results. Hence, the model is based on similar features where agents have a higher probability to recall their recent best/worst solutions than old ones.

3.6

Learning from Experience

The agents in the model have the capability to learn from their past success or failure events, which had occurred in the previous sessions, it could be seen from the example shown in Fig. 4. Learning from past success and failure are different as they have a different impact on the current situation. This implementation of learning from the success of an agent on its current solution depends on the following factors (Fig. 5): 1. Similarity between the current solution ‘in mind’ and recalled successful solution. If the recalled success is similar (closer on solution space) to the solution ‘in mind’, the agent is more influenced by its previous success than the success that is far in distance (not so similar) [32]. However, if the recalled success is too close or same as the solution ‘in mind’, an agent is less influenced by it.

Fig. 4 An example showing an agent recalling events while exploring solutions

Modelling the Dynamics of Influence on Individual Thinking …

49

Fig. 5 Different amount of learning from one’s own success

2. The amount of learning the previous success depends on the (i) experience of an agent, i.e. the position of the peak of the learning curve. It means that when an agent is more experienced, it will learn faster therefore a steeper slope than the agent who has lesser experience [33]. (ii) The time when the successful event occurred, i.e. the height of the learning curve from success i.e. more height/ intensity when the success was recent. The amount of learning from success recalled (magnitude of the success vector as shown in Fig. 6) can be represented by S(d’) in Eq. 3. 0

1 2p Bd 0 apffiffiffi

Sð d 0 Þ ¼ s @

 e



1

ðlnðd 0 ÞÞ 2a2

0:7

C A

ð3Þ

d 0 ¼ 4:0  d þ 0:1. Here d′ is the adjusted value of d such that 0  S(d’)  1. d is the similarity between the current design task and recalled success experienced task. In computational terms, d is the distance between the current agent (solution) position in session n and recalled success (solution) position of session Sn. (In Eq. 3 S(d’) is divided by 0.7 to get the desired value within 0–1). The other variables in the above equation are explained below:



ð3:1Þ

s ¼ 1  ð0:7  DtÞ

ð3:2Þ

Dt ¼ n  Sn =N;

ð3:3Þ

where n is the current session number of an agent and Sn is the session when the recalled success occurred. N = number of sessions.

50

H. Singh et al.

Fig. 6 The updated position on an agent is the sum of the vectors of the intended direction and the direction of the recalled success (20  20 Solution space only for the zoomed in visualization)

Success and failure have a different impact on the current situation. For example, humans try to avoid the failures they have committed in the past and tend to follow the path that led to previous success [34]. Unlike learning from success, learning from failure is done by forming circles of a certain radius r around the failed solution value. The circle is constructed by the agent around the failed solution whose radius varies from agent to agent. Similar to the real scenario where an individual remembers the failure zones on the solution space while exploring new solutions. The learning from failure depends on the recalled failure value (Fig. 7), where an agent learns maximum when the failure was severe. The radius or the size of the circle denotes the learning capacity from a failure of an agent and it will avoid the circle area around the recalled failure (Fig. 8). Fig. 7 Failure radius depends on the value of the recalled failure (where 5 units is the max radius for a 100  100 units of solution space)

Modelling the Dynamics of Influence on Individual Thinking …

51

Fig. 8 An example where an agent (in red) encounters a failure at session n which is being recalled in session n + 1, an area around the failure is avoided. (20  20 Solution space only for the zoomed-in visualization)

Fig. 9 Determining influence value

3.7

Effect of the Influencers

As mentioned in the previous section, some individual(s) in teams are regarded more influential in their views and judgements than others [20]. Therefore, in order to model this ‘influencing effect’, each agent has an influencing value from other agents in the team and it depends on the factors shown in Fig. 9. The influence value I (magnitude of the vector), for an agent i (in an example in Fig. 10 as agent A4) of agent j (as agent A2) is computed as Eq. 4: I ij ðDSE; T Þ ¼

ðDSE 1:5 þ T ij Þ 2

ð4Þ

where, DSE = difference in self-efficacy of agent i and agent j, T is the trust of agent i on agent j.

52

H. Singh et al.

Fig. 10 The updated position on an agent is the sum of the vectors of the intended direction, the direction of its own recalled success and the influence value vector (20X20 Solution space only for the zoomed-in visualization)

TðR; FÞij ¼ ðR j þ F ij Þ=2

ð4:1Þ

Trust, T of an agent i on agent j depends on R and F [25]. R is the reputation of an agent j and F is the familiarity of an agent i with agent j. Familiarity, F between two agents, is the number of session agents i and j have worked together, therefore familiar with each other. R ¼ Na =Np

ð4:2Þ

where, Na is the number of solutions that are accepted by the controller agent and Np is the total number of the solutions proposed by an agent. The increase in extrinsic motivation occurs when the controller agent accepts the final solution of the team. An increase in intrinsic and extrinsic motivation increases self-efficacy [24]. Other studies state that individuals contribute more in idea generation when the team accepts ideas from them [35]. This is due to their increase in intrinsic motivation that increases their self-efficacy. It was also found that individuals with high self-efficacy are less likely to be demotivated when other team members do not select their ideas [35]. Taking these findings into account, the agents in the model demonstrates similar behavior, where an increase/decrease in an agent self-efficacy depends on their current state of self-efficacy. Agents who have the highest and the lowest self-efficacy get a gradual boost in their existing state of self-efficacy than the agents having a moderate amount of self-efficacy. Moreover, highly motivated agents have a more steady decrease in their self-efficacy than the lowest self-efficacy agents who are more rapidly demotivated.

Modelling the Dynamics of Influence on Individual Thinking …

53

4 Methodology In order to address the research objective, which is to find out the effect of the influencers on individuals’ idea generation, the team self-efficacy distribution was varied while keeping other parameters (team size, team familiarity and design task) constant. To check the functionality of the model, two scenarios were framed and tested. The first scenario tested the situation when the team has high variance in the self-efficacy of its agents, i.e. some agents have high self-efficacy and others low when they start working on a design task. Three sub-scenarios here were: 1. One agent with high self-efficacy and others with low 2. Two agents with high self-efficacies and others with low 3. Three agents with high self-efficacies and others with low The second scenario tested the situation when the team has low variance in the self-efficacy of its agents, i.e. all agents either have high or low self-efficacy when they start working on a design task. Two sub-scenarios here were: 1. All with low self-efficacies (i.e. no influencer) 2. All with high self-efficacies (i.e. all influencers) These two scenarios would help in understanding the team dynamics that affect design output due to the presence of unequal social influence. Thus, in order to see the functionality of the model, the findings are related to (i) difference in learning, (ii) quality of the solutions and (iii) exploration of design space for measuring the design task outcome [36, 36]. The quality of the solution is the value of a 2D point on a design solution space, in order words; it is the value of the design task f(x) defined earlier in the paper. On the other hand, the exploration values are calculated in three different ways:

4.1

Exploration Quality Index (EQI) EQI ¼

no. of solutions [ t total soln present in solution space [ t

ð5Þ

Exploration quality index is the ratio of the number of the explored solution above a certain threshold, t (in this case t is above 0.5, where 0 is min and 1 is max solution quality value) on a reduced solution space (i.e., by a factor of 5 units hence, 20  20) to the total number of solutions available on the design solution space greater than the threshold value. This means that if an agent explores neighboring solution cells, the mean of the solution values of these cells is taken. It was done to avoid having an inaccuracy in the exploration quality that could arise, e.g. when an agent explores 5 immediate neighbor cells to an agent exploring 5 cells at a larger distance

54

4.2

H. Singh et al.

Exploration Index (EI) EI ¼

solutions explored on a reduced solution space reduced solution space area

ð6Þ

Exploration index is the number of solutions explored on a reduced solution space to the area of the reduced solution space. Reduced solution space is the original solution space (100  100) is decreased in size by a factor (5 in this case) so that the resultant is a smaller space (20  20). This means that if an agent explores neighboring solution cells, it is counted as one unit exploration. It was done to avoid having an inaccuracy that could arise, e.g. when an agent explores 5 immediate neighbor cells to an agent exploring 5 cells at a larger distance. Dispersion of the solution values is the mean of the dispersion of the solutions from the centroid of the solutions. Spread or the dispersion of the solutions obtained was calculated to see how different the solutions are from each other (variety of the solutions). The above-mentioned model description and methodology show how details of socio-cognitive characteristics of individuals and teams were taken into consideration during simulating idea generation. The goal of the work is not to get the optimal solution values, but to understand how individual thinking in design teams are affected by the presence of influencers during idea generation. Thus, the model is based on theoretical and empirical findings to mimic ‘real-world’ idea generation. Some of the early findings of the model are stated in the next section.

5 Results and Discussion Figure 11 shows the average change in solution quality over 1000 simulations, comparing agents that learn from their success and failure (with the effect of the influencer) with those that do not. The increase in the quality of ideas with each session could be due to recall, which is correlated with the number of ideas generated [42]. However, the way an agent with high self-efficacy (but lesser than the self-efficacy of an influencer) behaves during idea generation, is different from an agent with low self-efficacy in a team where there is an influencer. Figure 12 shows the distance between the solutions of a low and high self-efficacy agent with respect to an influencer (here the maximum sessions were 20). It can be seen that an agent with high self-efficacy explore solutions different from the influencer while the low self-efficacy agent generates solutions closer to that of an influencer. This aligns with expectations on the nature of influence in design teams. Since another parameter familiarity (f, as given in above the equation for trust (T)) that contributes to the degree of influence was constant, agents defined with high self-efficacy controlled the team processes (influencers). Figures 13 and 14 below show teams with a varying number of influencers learn from their success and failure. Learning from the past events in the form of success and failure (as

Modelling the Dynamics of Influence on Individual Thinking …

55

Fig. 11 Agent learning from past experience

Fig. 12 Distance between low and high self–efficacy agents from the influencer (for maximum sessions = 20)

explained in the section on Model description), where agents avoid the failures they have committed in the past and tend to follow the path that led to previous success. The curves shown in these figures are similar to those proposed empirically and theoretically in other works [38]. The teams in which all agents start at high self-efficacy (‘All influencers’) have a greater ability to learn from failure than the other combinations tested. With respect to learning from success, all the agents in the team with ‘No influencer’ or all agents with low self-efficacy, learn least from their own success. Social influence, which leads to the imitation in individuals to modify opinions, attitudes and behavior similar to the others they are interacting with, is referred to as social learning. As the influence of individuals is unevenly distributed in a team, consequently is social learning. The amount of social learning as expected is maximum in the teams of 3 influencers, while minimum when all agents have low self-efficacy (Fig. 15).

56 Fig. 13 Learning from failure

Fig. 14 Learning from Success

Fig. 15 Social learning

H. Singh et al.

Modelling the Dynamics of Influence on Individual Thinking …

57

Computationally, solution quality is the function value of a specific point in a design solution space. Figure 16 shows the mean solution quality of each agent in a team over a project of 10 sessions. There was no significant difference in the mean solution quality of each agent in a team however, some minor differences can be observed. As shown in Fig. 16, when all the agents in a team start working with low self-efficacy, they maintain consistently low performance. Interestingly, teams with only one influencer have the highest mean solution quality from session 2 onwards. Agents in teams of all the remaining combinations tested (2, 3 and all high self-efficacy), have similar mean solution quality at the end of the project (last session). The quality results of the model are consistent with the study done by [43], where it was shown that exposure to others’ ideas, may increase the quality of ideas generated. In contrast to [43], the results (normalized values) related to the exploration of the design solution space are shown in Fig. 17. It can be noted teams with small/no variations in their self-efficacy, have the highest exploration index while the least global quality exploration index, as found in [9] where teams who diverge less and focus on certain areas on design space perform better. The agents in these teams explore more areas of design solution space but encounter fewer above average quality solutions. This could mean that due to their low self-efficacies, they are less influenced by their own success; hence, they keep exploring new areas on solution space without producing a higher quality of exploration index. The agents in teams with 1 influencer have less exploration and more global quality exploration index, which decreases as the number of influencers are increased. The solutions of the agents in the teams of one influencer are most dispersed (different from each other) than other team compositions. The exploration rate, i.e. the number of solutions in a design space explored during a session, without considering the ones in the previous session is shown in Fig. 18. It can be noticed that the teams with well-defined influencers have relatively similar exploration rates over a different session and the exploration rate during sessions 3–6 is lower than in other sessions. It could be deduced that the influencers affect the exploration of design space somewhat in the middle of the project. Fig. 16 Quality of solutions

58

H. Singh et al.

Fig. 17 Normalized exploration values

Fig. 18 Session-wise exploration rate

It can also be seen in Fig. 18, where the teams are with no defined influencers, i.e. all the agents are either with low or high self-efficacy, have significantly different exploration rates as compared to the previous scenario. In the case of teams with all low self-efficacy, the exploration rate decreases gradually towards the end of the project. On the other hand, the exploration rate in the teams of all high self-efficacy agents abruptly drops after mid-project as these agents were more confident in their own solutions than others initially. It could be inferred that as the agents in the teams approach the end of the project, some agents among them might have started to emerge like influencers, hence affecting others in the exploration. It is known that social influence affects creativity. Some authors say that it restricts creativity by limiting variety while others say that it enhances quality [39]. Group-level creativity is a “function of the extent to which social influences affect individuals within the group at earlier stages” [39]. This behavior of individuals in design teams was clearly seen in the results during ideation sessions in the teams of uneven distribution of influence. Undoubtedly, more work needs to be done to see

Modelling the Dynamics of Influence on Individual Thinking …

59

how influencers in the design team affect team and organization creativity. On the other hand, the length of a session (time allotted for generating ideas) and the number of sessions also influence the exploration rate of design space [40]; as well, needs more consideration.

6 Conclusion This work investigates the unequal distribution of social influence within teams, where the agents with high influence on others, are referred to as influencers. This research specifically investigated the effect of influencers on individual team members’ exploration and quality as mediated by influence. The team of agents works on a project, which consists of multiple co-design sessions with idea generation and selection. Like any other co-design session, the simulation starts with a design task given to a team of agents who have to produce solutions. Similar to the output of a real world idea generation, solution quality and exploration of the design space were considered as parameters to determine idea creativity in the model. The summary of the results related to quality and exploration are as follows: • The results show that the agents in the teams without influencers have the least social learning and the least learning from their failure. Consequently, they also the lowest solution quality during all the sessions. However, agents in teams composed entirely of influencers learn the most from their failures. Interestingly, the agents in teams with only a single influencer have the highest solution quality at the end of a project. As the influencer(s) in the team also influence other agents’ solution quality during brainstorming as stated in [43]. • Coherent to the past literature, which support that the quality and exploration are negatively related, the teams in scenario 2 (teams without well-defined influencers) explore design space more than the ones in scenario 1 (teams with well-defined influencers), while having a lower global quality index. Similar to the above quality result, teams with one influencer have the highest global quality exploration index and the spread (dispersion) of the solutions. • The session-wise exploration rate was lowest during the middle and gradually increased towards the end of the project for teams in scenario 1. However, significantly different behavior was observed in teams in scenario 2 where the exploration rate decreased towards the end of the project. This suggests that the effect of the influencers is most prominent in the middle of a project.

6.1

Limitations

Although, the model provides insights related to different team compositions and ideation output, the ability of agent-based modelling to mimic human behavior is

60

H. Singh et al.

fundamentally limited. The results provided here are based on a simplified two-dimensional representation of the design space, which is not necessarily equivalent to actual design problems. The interactions and relations between different agent behaviors may have confounding effects and thus, the validation of the simulation framework is needed. A direct validation study should be a subject of future work. In addition, observational studies will be conducted on idea generation to further tune the model. Although this representation was adequate for the results produced here, future work should expand the design space to multiple dimensions. Moreover, future work should perform real human experiments to validate the results of the model and provide experimental feedback to improve the model. Ultimately, the inference from the model could provide a faster approach to study co-design activities. The outcome of the research could also assist in suitable people management strategies by project managers, leaders, tutors, facilitators and other leaders of problem-solving teams, making it more feasible to obtain near-optimal project outcomes.

References 1. Steen M, Manschot M, De Koning N (2011) Benefits of co-design in service design projects. Int J Des 5:53–60 2. Kleinsmann M, Valkenburg R (2008) Barriers and enablers for creating shared understanding in co-design projects. Des Stud 29:369–386 3. Becattini N, Cascini G, O’Hare JA, Morosi F (2019) Extracting and analysing design process data from log files of ICT supported co-creative sessions. In: Proceedings of the 22nd International Conference on Engineering Design (ICED19), Delft 4. Singh H, Cascini G, Casakin H, Singh V (2019) A computational framework for exploring the socio-cognitive features of teams and their influence on design outcomes. In: Proceedings of the 22nd International Conference on Engineering Design (ICED19), Delft 5. Bonabeau E (2002) Agent-based modeling: methods and techniques for simulating human systems. Proc Natl Acad Sci USA 99:7280–7287 6. Abar S, Theodoropoulos GK, Lemarinier P, O’Hare GMP (2017) Agent based modelling and simulation tools: a review of the state-of-art software. Comput Sci Rev 24:13–33 7. Salas E, Guthrie JW, Wilson‐Donnelly KA, Priest HA, Burke CS (2005) Modeling team performance: the basic ingredients and research needs. In: Organizational Simulation. Wiley Hoboken, NJ, pp 185−228 8. McComb C, Cagan J, Kotovsky K (2017) Optimizing design teams based on problem properties computational team simulations and an applied empirical test. J Mech Des 139:041101-1–041101-12 9. McComb C, Cagan J, Kotovsky K (2015) Lifting the veil: drawing insights about design teams from a cognitively-inspired computational model. Des Stud 40:119–142 10. Gero JS, Kannengiesser U (2004) Modelling expertise of temporary design teams. J Des Res 4:1–13 11. Perišić MM, Štorga M, Gero JS (2018) Exploring the effect of experience on team behavior: a computational approach. In: Design Computing and Cognition DCC’18, Lecco, Italy

Modelling the Dynamics of Influence on Individual Thinking …

61

12. Hinds PJ, Carley KM, Krackhardt D, Wholey D (2000) Choosing work group members: balancing similarity, competence, and familiarity. Organ Behav Hum Decis Process 81:226– 251 13. Lapp S, Jablokow K, McComb C (2019) KABOOM: an agent based model for simulating cognitive styles in team problem solving. Des Sci 4:1–34 14. Singh V, Dong A, Gero JS (2011) How important is team structure to team performance? In: International Conference on Engineering Design, ICED11. Technical University of Denmark 15. Singh, V (2009) Computational studies on the role of social learning in the formation of team mental models. Doctor of Philosophy thesis. Design Lab Faculty of Architecture, Design and Planning, The University of Sydney, Sydney 16. Schreiber C, Singh S, Carley KM (2004) Construct – a multi-agent network model for the co-evolution of agents and socio-cultural environments. CASOS – Center for Computational Analysis of Social and Organizational Systems, Carnegie Mellon University, Pittsburgh, USA 17. McComb C (2016) Designing the Characteristics of Design Teams via Cognitively Inspired Computational Modeling. Carnegie Mellon University, Pittsburgh 18. Myers DG (1982) Polarizing Effects of Social Interaction. Group Decision Making. Academic Press, London 19. Akers RL, Krohn MD, Lanza-Kaduce L, Radosevich M (1979) Social learning and deviant behavior: a specific test of a general theory. Am Sociol Rev 44:636–655 20. Brown R, Pehrson S (2019) Innovation and changes in groups: minority Influence. Group Process: Dyn Between Groups 3:85–100 21. Pei S, Morone F, Makse HA (2018) Theories for influencer identification in complex networks. In: Complex Spreading Phenomena in Social Systems. Springer, pp. 125–148 22. Bandura A (1977) Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev 84:191–215 23. Granovetter MS (1973) The strength of weak ties. Am J Sociol 78:1360–1380 24. Ryan RM, Deci EL (2000) Intrinsic and extrinsic motivations: classic definitions and new directions. Contemp Educ Psychol 25:54–67 25. Costa AC (2003) Work team trust and effectiveness. Pers Rev 32:605–622 26. Mui L, Mojdeh M, Halberstadt A (2002) A computational model of trust and reputation. In: Proceedings of the 35th Hawaii International Conference on System Sciences, Hawaii. IEEE 27. Stempfle J, Badke-Schaub P (2002) Thinking in design teams - an analysis of team communication. Des Stud 23:473–496 28. Goucher-Lambert K, Moss J, Cagan J (2019) A neuroimaging investigation of design ideation with and without inspirational stimuli, understanding the meaning of near and far stimuli. Des Stud 60:1–38 29. Oberauer K, Lewandowsky S (2008) Forgetting in immediate serial recall: decay, temporal distinctiveness, or interference? Psychol Rev 115:544–576 30. Murdock BB (1962) The serial position effect of free recall. J Exp Psychol 64:482–488 31. Banaji RM (1986) Affect and memory: an experimental investigation. The Ohio State University 32. Gentner D (1989) Analogical Learning. Similarity and Analogical Reasoning. Cambridge University Press, Cambridge 33. Novice versus Expert Learning. How People Learn, September 2010. https://serc.carleton. edu/econ/quantitative_writing/noviceVexpert.html 34. Wimmer EG, Shohamy D (2017) Preference by association: how memory mechanisms in the hippocampus bias decisions. The American Association for the Advancement of Science, New York, pp 270–273 35. Singh H, Cascini G, Mccomb C (2020) Analyzing the effect of self-efficacy and influencers on design team performance. In: 16th International Design Conference DESIGN 2020, Dubrovnik, Croatia 36. Shah JJ, Smith SM, Vargas-Hernandez N (2003) Metrics for measuring ideation effectiveness. Des Stud 24:111–134

62

H. Singh et al.

37. Dorst K, Cross N (2001) Creativity in the design process: co-evolution of problem–solution. Des Stud 22:425–437 38. Leibowitz N, Baum B, Enden G, Karniel A (2010) The exponential learning equation as a function of successful trials results in sigmoid performance. J Math Psychol 54:338–340 39. Agars MD, Kaufman JC, Locke TR (2008) Social influence and creativity in organizations: a multi-level lens for theory, research, and practice. In: Multi-Level Issues in Creativity and Innovation (Research in Multi-Level Issues, vol 7. Emerald Group Publishing Limited, Bingley, pp 3-61 40. Tsenn J, Atilola O, McAdams DA, Linsey JS (2014) The effects of time and incubation on design concept generation. Des Stud 35:500–526 41. Paulus PB, Dzindolet MT (1993) Social influence processes in group brainstorming. J Pers Soc Psychol 64:575–586 42. Dugosh KL, Paulus PB (2005) Cognitive and social comparison processes in brainstorming. J Exp Soc Psychol 41:313–320 43. Paulus PB (2000) Groups, teams, and creativity: the creative potential of idea-generating groups. Appl Psychol: Int Rev 49:237–262

The Psychological Links Between Systems Thinking and Sequential Decision Making in Engineering Design John Z. Clay, Molla Hazifur Rahman, Darya L. Zabelina, Charles Xie, and Zhenghui Sha

Abstract Systems thinking is a cognitive style that deals with complex systems and is essential for systems engineering; elucidation of its underlying mechanisms allows for the development of techniques to aid in systems design. This paper sets out to test the relationships between validated psychological measures and systems thinking ability. To capture systems thinking ability and sequential design decisions, a computer-aided design task was developed. Participants designed an energy-plus house, utilizing solar energy to maximize the ratio of annual energy output to building cost. The present study offers and tests for two hypotheses. First, we expect to find a positive correlation between performance on the design problem and psychological measures of divergent thinking and cognitive ability. Second, a difference will be found in participant’s sequential design decisions according to their psychological profile. The first hypothesis was supported by a correlational analysis, while the second hypothesis was not.

1 Introduction 1.1

What is Systems Thinking?

The term, “systems thinking” was first introduced in 1987 by Barry Richmond, who saw it as a method of system comprehension and prediction [1]. Subsequent definitions see it as antithetical to reductionism [2] and linear thinking [3], both of J. Z. Clay
D. L. Zabelina Department of Psychological Science, The University of Arkansas, Fayetteville, AR, USA M. H. Rahman Department of Mechanical Engineering, The University of Arkansas, Fayetteville, AR, USA C. Xie Institute for Future Intelligence, Natick, MA, USA Z. Sha (&) Walker Department of Mechanical Engineering, The University of Texas at Austin, Austin, TX, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. S. Gero (ed.), Design Computing and Cognition'20, https://doi.org/10.1007/978-3-030-90625-2_4

63

64

J. Z. Clay et al.

which strive to solve problems within systems through simplification. Senge defines systems thinking as a framework for seeing wholes and the interrelationships within them rather than singular components, along with considering trends as opposed to static snapshots [4]. Upon the review of thirty-three references deemed important in the field of systems thinking, Monat and Gannon provide a broad definition: systems thinking is a perspective, a language, and a set of tools [3]. Many different perspectives on systems thinking from various disciplines can be found, and a widely accepted and accurate definition is hard to achieve. However, most definitions share two defining features: systems thinking is a specific cognitive style directed towards systems, and is supported by a set of cognitive skills that allow for one to both understand and solve problems within systems.

1.2

Why is Systems Thinking Important?

Systems thinking is particularly powerful in handling the ever-increasing complexity of large-scale engineered systems that are not solvable using reductionist thinking [5]. Therefore, a better understanding of the role that systems thinking plays in engineering systems design offers great benefits in both engineering education and engineering practice. During a recent NSF-sponsored Workshop on Artificial Intelligence and the Future of science, technology, engineering and mathematics (STEM) and Societies, the Vice President for Digital Transformation at Lockheed Martin Jeffrey Wilcox discussed the importance of systems thinking in the creation of complex systems and products, and noted the lack of formal training of systems thinking in professional engineers [6]. Additionally, an increasing amount of governmental mission agencies and manufacturing corporations are exploring opportunities for applying systems thinking and design thinking principles in systems engineering projects [7–10]. A report prepared by International Council on Systems Engineering (INCOSE) titled, “A World in Motion, Systems Engineering Vision 2025,” called for the role of systems thinking to be explicitly introduced early in education to complement learning in STEM [11]. The report suggested that educational infrastructure needs to be established to emphasize systems thinking and systems analysis at all phases of an engineer’s curriculum. The Council’s prediction is that the education of systems engineers through the exposure to systems thinking will allow for the high demand of systems engineers with technical and leadership competencies in the engineering and management workforce to be met.

1.3

Why is Systems Thinking Elusive?

Research on systems thinking is challenging, as its exact structure has proven hard to concretize and define; thus, there exists no consensus on the factors that comprise

The Psychological Links …

65

systems thinking. While Sage [12] summarizes the eleven laws of systems thinking, Valerdi [13] describes seven systems thinking competencies. Meanwhile, Ballé argues for three basic points of systems thinking: the detection of patterns as opposed to events, the use of circular causality (feedback loops), and a focus on relationships rather than single elements [14]. Alongside the disagreement on the structure of the concept, systems thinking often overlaps with other related terms. This is especially apparent in the relationship between engineering systems thinking and design thinking. Moti Frank, an influential researcher on the former topic, distinguished engineering systems thinking from systems thinking [15], adapting Senge’s systems thinking laws to create thirty engineering systems thinking laws. He later developed a capacity for engineering systems thinking (CEST) Cognitive Competency Model, and identified eighty-three competencies of successful systems engineers. These eighty-three competencies were aggregated into thirty-five competencies, including sixteen cognitive competencies, nine skills/abilities, seven behavioral competencies and three related to knowledge and experience [16]. In the present study we adopt the CEST Cognitive Competency Model, particularly the sixteen cognitive competencies that make up engineering systems thinking. While this model has been influential and offers an imperative base for future research on systems thinking, it was intended to serve as theoretical grounding; thus, how these competencies may be measured was not addressed. In a later work, Greene and Papalambros [17] mapped these sixteen competencies to established concepts within psychology, so that they may be studied by widely used and validated tests. In Table 1 we present Frank’s competencies and Greene and Papalambros’ mappings. In bold are the competencies and corresponding psychological constructs that are measured in the present study, the rationale for which can be found in the “Rationale” section. Systems thinking is also related to design thinking. Dym and colleagues [18] define design thinking as a complex process of inquiry and learning that designers perform in the context of a system, making decisions as they proceed and often done collaboratively. Vinnakota [19] argues that design thinking and systems thinking are connected and can be leveraged to overcome the problem of a complex system. Greene and colleagues [20] demarcate engineering systems thinking and design thinking, and describe them as two complementary approaches to understanding cognition, organization, and other non-technical factors that influence the design and performance of engineering systems. In the same paper [20], four concept models that depict plausible relationships between design thinking and systems thinking for engineering design are presented: The Distinctive Concept Model, Comparative Concept Model, Inclusive Concept Model, and Integrative Concept Model. We adopt the Comparative Concept Model, which suggests that the underlying mechanisms between engineering systems thinking and design thinking are similar, but that these concepts have different applications and utilize divergent methods.

66

J. Z. Clay et al.

Table 1 Cognitive Competencies and the corresponding psychological constructs [17] Frank’s Cognitive Competencies

Greene and Papalambros’ Mappings

Understand the whole system and see the big picture Understand interconnections

Sensemaking; information integration; mental model formation; generalization Induction; classification; similarity; information integration Deductive inference Perspective taking

Understand system synergy Understand the system from multiple perspectives Think creatively Understand system without getting stuck on the details Understand the implications of proposed change Understand a new system/concept immediately upon presentation Understand analogies and parallelism between systems Understand limits to growth Ask good (the right) questions Are innovators, originators, promoters, initiators, curious Are able to define boundaries Are able to take into consideration non-engineering factors Are able to “see” the future Are able to optimize

Creativity Abstraction; subsumption Hypothetical thinking Categorization; conceptual learning; inductive learning/inference Analogical thinking Information integration Critical thinking Inquisitive thinking Functional decomposition Conceptual combination Prospection Logical decision making

In the present study, we adopt Dym’s definition of design thinking, and study designers’ sequential decision making [21, 22], one of the most essential components in design thinking, as well as its relationship with systems thinking. Many factors in systems thinking, such as the capability of handling problem complexity [1] and uncertainty [4, 23] can influence designers’ sequential actions and the final design quality. Moreover, in a systems context, designers often receive incomplete information due to partial observability [24] and require long-term memory of past information [25] for better design iterations. To better understand and model the sequential decision making by considering individual differences, the systems thinking factors and the characteristics of systems context must be considered.

The Psychological Links …

67

2 Research Overview The objective of this paper is to uncover the interrelations between systems thinking and sequential decision making. Fine-grained data representing sequential design decisions and actions were captured through the administration of a computer-aided design problem. To complete this design problem, participants were asked to design an energy-plus home which, while utilizing solar energy, maximized the ratio of annual energy output (E) to building cost (C), i.e., r ¼ CE . How well participants accomplished this goal portrayed the quality of their design. The design actions as well as the iterations that participants made, along with their order, were logged automatically in a non-intrusive way, allowing for the analysis of how effective their sequential decision making was in solving the design. To measure systems thinking, the six competencies from Greene and Papalambros’ mappings of Frank’s CEST Cognitive Competencies Model that best represented how one would solve the issues faced in the design problem were chosen. Established and validated measures of these six competencies were then administered.

3 Research Hypothesis The present study offered and tested for two hypotheses: 1. We expected positive correlations between participant scores on measures of six cognitive competencies and their performance on the design problem. 2. We expected a significant difference between the groups in which participants were placed in based on their scores on the psychological tests in the usefulness of their sequential decision making.

3.1

Rationale for Hypothesis 1

The six competencies that we chose to measure in the present study are the ones listed in bold in Table 1. The first of Frank’s cognitive competencies that we expected to be positively correlated with performance on the design task is the most direct mapping to an existing psychological construct: “think creatively.” Creativity is a widely studied phenomenon in the field of psychology, and though a widely agreed upon definition has been difficult to reach, most definitions refer to creativity as the generation of ideas that are both novel and useful [26]. The field has received a great deal of attention since Guilford’s 1950 address to the American Psychological Association [27], and through his efforts creativity was given a theoretical foundation. An important distinction made by Guilford was that between divergent thinking and convergent thinking [28].

68

J. Z. Clay et al.

Divergent thinking refers to idea generation and is generally viewed as the essential component of creativity. Guilford’s Structure of Intellect [29] model offers the first in-depth consideration of the construct, where he explains that ideas are generated through thought that proceeds in disparate directions, thus allowing for novelty [30]. Idea generation is a critical step in the creative process, and is especially relevant in design; in fact, design of any original object would be rendered impossible without ideation. Convergent thinking, also researched as, “creative problem solving,” refers to the ability to find solutions to a given problem that has only one correct answer. Both are vital to creative cognition, and it was the intent of the researchers to gather data regarding both; however, technical difficulties barred the analysis of participants’ convergent thinking. To measure divergent thinking, the Abbreviated Torrance Test for Adults (ATTA) was used [31]; an in-depth explanation of and the rationale for the use of this test can be found in the, “Measures” section. The remaining five constructs that were chosen were inductive and deductive reasoning, analogical and critical thinking, and logical decision making. A great deal of research on these and pertaining constructs can be placed in the category of, “cognitive ability,” a broad term that has been used to reference ability in language, reasoning, memory, learning, cognitive speed, and many other cognitive traits [32], and has been shown to be highly positively correlated with popular standardized tests [33, 34]. To measure cognitive ability, we administered the International Cognitive Ability Resource (ICAR) test [35]; again, further explanation on this test and the rationale behind its use can be found in the, “Measures” section.

3.2

Rationale for Hypothesis 2

For our second hypothesis, we expected the statistical difference between participant groups in sequential decision making to be shown through the average change (d) participants made in the ratio of annual energy output (E) to building cost P (C) between their iterations, i.e., d ¼ Ni¼1 ðr t  r t1 Þ, where N is the total number of design iterations and r t represents the ratio CE at time t. Participants were divided into four groups based on their scores on the psychological measures in relation to the median scores for the sample. The groups were made for analysis purposes only; participants completed all aspects of the study individually. Group one contains participants who scored above the median score on both the ATTA and the ICAR; the second group is comprised of participants who scored high on the ATTA but low on the ICAR; group three are those who scored low on the ATTA and high on the ICAR, while the final group contains the participants who scored below the median on both measures. Table 2 offers a visualization of the groups.

The Psychological Links … Table 2 Groups and corresponding scores on psychological measures

69

Group Group Group Group

1 2 3 4

Divergent thinking score

Cognitive ability score

Above median Above median Below median Below median

Above median Below median Above median Below median

We expect that group two (high divergent thinking, low cognitive ability) will show a lower d than those in the group three (low divergent thinking, high cognitive ability). Design can be accomplished through many avenues, and the designer must use the cognitive competencies that are available to them. For instance, successful divergent thinkers may accomplish design through the generation of many different possible designs, testing each one individually; however, without high reasoning ability their ideas are not guaranteed to be beneficial to the task at hand. In comparison, those who show high cognitive ability may quickly understand the design task and what must be done to accomplish the goal, and largely skip the ideation phase.

4 The Empirical Study 4.1 4.1.1

Methods Participants

Thirteen people (nine females, four males) participated in the study1 (mean age = 30.76, SD = 13.16). Participants were recruited through both advertisements in an online university newsletter and with flyers distributed across campus. All but one of the participants indicated that they had, “a little” knowledge on the engineering design process, with the other having spent time studying the topic. Three of the participants were familiar with the challenges that solar science created, and the relevant solutions to those problems; one participant was unaccustomed to the topic, and the remaining nine had heard of solar science. The present study was approved for administration through the university’s Institutional Review Board, and all participants provided informed consent. We did not expect any of the demographic information to impact the results of the study and include them solely to give the reader a better understanding of the sample. 1

The number of participants is a major limitation of this study. However, we would like to highlight that the motivation of this paper is to share our views on the relations between engineering systems thinking and sequential decision thinking, and present the overall methodology of studying such relations from the psychological point of view. With the limitation of the number of subjects, we are cautious to draw conclusions until sufficient data are collected.

70

J. Z. Clay et al.

It was our assumption that neither gender nor age would influence design, and though the design problem was complex in nature, the premise was simple enough that previous knowledge regarding solar science would not offer an advantage.

4.1.2

Procedure

The experiment was divided into two phases. In the first phase, participants were given one hour to design a solarized home, an engineering system design problem using Energy3D—a computer-aided design (CAD) software for solar systems design which is capable of supporting design thinking studies [36]. Before this phase, participants filled out a questionnaire with the demographics and domain knowledge information. To ease the learning process of the Energy3D, participants were also subjected to a thirty minute tutorial session before they completed the design task. In this session, participants were given a tutorial sheet which provided a step by step introduction to the different tools needed to perform the design task. Data collected from the tutorial were not used for analysis, and participants were allowed to utilize the tutorial information in the actual design challenge. In the second phase participants were asked to complete the ATTA and the ICAR measures; this took approximately thirty minutes. At the end of the session, participants were provided with monetary rewards determined by the quality of their final design.

4.2 4.2.1

Measures Collecting Sequential Decision Making Data

The design problem was to build a solarized house in Dallas, for which we provide a detailed problem description including the objective, budget, requirements and constraints. The main objective was to maximize the annual net energy while minimizing the design cost. They were able to check their progress towards this goal by performing either an energy or financial analysis of their design at any time; this was the only feedback they were given regarding the cost and energy efficiency of their design. The program logged the cost and energy output of the design each time they performed an analysis, which we used as the iterations of their design. With a construction budget of $200,000, participants needed to meet several requirements for their designs; for example, the final design required at least four windows, and a wall height of at least 2.5 m. Table 3 summarizes all the requirements of the energy-plus home design problem. Participants were told that they would be compensated in accord to the degree to which they maximized their energy to budget ratio and stayed within the constraints. Though designers work to satisfy their own goals, this was done to ensure that all participants were motivated to work towards a similar goal. As the design of all components was predetermined by Energy3D, the participants worked with identical tools.

The Psychological Links … Table 3 The design problem components and their required metrics

71 Components

Requirements

Story Number of windows Size of windows Number of doors Size of doors (Width  Height) Height of wall Distance between ride to panel

1 >4 >1.44 m2 1  1.2 m  2 m >2.5 m >0

To complete the design problem, participants needed to consider the subsystems that made up the system as a whole; relevant subsystems ranged from, but were not limited to, the arrangement of the walls, the location of the door and windows, and the height of the roof. Participants were required to work within the given design constraints and were also forced to consider how the different variables related to each other, resulting in an intensive design problem that had to be solved through a systems thinking approach. One constraint not enforced was the design strategy that was implemented; while one participant may have moved from the wall subsystem to the roof subsystem, another could instead then begin working on the windows subsystem. This order that participants used to go about their design was driven by their sequential decision making. Both participant systems thinking ability and sequential decision making strategies were relied upon to complete the design problem, thus allowing the task to quantify and capture both.

Fig. 1 One of the energy-plus homes designed by a participant in the present study. This design achieved an annual net energy of 6640 kWh with a building cost of $207,289

72

J. Z. Clay et al.

Figure 1 shows an example of a solarized energy-plus home that participants built through Energy3D, a computer-aided design program. Energy3D has great utility for conducting design research, and allows for the analysis of engineered systems, scientific simulation, and financial evaluation. The program has built-in tutorials and design examples to help novice designers to learn the software quickly, and offers interactive visualization and simulation tools to allow designers to perform analysis in real time. Additionally, Energy3D has the ability to log all performed actions at fine-grained scale in JSON files, capturing both design actions and the details associated with each of these actions; for example, when a user utilizes a design action to change the efficiency of a solar cell, the new efficiency value for the cell will also be recorded. The following box shows a sample of the design action data that was collected.

4.2.2

Measuring Cognitive Competencies

Abbreviated Torrance Test for Adults In order to measure participant divergent thinking, the Abbreviated Torrance Test for Adults (ATTA) was administered [31]. This test has roots in the Torrance Test of Creative Thinking (TTCT), first developed by Paul Torrance in the 1960’s [37] and then used extensively throughout his long and influential career. Torrance provided ample evidence for the TTCT’s validity in measuring creative ability, most famously through a longitudinal study showing a strong positive correlation between high-schoolers scores on the test and their later creative achievements [38]. For many years, the TTCT was the prevailing paradigm for measuring divergent thinking [39, 40]. However, to complete the TTCT takes over an hour, and those scoring it require approximately twenty minutes [27]; thus, the ATTA was later developed by Torrance and Goff as a shortened version that can be completed in under ten minutes, allowing for quick administration and scoring. The ATTA itself has been shown to possess both positive correlations with and predictive reliability for real life creativity [27, 41]. This measure of divergent thinking is widely used and trusted throughout psychology, and thus was chosen for the present study. The test consists of three activities: one measuring verbal, and two measuring figural divergent thinking. For each activity, participants are timed for three minutes, and are encouraged to, “be creative,” a primer that has shown to effect how creative answers can be [42]. In verbal activity, participants were asked to list the problems that would come with the ability to walk on air or fly without being in a vehicle. In the figural activities, they are presented with incomplete geometric figures and are asked to use these figures to complete drawings.

The Psychological Links …

73

Participant responses were measured across four constructs: fluency (the number of generated items per activity), originality (how original responses were when compared to the standardized norms), cognitive flexibility (the number of distinct domains that were referenced throughout the responses), and elaboration (the amount of detail given). To obtain an overall divergent thinking score, answers for each construct were z-scored, after which they were averaged together; method similar to that in [43]. International Cognitive Ability Resource Condon and Revelle’s [35] International Cognitive Ability Resource (ICAR) test was utilized to capture cognitive ability, a broad term used within psychology to reference reasoning ability that the present study adopts to reference the several different types of reasoning that Frank [16] cites in his model. Though the term lacks a precise definition, it has been used both interchangeably with and alongside intelligence [33, 43]; previous studies have measured the construct through scores in school and on standardized tests [44], along with other measures of intelligence [45]. The ICAR was developed to establish a reliable and validated public domain measure of cognitive ability, that was not only free and easy to obtain, but also quick to administer and score when compared to other measures of the same construct. Because of these reasons, the ICAR was chosen to capture the mappings from Greene and Papalambros’ mappings [17] of Frank’s model [16] that explicitly reference reasoning ability. The test is comprised of four item types: Letter and Number Series, Matrix Reasoning, Verbal Reasoning, and Three-dimensional Rotation; Table 4 offers a visualization of the types of reasoning that each of the items measure. The first, Letter and Number Series, tasks participants to predict the next item in a string of number or letter sequences (ex. “In the following alphanumeric series, what letter comes next? I J L O S”). Matrix Reasoning questions present a 3  3 display of shapes and ask participants to pick from a pool of 6 additional shapes the one that best completes the array; see Fig. 2 for a sample question. Verbal Reasoning items challenge participants with general logic questions (ex. “If the day after tomorrow is two days before Thursday, then what day is it today?”). Lastly, Three-dimensional Rotation tasks ask participants to correctly choose one of six cubes that is a rotation of an initially presented cube; see Fig. 3 for an example of this item type. When scoring, the number of total correct responses is taken as an indication of general cognitive ability.

Table 4 ICAR item types and the corresponding Cognitive Competencies mappings Item type

Cognitive Competencies Mappings

Letter and Number Series Matrix Reasoning

Induction, analogical thinking, critical thinking, logical decision making Induction, analogical thinking, critical thinking, logical decision making Induction, deductive inference, critical thinking, logical decision making Analogical thinking, critical thinking

Verbal Reasoning Three-dimensional Rotation

74

J. Z. Clay et al.

Fig. 2 A Matrix Reasoning item from the ICAR; participants must choose the correct option from the bottom row to complete the pattern shown in the 3  3 display

Fig. 3 A Three-dimensional Rotation item from the ICAR; participants are given the instruction to, “Select the choice that represents a rotation of the cube labeled X.”

5 Results Performance on the design challenge varied among participants. The average ratio of annual energy output to building cost was 0.083 and ranged from a minimum of 0.016 to a maximum of 0.121. All but two of the participants submitted a design under the $200,000 budget, spending an average of $191,832 per design. The highest annual net

The Psychological Links …

75

energy output achieved was 24,162.66 kWh; the lowest only yielded 5,684.89 kWh, with the average design showing an output of 15,103.57 kWh. For our first hypothesis, we expected to see significant positive correlations between two psychological measures, the ATTA and the ICAR, and the participant’s ratio of annual energy output to building cost in their design. Positive correlations, one of which reached significance, emerged between design performance and the ATTA; however, an insignificant negative correlation was found between design performance and the ICAR. The overall divergent thinking score was positively correlated with design performance, and showed marginal significance (r = 0.514, p = 0.087). The subcomponents also displayed positive correlations: originality was significantly correlated with the design metric (r = 0.592, p = 0.0442), and while fluency (r = 0.332, p = 0.291), flexibility (r = 0.486, p = 0.109), and elaboration (r = 0.261, p = 0.412) all failed to reach significance, they each showed moderate correlations with performance on the design task. There was no significant positive correlation between scores on the ICAR and design performance; instead, an insignificant small negative correlation was found (r = -0.211, p = 0.557). The second hypothesis posited that there would be a significant difference in the usefulness of sequential decision making between the participant groups that were created based on their scores on the ATTA and the ICAR. We were particularly interested in the relationship between the second (ATTA score above median, ICAR score below median) and third groups (ATTA score below median, ICAR score above median). Neither prediction was supported. A one-way between subjects ANOVA was conducted to measure the difference in d between all groups, and no significant difference was found (F(3,6) = 0.515, p = 0.686).

6 Discussion The research objective of the present study was to explore the relationships between psychological measures used to represent systems thinking and sequential decision making within the engineering systems design context. To measure systems thinking, six of Greene and Papalambros’ mappings [17] of Frank’s sixteen cognitive competencies from his CEST model [16] were chosen, based on their relevance to the demands of the design problem. The six chosen competencies can be seen in Table 1. To measure the first competency, the Abbreviated Torrance Test for Adults was administered; for the remaining five, participants were asked to complete the International Cognitive Ability Resource test. In order to capture sequential decision making, participants were asked to complete an energy-plus home design challenge through the computer-aided design program Energy3D. The challenge was to design a home that, through the utilization of solar energy, resulted in the highest ratio of annual energy output to

76

J. Z. Clay et al.

building cost that participants could achieve; their performance was used to interpret their systems thinking ability and sequential decision making. The present study had two hypotheses. First, we expected to find positive linear relationships between systems thinking and design thinking; specifically, between the measures of divergent thinking and cognitive ability in comparison to performance on the design challenge. Second, we predicted a significant difference in d between the participant groups. Total divergent thinking and each of the subcomponents showed positive correlations with design performance; only the relationship with participant originality showed significance. Divergent thinking is an essential component of the creative process; without the generation of testable ideas, design would be rendered near impossible. Cognitive ability displayed a small negative correlation with the performance metric; however, the researchers stress that the high insignificance of the correlation (p = 0.557) must be considered when interpreting this relationship. The results do not suggest that cognitive ability is detrimental to engineering design, but rather that the ICAR likely does not measure any pertinent psychological constructs. We found no support towards our second hypothesis. There was no significant difference in d between participant groups, which has several implications. First, this suggests that there was no benefit to performance in the design challenge through the possession of both high divergent thinking and high cognitive ability. Additionally, these results imply that there is no benefit in showing high ability in only one of these traits, regardless of which the participant was skilled in.

7 Limitations The chief limitations of the presents study reside in the sample that was used. It must first be addressed that our participants were undergraduates, not professional engineers. Thus, the findings are not directly applicable to and do not represent experts and those already in the workforce; it is possible that divergent thinking and cognitive ability play different roles in design when comparing undergraduates and professionals. Second, the small sample size must be noted. The researchers stress that the results should be taken tentatively, and that any conclusions drawn must be considered in tandem with this limitation. However, as the purpose of the present study was to set a groundwork for future research on this and related topics, we feel it is necessary to document our theoretical and methodological approaches to studying systems thinking and sequential decision making.

The Psychological Links …

77

8 Conclusions The present study set out to build a foundation for the empirical analysis of systems thinking through a psychometric approach, and offered tentative results suggesting which aspects of cognition play a role in engineering design. Results showed that divergent thinking is closely positively related to performance on the design task, with the originality subconstruct showing significance. Our results also indicate that either cognitive ability played no role in our design task, or that the test used to measure cognitive ability failed to capture any competencies relevant to the design challenge; as the ICAR was employed to measure multiple cognitive competencies, it is difficult to determine how each of the five competencies factor into this relationship. Lastly, analysis did not find a significant difference in sequential decision making based on high ability in either divergent thinking or cognitive ability.

9 Future Directions The present study only looked at the relationship between engineering design and six of the sixteen cognitive competencies given in Frank’s model [16]. These six were chosen due to the availability of and convenience of psychological measures for the constructs, and the exploratory nature of the present study; at no point did we believe that these were the only competencies relevant to design. In the future, additional psychological tools measuring different cognitive competencies must be leveraged in order to establish a psychometric approach to systems thinking research. Additionally, future research should address the limitations that the present study faced. To obtain more sound results, larger samples must be utilized both on undergraduate and professional samples. Acknowledgements The authors gratefully acknowledge the financial support from the U.S. National Science Foundation (NSF) via grants #1842588 and #1503196. Any opinions, findings, and conclusions or recommendations expressed in this publication, however, are those of the authors and do not necessarily reflect the views of NSF.

References 1. Richmond B (1994) Systems thinking/system dynamics: let’s just get on with it. Syst Dyn Rev 10(2–3):135–157 2. O’Connor J, Dermot IM (1997) The art of systems thinking: essential skills for creativity and problem solving. Thorsons, London 3. Monat JP, Gannon TF (2015) What is systems thinking? A review of selected literature plus recommendations. Am J Syst Sci 4(1):11–26

78

J. Z. Clay et al.

4. Senge PM (1990) The fifth discipline: the art and practice of the learning organization. Doubleday/Currency, New York 5. Tomko M, Nelson J, Nagel RL, Bohm M, Linsey J (2017) A bridge to systems thinking in engineering design: an examination of students’ ability to identify functions at varying levels of abstraction. Artif Intell Eng Des Anal Manuf 31(4):535–549 6. Wilcox J (2019) Workshop on artificial intelligence and the future of STEM and societies. Keynote Speaker 7. Souza J, Barnhöfer U (2015) Design thinking: it’s the flare that adds another dimension to systems engineering. Insight 18(3):25–32 8. Darrin MAG, Devereux WS (2017) The Agile Manifesto, design thinking and systems engineering. In: Annual IEEE International Systems Conference (SysCon) 9. McGowan A-MR, Daly S, Baker W, Papalambros P, Seifert C (2013) A socio-technical perspective on interdisciplinary interactions during the development of complex engineered systems. Procedia Comput Sci 16:1142–1151 10. McGowan A-MR, Bakula C, Castner RS (2017) Lessons learned from applying design thinking in a NASA rapid design study in aeronautics. In: 58th AIAA/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference 11. International Council on Systems Engineering (2014) A world in motion: systems engineering vision 2025 12. Sage AP (1995) Systems management for information technology and software engineering. Wiley, New York 13. Valerdi R, Rouse WB (2010) Why systems thinking is not a natural act. In: 2010 IEEE International Systems Conference 14. Ballé M (1994) Managing with Systems Thinking: Making Dynamics Work for You in Business Decision Making. McGraw-Hill, London 15. Frank M (2000) Engineering systems thinking and systems thinking. Syst Eng 3(3):163–168 16. Frank M (2010) Assessing the interest for systems engineering positions and other engineering positions required capacity for engineering systems thinking (CEST). Syst Eng 13(2):161–174 17. Greene MT, Papalambros PY (2016) A cognitive framework for engineering systems thinking. In: 2016 Conference on Systems Engineering Research 18. Dym CL, Agogino AM, Eris O, Frey DD, Leifer LJ (2005) Engineering design thinking, teaching, and learning. J Eng Educ 94(1):103–120 19. Vinnakota T (2016) A conceptual framework for complex system design and design management. In: Annual IEEE Systems Conference (SysCon) 20. Greene MT, Gonzalez R, Papalambros PY, McGowan A-MR (2017) Design thinking vs. systems thinking for engineering design: what’s the difference? In: International Conference on Engineering Design, Vancouver, Canada, 21–25 August 2017 21. Rahman M, Xie C, Sha Z (2019) A deep learning based approach to predicting sequential design decisions. In: ASME 2019 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference, Anaheim, CA, 18–21 August 2019 22. Rahman M, Gashler M, Xie C, Sha Z (2018) Automatic clustering of sequential design behaviors. In: ASME 2018 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference, Quebec City, Canada, 26–19 August 2018 23. Meadows DH, Wright D (2015) Thinking in systems: a primer. Chelsea Green Publishing, White River Junction 24. Sweeney LB, Sterman JD (2000) Bathtub dynamics: initial results of a systems thinking inventory. Syst Dyn Rev 16(4):249–286 25. Kim DH (1999) Introduction to systems thinking. Pegasus Communications Inc, Arcadia 26. Runco MA, Jaeger GJ (2012) The standard definition of creativity. Creat Res J 24(1):92–96 27. Althuizen N (2010) The validity of two brief measures of creative ability. Creat Res J 22 (1):53–61 28. Guilford JP (1956) The structure of intellect. Psychol Bull 53(4):267–293

The Psychological Links …

79

29. Guilford JP (1968) Intelligence, creativity and their educational implications. Knapp, San Diego 30. Runco MA (2010) Divergent thinking, creativity, and ideation. In: Kaufman JC, Sternberg RJ (eds) The Cambridge handbook of creativity. Cambridge University Press, Cambridge, pp 413–446 31. Goff K, Torrance EP (2002) Abbreviated torrance test for adults manual. Scholastic Testing Service Inc., Bensenville 32. Carroll JB (2004) Human cognitive abilities: a survey of factor-analytic studies. Cambridge University Press, Cambridge 33. Koenig KA, Frey MC, Detterman DK (2008) ACT and general cognitive ability. Intelligence 36(2):153–160 34. Jensen A (1998) The g factor: the science of mental ability. Prager, Santa Barbara 35. Condon DM, Revelle W (2014) The international cognitive ability resource: development and initial validation of a public-domain measure. Intelligence 43:52–64 36. Rahman M, Schimpf C, Xie C, Sha Z (2019) A computer aided design based research platform for design thinking studies. J Mech Des Trans ASME 141(12):1–12 37. Torrance EP (1966) Torrance Tests of Creative Thinking: norms technical manual. Personnel Press, Princeton 38. Torrance EP (1972) Predictive validity of the Torrance Tests of Creative Thinking. J Creat Behav 6(4):236–252 39. Davis GA (1997) Identifying creative students and measuring creativity. In: Colangelo N, Davis GA (eds) Handbook of gifted education. Viacom, Needham Heights, pp 269–281 40. Plucker JA, Renzulli JS (1999) Psychometric approaches to the study of human creativity. In: Sternberg RJ (ed) Handbook of creativity. Cambridge University Press, Cambridge, pp 35–61 41. Shen T, Lai JC (2014) Exploring the relationship between creative test of ATTA and the thinking of creative works. Procedia Soc Behav Sci 112:557–566 42. Nusbaum EC, Silvia PJ, Beaty RE (2014) Ready, set, create: what instructing people to “be creative” reveals about the meaning and mechanisms of diver-gent thinking. Psychol Aesthet Creat Arts 8(4):423–432 43. Stanovich KE, West RF (2008) On the relative independence of thinking biases and cognitive ability. J Pers Soc Psychol 94(4):672–695 44. Benjamin DJ, Brown SA, Shapiro JM (2013) Who is ‘behavioral’? Cognitive ability and anomalous preferences. J Eur Econ Assoc 11(6):1231–1255 45. Gevins A, Smith ME (2000) Neurophysiological measures of working memory and individual differences in cognitive ability and cognitive style. Cereb Cortex 10(9):829–839

Patterns of Silence and Communication Between Paired Designers in Collaborative Computer-Aided Design Meaghan Vella, Alison Olechowski, and Vrushank Phadnis

Abstract We present the initial findings of an experiment where designers worked in pairs to complete a series of Computer-Aided Design (CAD) tasks via Parallel CAD Control, a working style where both partners are able to simultaneously manipulate, modify and build-on the same part model from their own workstations. We find that the level of communication varies widely amongst the pairs, but equality of communication across the pairs remains relatively even. We unexpectedly find that pairs that include a Novice designer have a relatively lower level of communication, despite maintaining equality of conversation. Finally, we find that participants with high CAD skill use technical CAD words at a relatively lower frequency. These initial experimental findings, analyzed via a limited sample size, motivate research questions for further study, with the potential to eventually inform best practice in collaborative CAD.

1 Introduction The generation of digital models via computer-aided design (CAD) is a core step in the modern design process. Driven by trends towards flexible workplaces and geographically distributed teams [1], CAD software has evolved to better enable collaboration [2]. While there exists a foundational knowledge of the best practices in virtual collaboration [3, 4], in engineering and product development contexts [5, 6], and in other technical partnerships like pair programming [7], there is a lack of understanding and corresponding guidance regarding designer behavior and outcomes using collaborative CAD [8]. Studies of collaborative CAD have focused on: comparing collaborative CAD to traditional CAD [8, 9]; determining

M. Vella
A. Olechowski (&) University of Toronto, Toronto, Canada e-mail: [email protected] V. Phadnis Massachusetts Institute of Technology, Cambridge, USA © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. S. Gero (ed.), Design Computing and Cognition'20, https://doi.org/10.1007/978-3-030-90625-2_5

81

82

M. Vella et al.

appropriate team size via part analysis [10]; and, analysis of designer emotion while designing collaboratively [11]. Given that communication is a core element of collaboration and design [12, 13], and design representation is particularly relevant in collaborative design [14], this paper focuses specifically on analysis of the communication between pairs of designers as they work collaboratively on a CAD task. We aim to identify compelling research questions regarding communication in paired collaborative CAD via evidence from a limited experiment of paired CAD designers.

1.1

Communication in Design

Communication is thought to reflect the thinking and problem-solving process of design teams [15], and thus a number of techniques have been adopted to analyze the conversation of design teams. Natural language processing (NLP) takes advantage of large data sets to systematically understand human language. One NLP technique used to determine topic of conversation is latent semantic analysis [16, 17], while other studies simply look at frequency of words as an indicator of topic [18–20]. While papers that use NLP topics often analyze large data sets of emails or other design documentation from real industry design teams, few have analyzed experimentally derived communication data. Experiments allow the controlled analysis of particular phenomenon and variables of interest, but resulting analysis can be challenging because data generated is often smaller in scale. Lack of communication - sometimes discussed as silence, or pause - is of interest in the design process and has been conceptualized in a number of ways in past studies. Pauses have been examined in co-located physical design work, conceived as processing time, spent in response to specific stimuli [21]. This study found that individuals tended to pause more later on in the modeling task, as that is also when participants spend more time processing information. Thus patterns of communication over time will be examined in the present study. In a study of collaborative, remote, early-stage design, silences are considered “moving,” where they are adding to the representation or gesturing, ultimately transforming the design situation [22]. Another relevant aspect of communication in the design process is the presence of questions. Question asking is thought to influence problem framing and idea shaping [23], and therefore may be indicative of good process when design is viewed as problem solving. Building on these works in a new collaborative design context, this study presents preliminary data towards understanding the verbal communication and silence between partners of varying skill-levels as they collaboratively tackle a series of CAD tasks. We look at trends in level and equality of communication, as well as CAD-specific topics of communication.

Patterns of Silence and Communication Between Paired Designers …

83

2 Methods 2.1

Experiment Overview

Participants with at least one year of 3D CAD experience were recruited via email and posters around campus. Of the 20 participants, 12 were men and 8 were women. Participants were randomly assigned to pairs for the experiment. No participants had a previously existing relationship. While the full experiment featured two different working configurations for pairs, this paper focuses only on one, which we have termed “Parallel CAD Control”. Parallel CAD Control is a collaborative workflow where both partners are able to simultaneously manipulate, modify and build-on the same model from their own workstations. It is analogous to “Google Docs” style collaborative editing, for CAD. We achieve this synchronous collaborative feature using Onshape [24], a commercially available CAD software which enables part-level synchronous collaborative editing. During the course of the experiment, test participants were led through four phases. A description of the phases can be found in Fig. 1. For the scope of this paper, Phase 2 and 4 will be examined. Phase 2 of the experiment is where participants worked individually. This phase is used to benchmark the productivity of the participants (used as a skill-level metric for analysis), and to see if any participants did not meet the required skill level to participate in the remainder of the experiment. Individuals were asked to leave after Phase 2 if they did not reach a predetermined level of progress. Each step of the experimental task is a set of modeling instructions for the participant to follow. Participants can move on to the

Fig. 1 Overview of experiment. For the duration of the experiment, white noise played in the room and physical barriers were used to mimic participants working in geographically different places. Users could communicate with each other through headsets connected to their computers

84

M. Vella et al.

next step once they have completed their current step. The participants are instructed to focus on both timeliness for finishing their designs and on the quality of the model itself. Phase 4 is the core of the experiment, where users are tasked with a number of requests to add features to an existing 3D model of a phone holder. The participants were told that they would be assessed on both the number of steps they could complete in the allotted time, as well as the quality of their CAD model.

2.2

Analysis: Audio and Silence Detection

Because of white noise and varying speaker volumes in the recordings, files were edited to achieve dynamic range compression. Audacity [25] was used to complete the editing of all of the audio files. The files were manually listened to, to ensure that the audio quality was not lost in the process. Next, the audio files were analyzed for “Silence,” i.e. any period of time when neither individual is talking. Silence was calculated based on the amplitude of the audio files using a Python script and Pydub. The script iterated through the entire audio file, and if the dbfs (decibels relative to full scale) level was below −45 dbfs, the time step was labeled as silence. The threshold of −45 dbfs was determined through trial and error as appropriate.

2.3

Analysis: Verbal Communication

The audio recordings of the conversations were sent to the transcription service Rev. The transcription was returned with each speaker labeled. The transcriptions were then manually reviewed by the investigator to fix any inaccuracies. The text files were parsed, separating the two speakers. For each speaker’s transcription, data was collected including the total number of words, the share of words per individual in each pair, the total number of questions asked and the proportion of questions to words spoken (see Table 1). The word counts from the conversation transcripts were used to determine the amount of talking per person. The data was cleaned to remove punctuation, numbers and nonsensical text, was converted to lowercase and lemmatized. Python package scikit-learn was used to create a document-term matrix for all of the speakers. The most frequent words spoken by individuals were then further examined.

Patterns of Silence and Communication Between Paired Designers …

85

Table 1 Overview of communication variables of interest

Name

Level

Calculation

Proportion of Speaking to Pair Total Experiment Time Partner Share of Total Individual Speaking Partner Share of Total Individual Questions

2.4

Analysis: Determining Participants’ Skill Levels

The participant skill level was classified via the step reached in Phase 2 of the experiment (see in Fig. 1). The individual participants were grouped into three types of users: Standard, Novice and Expert, shown in Table 2. The average step completed was 4.75, with the highest step being 6, and the lowest step being 2. An overview of the pairing combinations present is shown in Table 2; it can be noted that not all possible pairing combinations were present in the experiment. The pairings were randomly assigned, before skill-level was known. This experimentally-derived level of skill reflects an overall productivity, likely a combination of CAD expertise, and expertise with the specific Onshape user interface.

Table 2 Overview of user skill levels and pairing combinations

User Phase 2 NumType Step ber Novice 5 4

Pairing Combination Standard-Standard Standard-Expert Standard-Novice Expert-Novice

Shortform

Count SS SE SN EN

3 3 3 1

86

M. Vella et al.

3 Results and Discussion 3.1

Patterns of Silence over Time in Parallel CAD Control

On average, participants communicated for 41% of the total experiment time. Trends in communication level over the course of the experiment are visualized in Fig. 2, which divides the experiment into thirds to look for trends. We might expect levels of communication to decrease over time, as has been seen in previous studies [21]. When examining individual teams in Fig. 2, individual pairs’ communication levels vary, and there does seem to be a small pattern of communication increasing in the second third and decreasing in the last third. Next, communication was analyzed by the level of participant CAD skill. As can be seen in Fig. 3a, pairs including Novice participants tend to exhibit less communication, as the average percent of communication for a pair is lower than for Standard and Expert participants. Novices seem to be in groups that communicate on average 10% less than their Standard and Expert counterparts. This could indicate that there is a behavior of Novice participants that tend to bring the total communication of the group down. Perhaps Novice participants must expend more concentration on the task at hand, or perhaps they feel hesitant to communicate to a more experienced partner. We can further investigate this conjecture with an analysis of questions. To investigate whether this trend simply reflects the low contribution of the Novice designers themselves, we next look at partner share of total speaking. As shown in Fig. 3b, the proportion of words spoken by each participant remains relatively equal when examined by skill level. This could reflect a natural pattern of conversational turn-taking in partnered communication where individuals tend to speak at around equal proportions. Further, this equality of conversation

Fig. 2 Amount of speaking in each third of the experiment for 10 pairs. The dashed line is the average proportion of speaking for all pairs

Patterns of Silence and Communication Between Paired Designers …

87

Fig. 3 a) Percent of time spent in communication per skill level group. b) Equality of communication, measured as share of total speaking

could reflect the nature of the conversations happening during the experiment. Most team members were asking questions and checking in on their partners, which are reciprocal interactions. With this type of discussion it would be expected for pairs to have a proportional number of words spoken to their partner. Overall, teams with a Novice member have on average less communication, yet the individual Novices did not speak proportionally less relative to their partner than average. Further research is required to better understand this phenomenon.

3.2

Analysis of Questions

On average, participants asked their partner 27 questions during the course of the 45 min experiment. As can be observed in Fig. 4, many but not all participants ask roughly the same proportion of questions as they contribute to talking time in their pair. There are, however, interesting exceptions to this pattern, indicated by dots far from the diagonal in Fig. 4. Some participants in the upper left ask fewer questions that one would expect, and those in the lower right ask more questions that we might expect based on their contribution to conversation. When considering the element of skill level for the proportion of questions asked, on every level the proportion of questions asked on average is consistent at around 50%. One interesting note is that it appears as if Novices and Experts ask proportionally more questions than the Standard participants. One possible reason for this could be Novices asking more questions because they need more clarification, or Experts asking more questions because they are checking in on their partners more.

88

M. Vella et al.

Fig. 4 Contribution to conversations and questions asked

3.3

Words Spoken by Participants

Next we examine the frequency of word use. A list of most popular CAD-specific words (based on the authors’ domain knowledge) is shown in Table 3. As seen in Fig. 5, Novice and Standard participants use slightly more CAD-related words than Experts. The average percent of CAD words to non-CAD words is 2% for Novice and Standard participants, while for Expert participants it is only 0.6%. The highest number of CAD words (60) was said by an Expert. There was also one Expert that said 0 CAD specific words throughout the experiment. It should be noted that the overall sample size, and sample size of Experts (n = 4) is small, and so additional experiments are needed to draw further conclusions about skill level and technical communication in CAD.

Table 3 Frequency of CAD-specific words (10 most frequent)

CAD-Specific Word sketch mirror pattern dimensions feature

Frequency of Use 92 37 32 31 31

CADSpecific Word plane extrude linear fillets rotate

Frequency of Use 30 27 22 17 16

Patterns of Silence and Communication Between Paired Designers …

89

Fig. 5 Percentage of CAD-related words spoken by individuals

4 Conclusion This study presents an initial investigation of variables of interest to communication in paired collaborative CAD. Communication levels fluctuate, where teams on average increased communication midway through the experiment and dropped at the end of the experiment. This could indicate that participants are pausing more to process information in the earlier and later stages of the design task, or could represent a pattern of “collaborative moving”. Communication tends to lessen in the presence of Novice designers, an unexpected result which is not explained by inequality of contribution in pairs with Novices, or by an over- or under-abundance of questions. In fact, equality of communication between partners is on average quite consistent, at equal, regardless of other variables. Finally, we unexpectedly found that experts discussed technical topics at a lesser rate than Novices. We expect further insight to be generated from a comparison of the patterns revealed here and the experimental outcomes. We will further explore more advanced natural language processing techniques. Further, the sample for our experiment is relatively small, and thus we seek further replication in order to statistically verify trends observed thus far. A study investigating the effect of team experience [26] over a more significant time would further reveal important trends. Patterns of communication are known to be culturally dependent [27], and thus further analysis via a cultural lens is an important future step. Important insight may be generated from teasing out the difference between communication focused on the CAD tool versus the design itself, as has previously been studied in parametric design environments [28]. We envision a future where extensions of this study reveal best practice in paired communication, which may lead to new features or capabilities of CAD tools, such as communication nudges or real-time display of communication contribution.

90

M. Vella et al.

Acknowledgements Thank you Ready Lab, especially Hamza Arshad and Xiangjiu Wu, for your help with the experiments and data. Thank you to four anonymous reviewers for their thoughtful feedback on the paper draft.

References 1. National Academies of Sciences, Engineering, and Medicine (2017) Information Technology and the U.S. Workforce: Where Are We and Where Do We Go from Here? National Academies Press 2. Wu D, Rosen DW, Wang L, Schaefer D (2015) Cloud-based design and manufacturing: a new paradigm in digital manufacturing and design innovation. Comput Aided Des 59:1–14 3. Martins LL, Gilson LL, Travis Maynard M (2004) Virtual teams: what do we know and where do we go from here? J Manag 30(6):805–835 4. Schaubroeck JM, Yu A (2017) When does virtuality help or hinder teams? Core team characteristics as contingency factors. Hum Resour Manag Rev 27(4):635–647 5. Loch CH, Terwiesch C (1998) Communication and uncertainty in concurrent engineering. Manage Sci 44(8):1032–1048 6. Montoya MM, Massey AP, Hung Y-TC, Brad Crisp C (2009) Can you hear me now? Communication in virtual product development teams. J Prod Innov Manag 26(2):139–155 7. Balijepally B, Mahapatra R, Nerur S, Price KH (2009) Are two heads better than one for software development? Productivity Paradox of Pair Programming. MIS Q 33(1):91 8. Phadnis VS, Leonardo KA, Wallace DR, Olechowski AL (2019) An exploratory study comparing CAD tools and working styles for implementing design changes. Proc Des Soc Int Conf Eng Des 1(1):1383–1392 9. Eves K, Salmon J, Olsen J, Fagergren F (2018) A comparative analysis of computer-aided design team performance with collaboration software. Comput Aided Des Appl 15(4):476– 487 10. Stone B, Salmon J, Hepworth A, Red E, Killian M (2016) Methods for determining the optimal number of simultaneous contributors for multi-user CAD parts. In: Proceedings of CAD 2016 11. Zhou J, Phadnis V, Olechowski A (2019) Analysis of designer emotions in collaborative and traditional computer-aided design. In: ASME 2019 IDETC and CIE Conference. https://doi. org/10.1115/detc2019-98516 12. Jiang H, Gero JS (2017) Comparing two approaches to studying communications in team design. Des Computing Cogn. 16:301–319. 13. Ostergaard KJ, Summers JD (2009) Development of a systematic classification and taxonomy of collaborative design activities. J Eng Des 20(1):57–81 14. O’Hare J, Dekoninck E, Mombeshora M, Martens P, Becattini N, Boujut J-F (2018) Defining requirements for an augmented reality system to overcome the challenges of creating and using design representations in co-design sessions. CoDesign:1–24 15. Stempfle J, Badke-Schaub P (2002) Thinking in design teams - an analysis of team communication. Des Stud 23(5):473–496 16. Dong A (2005) The latent semantic approach to studying design team communication. Des Stud 26(5):445–461 17. Georgiev GV, Georgiev DD (2019) Semantic analysis approach to studying design problem solving. Proc Des Soc Int Conf Eng Des 1(1):1823–1832 18. Snider C, Škec S, Gopsill JA, Hicks BJ (2017) The characterisation of engineering activity through email communication and content dynamics, for support of engineering project management. Des Sci 3

Patterns of Silence and Communication Between Paired Designers …

91

19. Piccolo SA, Wilberg J, Lindemann U, Maier A (2018) Changes and sentiment: a longitudinal email analysis of a large design project. In: Proceedings of the DESIGN 2018 15th international design conference 20. Roschuni C, Oehlberg L, Beckman S, Agogino AM (2009) Relationship conflict and feeling communication in design teams. In: 14th design for manufacturing and the life cycle conference; 6th symposium on international design and design education; 21st international conference on design theory and methodology, vol 8 21. Patel A, Kramer WS, Flynn M, Summers JD, Shuffler ML (2018) Function modeling: an analysis of pause patterns in modeling activities. In: 30th international conference on design theory and methodology, vol. 7 22. Dorta T, Kalay Y, Lesage A, Pérez E (2011) Design conversations in the interconnected HIS. Int J Des Sci Technol 18(2):65–80 23. Cardoso C, Badke-Schaub P, Eris O (2016) Inflection moments in design discourse: how questions drive problem framing during idea generation. Des Stud 46:59–78 24. Onshape Inc. (2020) Onshape: A PTC Business. Main. https://www.onshape.com/. Accessed 6 Mar 2020 25. Audacity (2020) Free, open source, cross-platform audio software. Audacity. http:// audacityteam.org/. Accessed 6 Mar 2020 26. Perišić MM, Štorga M, Gero JS (2019) Exploring the effect of experience on team behavior: a computational approach. Des Comput Cogn 18:595–612 27. Panteli N, Fineman S (2005) The sound of silence: the case of virtual team organising. Behav Inf Technol 24(5):347–352 28. Yu R, Gu N, Ostwald M, Gero JS (2015) Empirical support for problem–solution coevolution in a parametric design environment. Artif Intell Eng Des Anal Manuf 29(1):33–44

This Is How I Design: Discussing Design Principles in Small Multidisciplinary Teams of Design Professionals Nicole B. Damen and Christine A. Toh

Abstract When addressing complex design problems, multidisciplinary teams are faced with the challenge of successfully communicating their knowledge, skills, and abilities. This study explores an approach to identifying design principles from experienced designers as they engage in design discussion. Two teams of three designers were asked to generate ideas, identify design principles and engage in design discussion to address a hypothetical design challenge in a controlled setting. Four themes emerged from the analysis of the self-reported principles (Problem Understanding, Idea Generation, Idea Refinement, and Solution Implementation), and reflect existing best practices identified in prior design literature. These findings suggest that simulating a design discussion to discuss design knowledge in relation to specific ideas could be a promising approach solicit insights from experienced designers to understand more about design principles and their practical application.

1 Codifying and Sharing Design Knowledge in Multidisciplinary Design Teams Multidisciplinary or cross-functional teams are formed in response to the rapid pace of innovation across industries and the increasing complexity of engineering problems [24]. The challenge of solving these complex design problems requires people to leverage their knowledge, skills, and abilities from different domains [28]. As such, effective knowledge sharing processes are increasingly important because each team member brings their own experience and domain expertise, and integrating this diverse knowledge base across domains is crucial for team success [21]. Research on expertise typically focuses more on the development and application of expertise than its integration across disciplines. Expertise has been argued as both a bane and a boon to problem solving in design [6, 33]. On the one hand, experts are also affected by design fixation [31]. On the other hand, experts have N. B. Damen
C. A. Toh (&) The University of Nebraska at Omaha, Omaha, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. S. Gero (ed.), Design Computing and Cognition'20, https://doi.org/10.1007/978-3-030-90625-2_6

93

94

N. B. Damen and C. A. Toh

acquired the essential procedural and domain knowledge required to more efficiently organize ill-structured information through their ability to diagnose the relevancy of information and subsequently apply that knowledge [25]. As information relevance is highly dependent upon the context, variations in information diagnosticity may differ greatly between domains. Over time, expert designers build a wealth of knowledge and experience to draw upon when faced with design problems. One way designers navigate their mental knowledge library is by creating ‘rules of thumb’ that help them navigate and make sense of information. These design principles, also known as design heuristics, are “a fundamental rule or law, derived inductively from extensive experience and/or empirical evidence, which provides design process guidance to increase the chance of reaching a successful solution” [10, p.3]. Principles and heuristics capture the designers’ ability to leverage their existing experience and knowledge to support their solution finding process. Designers also use design principles as a way to codify design knowledge and communicate innovative, archival practices to others in order to more creatively address design problems [34]. These domain specific references serve as a powerful tool for designers to concisely express complex concepts [9], but accurate interpretation is reliant upon the recipients’ understanding of the domain [26]. For example, the principles applied to the development of social systems [16] are likely to differ from those used to design secure and usable computer systems [27] or intelligent robotic systems [23]. Further complicating the study of design principles is work that has shown that there is a gap between what people think drives their actions and what can be inferred from observing their actual actions. Argyris and Schön delineate between these two concepts using two separate, co-existing theories which they respectively call espoused theory and theory-in-use [2]. While people are generally aware of their espoused theory (or can articulate one), they are often unaware of their theories-in-use that are implied by their actions. This disconnect is relevant to team interactions, as designers are likely to articulate their espoused theories while the observing designers are more likely to pick up on the designer’s theories-in-use that they display through their behavior. There is also a temporal component to design principles. As a reflection of the designer’s knowledge base, design principles are not static records but dynamically adapt as designers accumulate more knowledge and experiences [19]. Prior work has shown that designers’ update these rules of thumb and add new heuristics while retiring obsolete heuristics that are no longer useful [17]. However, it is likely that there is more to this process than mere exposure to new information. People have been shown to respond slowly to corrections that are not in line with their views [18], and misinformation has been shown to affect behavior even when the information was acknowledged to be false [8]. These studies demonstrate the dynamic way that design principles evolve and are applied, and suggest that there is a complex relationship between individual knowledge and context of application during the design process. Due to the highly personal and often unstated nature of design principles in practice, understanding the principles used by designers can be challenging. One way to elicit implicit

This Is How I Design: Discussing Design Principles …

95

knowledge held by designers is through collaborative discussions between designers working towards a mutual goal. This forces the designer to reflect upon their design knowledge and, if appropriate, accommodate new insights. Therefore, team discussion about design can provide more insights into the concept of design principles and how they are applied in a team setting. This study aims to contribute to the design literature by identifying the design principles used in practice by exploring the following research questions: RQ1: What principles do designers use during early phase ideation? RQ2: What themes emerge across the principles used by the designers?

To address these questions, a qualitative laboratory study was conducted with practicing designers who formed small temporary design teams to address a hypothetical open-ended design problem.

2 Methodology 2.1

Participants

An in-depth qualitative study was conducted with a total of six practicing designers with between 3 to 17 years of experience (see Table 1 for relevant participant characteristics). All participants were identified through the authors’ professional networks and through snowball sampling. Only designers who had obtained at least three years of software, graphic, or UI/UX design experience and currently engage in design activities as their primary function in their full-time jobs were recruited for this study.

Table 1 Relevant characteristics of the designers Team number, Participant number

Design experience

Position title & Time in current position

Organization size and sector

Team 1, D1

3 yrs

CTO, =L1 >= 3'

Fig. 1 Selected rules of the Hajjar grammar divided in four groups: (1) rules 1–2: capturing how houses are situated on the lots, (2) rules 3–5: describing the formal relationship between the volumes, (3) rules 6–31: describing the division of interior spaces, and (4) rules 32–41: generating details

6 Walter Gropius, Marcel Breuer, and the Bauhaus Legacy The Bauhaus school of design, founded by a group of architects and artists led by Walter Gropius, formally came into existence in 1919 in Weimar, Germany. Their fundamental focus was on finding solutions to the problems experienced by the

Tracing Hybridity in Local Adaptations of Modern Architecture …

177

working classes after World War I. The core concept of the group was to reimagine the material world in order to express unity among all the arts. Gropius described this concept of unifying arts and design in the Proclamation of the Bauhaus (1919), in which the Bauhaus is described as a craft organization combining architecture, sculpture, and painting in a unified creative expression [16]. Gropius resigned from the Bauhaus in 1928, followed by Marcel Breuer, Laszlo Moholy-Nagy, Herbert Bayer, and Xanti Schawinsky, because of the rise of the political dictatorship in Europe and its detrimental effects on German culture. After the closure of the school in 1933, most members of the Bauhaus left Germany to take teaching positions abroad. By the time World War II broke out, most were teaching at major schools in the United States, where they were to influence an entire generation of American artists and architects. In 1937, with the appointment of Walter Gropius as the director of its Department of Architecture, Harvard’s Graduate School of Design (GSD) became the nation’s most prominent school training students in modernist architecture. With the exception of Harvard and the Armour Institute of Chicago, the latter of which was led by Mies van der Rohe from 1938 to 1959, most U.S. architecture programs remained under the Beaux-Arts system of education (or alternatives) until after World War II. The fact that Harvard’s GSD and the Armour Institute were the two pioneer schools in training students according to modernism shows the importance of Gropius (and Breuer) and van der Rohe in the United States, and also the importance of the Bauhaus legacy to the country’s architecture pedagogy. While many other schools continued to focus on the Beaux-Art, some architecture professors and architecture programs followed Gropius and Mies by teaching students according to the principles of modern architecture. Among these was Lawrence Anderson at MIT. It is worth noting that with William Wurster’s appointment as Dean in 1945, MIT became the third major program to promote modernism. Soon after Gropius became the director of Harvard’s Department of Architecture, Breuer joined him in the US, not only to teach with him at Harvard, but also to form a brief architectural partnership. While in Germany, Gropius focused on large-scale buildings, such as apartment buildings and institutional projects. In the US, however, in partnership with Breuer, he started to design single-family architecture. Gropius House, their first project, was a single-family home in Lincoln, MA, created for the Gropius family. It was designed in 1937 with construction ending in 1938. As the authors explain in a previous paper [3], modest in scale in comparison to other houses in the area, Gropius House was revolutionary in terms of its impact. Traditional elements of New England architecture, such as wood, brick, and local stone were combined with modern materials, such as glass block, acoustical plaster, and chrome banisters. A National Historic Landmark, Gropius House is known for localizing Bauhausian architecture in the New England area and more generally in the United States (Fig. 2).

178

M. Hadighi and J. Duarte public

R-A-4:

In.S

L1

Tr.

L2

L1 In.S

W1

private

Con.

In.S

W2

W1

L3

R-A-3:

L1

W1

In.S

W1

private

W1

In.S

L1

Br

Tr.

W

R-A-1:

public

L1 17 >= W1

W3

Liv

L1

R-B-2

L

R-B-3:

L

W1

24 >= W1 >= 36

W

L

In.S

W

W1 W2

W3

L1

W1

R-B-4:

L1

R-B-20:

L

W1

L2 L3

L1

W = W1 + W2 L = L1 + L2

W1

W

R-B-19:

L1 St

L1

L1 Bat W

L1 =< L W1 =< W

W1

W1

W1

R-B-14:

L1

R-B-15:

L1

Br W W1

L1

Kit

W2

R-B-21:

L1

In.S

In.S

L1

W1

L2

W1 W2 W

R-C-4:

L

L2 L1 W1 >= 0 3' 4" >=W2 >= 6' 2' 4" >=L1 >= 3'

Liv Fi

Din

W

Fi= Fireplace

Liv

W1 L1

L

R-C-5:

W

L1

F

L1

F

L1

F

L1

F

0 >=L2 > L-L1

Fig. 2 Selected rules of the Gropius-Breuer grammar divided in four groups: (1) rules A1–A7: describing the formal relationship between the volumes, (2) rules B1–B11: division of the interior space to smaller rooms, (3) rules B12–B24: allocation of the interior functions and (4) rules C1– C9: interior detailing

Many features of Gropius House can be seen in Hajjar’s houses in the State College area, given that many of the latter combine minimal, simple, and modest modern design with local materials; have large panes of glass that afford a view of the landscape; and include a screened porch as an American architectural element. Additionally, Hajjar’s use of the rectangular pattern/grid and the driving elements of his interior plans are to some extent similar to the interior plan of Gropius House. As explained in a previous paper [3], similarities between Hajjar’s designs and those of Gropius–Breuer in the United States became more pronounced following the period Breuer spent studying binuclear organization for American houses. That is, in 1943, Breuer spent time developing his idea of using a two-part organization

Tracing Hybridity in Local Adaptations of Modern Architecture …

179

for residential houses. The Geller House in Lawrence, NY, is one of the first houses to reflect Breuer’s idea of a binuclear house, with two wings/parts for day-time activities and night-time activities separated by an entry hall. Most of the houses designed by Breuer-Gropius from this point onwards, including Robinson House (1946), Alworth House (1954–1955), and Hooper House (1956–1959), have a similar organization. Likewise, many of the houses that Hajjar designed in the 1950s also have a two-part organization in the same style.

7 Gropius–Breuer Grammar In order to compare grammars, they should be developed in the same way and at the same level of detail. Therefore, the grammar for the single-family houses designed by Gropius and Breuer in the United States was developed with the same strategy used for the grammar of Hajjar’s work. When grammars are developed with the same strategy, it is easiest to compare them by determining which rules are maintained, deleted, changed, or added (created) from one grammar to the other. Similar to the way in which Hajjar houses were analyzed to develop the grammar for his work, all single-family houses designed by Gropius and/or Breuer in the United States were analyzed to reveal the volumetric design strategies, interior organization, contextual and functional relationships, and cultural, structural, and material dependency. Rules were developed based on these analyses and with the same strategy followed in the development of Hajjar’s grammar. Figure 2 shows selected rules of the grammar developed for the single-family houses designed by Gropius and/or Breuer in the United States. The grammar can produce both early houses, which the architects designed in a style that closely resembles the

In.S

Connection

In.S

In.S

Connection

In.S

In.S

Connection

Connection

In.S

In.S

Connection

Connection

In.S

Connection

In.S

Connection

In.S

Garage

In.S

Garage

Connection

In.S

Garage

In.S

Connection

In.S

Garage

In.S

Connection

Garage

Garage

Connection

Garage

In.S

Connection

Garage

In.S

Connection

Connection

... Garage

Garage

Garage

In.S

Connection

In.S

Connection

Garage

Garage

Garage

In.S

Connection

Garage

In.S

Connection

Garage

Connection

Connection

Connection

Connection

Connection

Garage

Connection

Garage

Connection

Garage

Garage

Connection

Connection

Garage

Garage

Garage

Garage

Garage

Garage

Garage

Garage

Garage

Garage

Garage

Garage

Garage

Connection

Garage

Garage

...

Garage

Fig. 3 Derivation of the Robinson House designed by Breuer in 1946

Garage

Garage

180

M. Hadighi and J. Duarte

Bauhaus and the binuclear plans. Figure 3 shows a step-by-step derivation of Gropius and Breuer’s Robinson House, as an example of a design used to infer the grammar. The grammar can generate additional plans in the architectural language of the Gropius–Breuer partnership in addition to all the houses designed by these architects in the US.

8 Traditional American Architecture in State College, PA Houses found in typical American neighborhoods are generally either “folk houses”—built usually by the occupants or non-professional builders without any specific intention of following current fashion—or “styled houses”—built with “at least some attempt at being fashionable” [1]. As Virginia and Lee McAlester state, most American houses that have survived from the nineteenth century are not folk houses but styled houses. Typical styles found in American neighborhoods are Colonial houses (1600–1820), Romantic houses or revival houses (1820–1880), Victorian houses (1860–1900), Eclectic houses (1880–1940), and houses since 1940 (including contemporary and neo-eclectic). The present study, however, focuses on American houses in State College, PA, with the aim of uncovering their architectural influences on Hajjar’s mid-twentieth-century architecture. State College is a college town located in central Pennsylvania. Similar to other typical American college towns, it is dominated both economically and demographically by the university that it hosts, in this case, the University Park campus of the Pennsylvania State University, commonly known as Penn State. Evolving from a village to serve the needs of Penn State (founded as the Farmers’ High School of Pennsylvania in 1855), State College was incorporated as a borough in 1896. Neighborhoods adjacent to the campus started to be developed mostly in the early twentieth century to expand with the growth of the university. To study single-family domestic architecture designed in traditional styles in the area, it is instructive to explore the College Heights Historic District, a national historic district located north of campus that was added to the National Register of Historic Places in 1995 (National Register Information System). As stated in the National Park Service’s registration form and explored in a previous paper [4], College Heights encompasses land and historic buildings associated with the early residential history of the town and “represents its growth and architectural development as an emerging college town” (p. 2). All historic districts, including College Heights, consists of “contributing” and “non-contributing” properties. The registration form for the College Heights district indicates that there are 278 contributing properties in this area. Although all the contributing houses

Tracing Hybridity in Local Adaptations of Modern Architecture …

181

have a special characteristic(s) in relation to the history of the neighborhood, the registration document highlights some properties as best examples of houses designed by local architects/contractors or popular mail-order catalogues. Most of these examples, constructed in the 1920s and 1930s, are built in bungalow, colonial (Dutch and Georgian), colonial revival, Georgian revival, and four-square styles. Of these architectural styles, two interior plans are particularly popular in the neighborhood: a four-square organization and a four-room organization with a hallway in the center, the latter of which is very similar to Hajjar’s first house in the area.

9 Grammar for Traditional American Houses The corpus of designs includes thirty-eight houses highlighted in the National Park Service’s registration form for College Heights Historic District as part of the 278 contributing properties in this area. As the authors explain in more detail in a previous paper [4], interior organization was the main feature considered in the process of analysis, with an emphasis on the two most popular interior plans in the area: the four-square and the four-room organization with a central hallway. As many of the houses in the district are bungalows, the grammar developed for this kind of house in Buffalo, NY, by Downing and Flemming [17] was also considered in the development of the grammar presented herein. It is important to note that although Downing and Flemming developed their grammar for houses in Buffalo, NY, as most of the houses in their corpus were catalogue homes similar to bungalow houses in State College, it is rational to use their rules in developing a grammar for traditional houses in State College. Based on the houses in the corpus and the rules adapted from the grammar for the Buffalo bungalows, a grammar was developed for traditional American houses in the State College area. Similar to the grammar for Hajjar’s single-family houses, the grammar for traditional American houses encompasses distinct groups of rules: rules to define the overall inhabitable space; rules to describe the way in which the interior space is divided into smaller spaces or rooms; rules to allocate the interior functions; and rules to generate details such as the placement of closets, the placement of a fireplace, and wall thickness. Figure 4 shows selected rules of the grammar developed for traditional houses in the area.

182

M. Hadighi and J. Duarte W2

R-A-1

W2

L1

L2

L1 DA : Design Area S : Setback L3 >= 0

W1

W

c

L3

d

L

DA b

a

L1

K (and f) is on ab in DA

r

In.S

W

R-B-2

L1

L

f

W1

W

r

r

L

In.S

f

28 >= W1 >= 22 f

W1

In.S

L1

W1

r

R-B-1

(0,0):K

R-A-2

S

28 >= W1 >= 22

f

28 >= L1 >= 22

44 >= L1 >= 36

f

W1

W

W1 W2

r

r

W>L W

R-B-6

W1

L

R-B-3

L

L

L f

W >= W1

W/2=W1+W2 -1'< W1 - (L/2) =L1 >= 3'

Fi= Fireplace

Fig. 4 Selected rules of the grammar developed for traditional houses divided in four groups: (1) rules A1–A2: describing the location of the house within the lot, (2) rules B1–B13: dividing the main space into smaller rooms, (3) rules C1–C12: allocating the interior functions, and (4) rules D1–D9: detailing the interior

10

Discussion and Comparison

As noted in the “Introduction,” in this paper, the authors focused on using the shape grammar methodology to compare Hajjar’s architecture in State College with both single-family houses designed by Gropius and/or Breuer in the United States and the traditional houses in the context. In order to do this, we identified three strategies to compare shape grammars and determine to which extent their corpus share similarities: (1) compare the rules of the grammars; (2) compare the derivation of similar houses by the three grammars, and (3) use one grammar to generate a house from the corpus of another grammar. The use of these strategies to determine to which extent Hajjar’s houses are similar to modern and traditional architecture is further described in following.

Tracing Hybridity in Local Adaptations of Modern Architecture …

183

The first strategy was to compare and contrast the rules of the three grammars. Developing the three grammars following the same process and with the same level of detail made it possible to make such a comparison in an appropriate way. The rules of the three grammars were organized according to eleven categories, e.g., rules for volumetric organization, rules for dividing the interior space, and rules for interior organization-circulation. As shown in Figs. 1, 2, and 4, there were four main groups of rules in development of the three grammars. Some categories/ groups of rules were broken down into subcategories, mainly for easier comparison. Then, the rules were compared to determine which rules in Hajjar’s grammar were borrowed with or without changes from the other two grammars and which rules were created from neither of the two grammars. Figures 5, 6, and 7 show the comparison of selected rules of the three grammars in eight of the eleven categories. Although the main purpose of this comparison was to analyze the qualities of Hajjar’s architectural language and to determine how it reflects traditional American architecture and the influence of European modern architecture (the latter represented here by the Gropius–Breuer partnership), the shape grammar methodology provides a way to quantify these influences. A comparison of the respective rules reveals the following: In relation to the Gropius–Breuer grammar, 54% of Hajjar’s rules are the same (or have minimal changes), 13% reflect an adaptation (used with some modifications), and 33% are new rules. In relation to the traditional grammar, 29% of Hajjar’s rules are taken without changes, 17% reflect an adaptation (used with some modifications), and 54% are new rules. In relation to Hajjar’s generation of his own rules, 25% of the rules in his grammar are not used in either the Gropius–Breuer grammar or the traditional grammar. Finally, in relation to a comparison across all three grammars, 25% of the rules used by Hajjar are also the same in the other two grammars, meaning that the three grammars all have the same 25% of their rules in common (Fig. 8). This 25% commonality does not necessarily indicate the influence of one grammar on the other two. Instead, it reflects the idea that a house is a house regardless of its designer, its style, or the time period in which it was built. It is also worth noting that the Gropius‒Breuer grammar has 30% of its rules in common with the traditional grammar, and 7% of the rules reflect an adaptation (are used with some modifications). Based on these same rules and similar rules, we can expect to find similar results in many architectural languages, reflecting the extent to which a house is a house. However, 10% of the rules in the grammar for Gropius and/or Breuer’s work in the US are similar to the rules in the traditional grammar in a way that suggests that their work in the United States was to some extent influenced by American traditional architecture. The second strategy to verify possible similarities between shape grammars is through a step-by-step comparison of their derivations, in this case between those of a house designed by Hajjar, a house designed by Gropius–Breuer, and a house from the traditional context. The authors have explored this approach in detail in previous papers [3] and [4] and showed the way in which a step by step comparison of

184

M. Hadighi and J. Duarte

Traditional W2

W2

L2

L1

L1

W2

a

b

Finding the inhabitable space

L1

L

DA

DA : Design Area S : Setback L3 >= 0

W1

W2

L3

d

W

c

Gropius-Breuer

Hajjar

S

W L

DA

W1

0

W

Wd2

Wd1 0

W1

L

DA

L S

S = Setback DA = Design Area

W1

W1

W

L1

L2

BA

L1

a

-10° = W1

Fig. 5 Comparison of the rules of the three grammars for locating the inhabitable space, dividing the inhabitable space, defining the garage space, and defining private vs. common spaces

Tracing Hybridity in Local Adaptations of Modern Architecture …

Traditional

185

Gropius-Breuer

Hajjar public

L1

In.S

L2

L1 In.S

Tr.

private

Con.

In.S

W2

public

L3

L1

L2

L1

Tr.

private

Con.

In.S

In.S

W3

W1

W1

In.S

W2

L3

W3

W1

W1

W1

Binuclear organization

In.S L1

In.S

W3

W2 L2

L1

In.S

Con.

W1

L1

W1

L2

L1

L1

Con.

L3

In.S

W2

In.S

G

Con.

L2

L1

In.S

L3

L4 W1

L1

G

L4

L3

L2

L1

W2

W3

In.S

W3

W2

In.S

L3

W3

W1

W1

W1

W1

W2 In.S

L1

Ci

L1

W3 L3

L2

This Rule is true if Rule R5-b is true W1

W1 W3

W2 L1

L2

Ci

L3

Ci

L1

W2

L1

L3

Con. W3

L1 W2

W1

W W = W1 + W2

W L1

W

W

F1

L2

L1

F1

W

F1

L2

F1

L1

L1 F2

L1

L2

F2

W

F1

L2

F1

F2

L2

F2

L1 F2

L1

L2

F2

L1

F: Any function

F: Any function

W

W L1

W1 W2

r

F: Any function

W

r

W

W1 W2

L

L2

L2 L3

f

W1 W2

W/2=W1+W2 W>L -1'< W1 - (L/2) = 1 3. For each voxel, check its possible neighbours as defined by the chosen connection types (Fig. 3). Create a graph edge connecting the voxel node to the respective neighbouring nodes if they exist. 4. For each voxel in List A, check if there is a path leading to at least one voxel in List G by performing DFS (Depth-First Search) algorithm. 5. (Optional) Store the shortest path to the first available ground voxel. To reduce unnecessary computation overheads, this path is not always the shortest one to the ground. 6. Voxels without paths are those that are ‘floating’ in space (Fig. 4). 7. These ‘floating’ voxels (noise) are deleted.

3.3.5

Spatial Proportionality

The other important architectural consideration is the notion of human scale. This ‘spatial proportionality’ parameter is directly defined by the real-world mapping between the voxel dimension and the human dimension. In our experiment setup, to be discussed in the next section, each side of a voxel unit has a metric equivalent of 225 mm. Thus, eight single voxels stacked vertically would approximate the dimension of a human figure. With this proportional referencing system, we could then compute the head clearance of any generated architectural space and evaluate their usability and reachability by the intended human inhabitants.

Fig. 3 (Left Column) All 26 neighbouring voxels to be checked for their connectivity with the focal voxel in red. (Middle Column) Three different definitions of voxel-to-voxel connectivity: Face-to-Face, Edge-to-Edge and Corner-to-Corner. (Right Column) The three definitions of voxel-to-voxel connectivity in their respective exploded view

Voxel Synthesis for Architectural Design

307

Fig. 4 An example of a generated output with N = 3. Voxels in List A with at least a shortest path (shown with red arrows) to any available ground voxels in List G are identified as structurally connected (shown within the black outline in bold). The ‘floating’ voxels (in light grey) will be deleted automatically by the post-processing algorithm

4 Experiment Setup In this section, we will describe the set of architectural models and parameter settings used for the design experiments. Beginning with the critical selection of the architectural input models, we perform 4 different experiments consisting of arithmetic operations on two positive inputs and one negative input.

4.1

Input Data

Le Corbusier’s Maison Dom-ino is both an architectural idea and a project, that has led to the birth of the Modernist formal vocabulary in his own Five Points of Architecture [25]. The Dom-ino has been chosen as the core input model for our study in this paper. The bounding box’s dimensions of our Dom-ino voxel model are as follows: 37 units along the X axis, 40 units along the Y axis and 33 units along the Z axis, thus a total of 48,840 voxels. In this case, each orthogonal side of our cubical voxel is at the architectural scale of 225 mm. This unit of 225 mm also agrees with the standard co-ordinating size for brickwork which measures 225 mm by 112.5 mm by 75 mm (length
depth
height). In order to have a comparative analysis and evaluation of the generated outputs, the experiments have been carried out using two different versions of the Maison Dom-ino – one (Maison Dom-ino A) with the original open free plan and facade configuration based on columns and cantilevered slabs in concrete; the other

308

I. Koh

Fig. 5 (LEFT) Maison Dom-ino A. (RIGHT) Maison Dom-ino B. Both input voxel models used for our design experiments. For ease of reference in the text, the longer edge of the bounding box is the X-axis, the shorter the Y-axis and the vertical edge the Z-axis, with the origin (0,0,0) at the corner nearest to the bottom edge of the image

(Maison Dom-ino B) with a closed plan and facade configuration based on shear wall construction in masonry (Fig. 5). In our experiment, we will observe how different N parameters might give rise to different outputs given each of these models both individually and in combination. This fundamental contrast in both model’s architectural spatial concept and qualities will enable us to visually detect and identify the formal coherence of our generated outputs, in relation to the respective input models learned and the parameters used. A recent work on the original WFC algorithm [26] is the inclusion of negative exemplar input sources, in addition to the regular sources, which we will call positives from now on. The relations between patterns inferred from negative sources are considered negative relations, and are forbidden—overriding the possible relations learned in the positives. For our experiment, we have used a subset of our positives from Maison Dom-ino A as negatives (Figs. 6 and 7) to test if such an approach would indeed remove all traces of staircases from any outputs generated from Maison Dom-ino A. We shall call this negative set as Dom-ino-Stairs from now on.

4.2

Prototype Interface

To facilitate quick iterations with different input models and parameter settings, a web-application prototype was implemented. More importantly, the application allows the designer to observe the algorithm’s behaviour in relation to the specificity of the given voxel model input. This is achieved by visualizing the SCP procedure and the corresponding states of all the regions in real-time (Fig. 8). The grayscale translucent regions represent the regions’ current entropy (dark grey = low and light grey = high). In the event of a contradiction, the region turns red accordingly. After each generation, the designer can change the parameters and start

Voxel Synthesis for Architectural Design

309

Fig. 6 The Dom-ino-Stairs: The 30 negative patterns (in red) selected from a subset of the 447 positive patterns (in black) of the input model Maison Dom-ino A with N = 3

Fig. 7 The Dom-ino-Stairs: Close-up view of the selected negative patterns

Fig. 8 Sequence of screenshots showing the propagation in real-time on the web interface, where each grey cube represents region that is yet to be collapsed. The darker cubes have lower entropy than the lighter ones. The parameter settings are as follows: N = 6, Size Factor = 5, Input Model = Maison Dom-ino A, ‘Allowable Contradictions’ = 0.5, ‘Grounding’ = False and ‘Orientation Specificity’ = True

310

I. Koh

another creation. The different outputs are stored throughout the creation process, and the designer can revert to any of the previous versions. The designer can also create new input models using the built-in voxel editor. After adding any number of input model types (positive or negative) and tweaking the relevant parameters, the designer can begin the generation process until he/she is satisfied with the creation.

5 Results In this section, we will review and analyse the resulting outputs generated from the 4 different experiments namely Maison Dom-ino A, Maison Dom-ino B, Maison Dom-ino A [plus] Maison Dom-ino B and Maison Dom-ino A [minus] Dom-ino-Stairs (Fig. 9). Each experiment has been tested with these settings —‘Allowable Contradictions’ = 0.5, ‘Orientation Specificity’ = True and ‘Grounding’ = False. With the exception of the fourth experiment where N is determined by that of the negative patterns, all the N values used are 3, 5 and 6, with their corresponding Size Factors as 10, 6 and 5; thus, all results will yield the same output size of 27,000 voxels. With N = 3, each pattern would approximate a cube with side measuring 675 mm (225
3); with N = 5, a cube with each side measuring 1125 mm (225
5), and with N = 6, a cube with each side measuring 1350 mm (225
6). According to the Modulor [27], N = 8 is then at the human scale (HS) of a standing person. Thus, N = 3 equates to < 0.5*HS, N = 5 equates to > 0.5*HS but < 0.75*HS, and N = 6 equates to >= 0.75*HS but 0.05; two-way ANOVA with factors of design phase and creativity mode, type 3 sum-of-squares, p > 0.05; Kruskal–Wallis test with factor of card, p > 0.05). We thus cannot argue that particular design phases, creativity modes, combinations of design phases and creativity modes, or cards were perceived to be more useful than others.

4.2

Perceived Utility by Design Team and Project Type

To examine how team influenced perceived utility of cards, we observe team-specific distributions of utility (Fig. 3). Team 7, a product innovation team with four members, found the cards most useful (mean = 4.56, sd = 0.60, n = 54), while team 5, a product-service system team with three members, found the cards least useful (mean = 3.39, sd = 1.07, n = 28). The differences between team means was deemed to be statistically significant (Kruskal–Wallis test comparing response values across teams; p < 0.05). A post-hoc Dunn test was conducted to determine the significance of pairwise differences between teams. Among 36 comparisons, 13 differences were deemed significant (p < 0.05). Among these comparisons, however, the only team that was consistently significantly different from all other teams was team 7, accounting for 8 of the significant comparisons. Thus, we can conclude that team 7, pursuing a product innovation project, found cards more useful than other teams. We also note that teams 1, 2, and 3, differed significantly (p < 0.05) from one another. All of these teams were pursuing spatio-social projects. To examine how project type influenced perceived utility of cards, we observe type-specific distributions of utility. Product innovation projects found the cards most useful (mean = 3.91, sd = 0.81, n = 181), while Product-Service System innovation projects found the cards least useful (mean = 3.65, sd = 1.02, n = 75). The differences in perceived utility between project types, however, were not statistically significant (Kruskal–Wallis test, p > 0.05). Examining differences between teams of each project type (Fig. 4), we note that as aforementioned, Team 7 exhibited significant differences from others (Kruskal– Wallis test with post-hoc Dunn test, p < 0.05). Among product-service system project teams, no significant difference was found (Student’s t-test, p > 0.05). Among spatio-social project teams a significant difference was found (Kruskal– Wallis test, p < 0.05), and a post-hoc analysis revealed Team 2 to be significantly different than other spatio-social teams (Dunn test, p < 0.05). Examining differences by cybersecurity risk (Fig. 5), we detect no significant pairwise differences (Kruskal–Wallis test with post-hoc Dunn test, p > 0.05). However, we note that projects with the least cybersecurity risk had the highest average perceived utility from the cards.

Design for Cybersecurity (DfC) Cards: A Creativity-Based Approach …

361

Fig. 3 Perceived utility of all card-based interventions, by project team. Team numbers, shown above each graph, are matched to project topics in Table 1

Fig. 4 Perceived utility of all card-based interventions, by project type

362

V. Rao et al.

Fig. 5 Perceived utility of all card-based interventions, by cybersecurity risk

5 Discussion The cards were shown to be perceived as helpful aids to the overall design process for the entire cohort of designers, given an above-positive class average assessing perceived usefulness. We find this result promising to support a hypothesis in response to our original RQ1: that card-based interventions to support cybersecurity’s integration across discrete phases of the design process, organized by creativity mode, could be an effective way to support novices as they design for cybersecurity. In response to our RQ2, however, we find inconclusive results. First, we cannot determine with confidence how card interventions are differentially perceived to be useful, whether organized by creativity mode, design phase, or intervention itself. This owes to insignificant differences in data organized by creativity mode, design phase, or card intervention outlined previously. However, the lack of a significant difference leads us to revise our hypothesis: that card interventions are perceived as equally helpful across the entire design process. We do identify significant differences in results in response to our RQ3. While we did discover that a team focused on a product innovation project found greater average usefulness than other teams, other product innovation project teams did not exhibit notably high nor consistent perceived utility scores. Similarly, among teams pursuing spatio-social innovation projects, there was a significant difference between team two, suggesting that within the same project type, perceived utility of the card intervention could vary significantly. The differences between perceived utility by project type were not significant, however. Results related to RQ2 suggest several interesting directions for inquiry. Regarding the lack of a discrete effect from a single card, this suggests that

Design for Cybersecurity (DfC) Cards: A Creativity-Based Approach …

363

longitudinal intervention across a design project may be superior to discrete interventions at specific phases, creativity modes, or intersections of the two. Further research on this topic could deepen the design community’s understanding of the nature of process support, and under what conditions discrete support is superior to longitudinal, and vice versa. We are intrigued by the lack of significant difference of perceived utility between project types in this study, as evidenced in RQ3. That product innovation teams and spatio-social teams find equal utility in cybersecurity support suggests that novice designers value cybersecurity process support, even if such support may not be relevant to their designs. For example, Team 7 was pursuing a project around the future of tableware and cutlery, and their final prototype was a physical set of utensils with no digital element. While there is no evident overlap of cybersecurity with the team’s final prototype direction, it is interesting that the designers found cybersecurity support to be valuable throughout the design process, even if it had little evident impact on their final prototype. Among spatio-social teams, Team 2 developed a system to safely assign work spaces in shared facilities. Unlike other spatio-social projects, ‘safety’ was explicitly the team’s goal, leading us to speculate this was a driver of increased perceived utility, though the exact drivers are unknowns. However, the exact drivers remain unknown. This finding is further bolstered by the lack of significant difference between project teams’ perception of the cards’ utility when examined by relevance of cybersecurity to the project. Our results suggest that even with minimal influence on the design outcome or topic, cybersecurity is a topic students find significantly useful in design practice. We speculate that cybersecurity’s appeal to design students is grounded in provoking other paradigms of thinking about the team’s project. Furthermore, as our sample size was limited to novice (student) designers, students might be finding relevance of our design interventions in other aspects of their design work. As cybersecurity concerns continue to permeate products, services, and experiences, we plan to continue to explore how interventions like the DfC cards can support designers’ awareness of cybersecurity in future work.

5.1

Implications for Including Cybersecurity in the Design Process

As described earlier, there is pressing need to integrate cybersecurity into the design process. The results here aim to begin a discussion on how to do so, and highlight two immediate implications. First, the concept of cybersecurity appears to offer a helpful influence on human-centered design projects of a range of types and topics. As a relatively new topic for many designers, cybersecurity may be valuable content to deliver in absolute. Second, given the lack of significance in differences between cards and perceptions thereof, we suspect that a question-based modality does not allow designers enough engagement to differentiate between design

364

V. Rao et al.

interventions. We expect that questions must be supplemented with curated exercises and content to create more differentiable value for designers.

5.2

Limitations

This study has several important limitations. First, the card-based interventions were examined in a linear design process in a classroom. In contrast, much of design practice is nonlinear. Second, participants had significant autonomy in how to engage with cards, and our measurements of perceived utility are sensitive to external factors, like team dynamics. Third, we did not capture qualitative evaluations of the cards themselves, e.g., student explanations of why they used the cards, essential data for deeper conclusions. Finally, we did not examine the quality of designs, nor evaluate the presence of pro-cybersecurity features in final designs. We are actively seeking to pursue studies resolving all of the above in future work.

6 Conclusions This work presents Design for Cybersecurity (DfC) cards based on an engineering creativity model to help designers engage with cybersecurity in human-centered projects. Each card stimulates designers’ creativity in a specific design phase, and fifteen cards were produced and studied in a longitudinal participant in a project-based course. The cards were perceived to have utility by participants, and while differential utility between projects or phases could not be determined with statistical significance, our findings invite further study. Acknowledgements The research team gratefully acknowledges support from the Center for Long-Term Cybersecurity (UC-Berkeley).

References 1. Tweneboah-Koduah S, Skouby KE, Tadayoni R (2017) Cyber security threats to IoT applications and service domains. Wirel Pers Commun 95:169–185 2. Suo D, Siegel JE, Sarma SE (2018) Merging safety and cybersecurity analysis in product design. IET Intell Transp Syst 12:1103–1109 3. Abomhara M, Ien GMK (2015) Cyber security and the internet of things: vulnerabilities, threats, intruders and attacks. J Cyber Secur Mobil 4:65–88 4. Williams PA, Woodward AJ (2015) Cybersecurity vulnerabilities in medical devices: a complex environment and multifaceted problem. Med Devices Auckl NZ 8:305–316 5. Reid SE, Brentani UD (2004) The fuzzy front end of new product development for discontinuous innovations: a theoretical model. J Prod Innov Manag 21:170–184

Design for Cybersecurity (DfC) Cards: A Creativity-Based Approach …

365

6. Zhang Q, Doll WJ (2001) The fuzzy front endand success of new product development:a causal model. Eur J Innov Manag 4:95–112 7. Roschuni C, Kramer J, Agogino A (2016) Design talking: how design practitioners talk about design research methods. https://doi.org/10.1115/DETC2015-47843 8. Roschuni C, Kramer J, Zhang Q, Zakskorn L, Agogino A (2015) Design talking: an ontology of design methods to support a common language of design. In: Proceedings of the international conference on engineering design 9. Schwartz S, Ross A, Carmody S, Chase P, Coley SC, Connolly J, Petrozzino C, Zuk M (2018) The evolving state of medical device cybersecurity. Biomed Instrum Technol 52:103– 111 10. Yuan X, Yang L, Jones B, Yu H, Chu B-T (2016) Secure software engineering education: knowledge area, curriculum and resources. J Cybersecur Educ Res Pract 2016:3 11. Lukowiak M, Radziszowski S, Vallino J, Wood C (2014) Cybersecurity education: bridging the gap between hardware and software domains. ACM Trans Comput Educ TOCE 14:2 12. Assal H (2018) The human dimension of software security and factors affecting security processes. Carleton University, Ottawa 13. Navas J, Voirin J-L, Paul S, Bonnet S (2019) Towards a model-based approach to systems and cyber security co-engineering. INCOSE Int Symp 29:850–865 14. Hatchuel A, Chen MK (2017) Creativity under strong constraints: the hidden influence of design models. Eur Rev 25:194–207 15. Islam T, Becker I, Posner R, Ekblom P, McGuire M, Borrion H, Li S (2019) A socio-technical and co-evolutionary framework for reducing human-related risks in cyber security and cybercrime ecosystems. In: Wang G, Bhuiyan MZA, De Capitani S, Vimercati S, Ren Y (eds) Dependability in sensor, cloud, and big data systems and applications: 5th international conference, DependSys 2019, Guangzhou, China, November 12–15, 2019, Proceedings. Springer, Singapore, pp 277–293. https://doi.org/10.1007/978-981-15-1304-6_22 16. Yilmaz S, Christian JL, Daly SR, Seifert C, Gonzalez R (2012) How do design heuristics affects outcomes? In: Proceedings of DESIGN 2012, the 12th international design conference (DS 70), Dubrovnik, Croatia, pp 1195–1204 17. Yilmaz S, Seifert C, Daly SR, Gonzalez R (2016) Design heuristics in innovative products. J Mech Des. https://doi.org/10.1115/1.4032219 18. Method Cards. https://www.ideo.com/post/method-cards. Accessed 30 Nov 2019 19. AI & Ethics: Collaborative Activities for Designers. https://www.ideo.com/post/ai-ethicscollaborative-activities-for-designers. Accessed 30 Nov 2019 20. Superpowers Card Deck. In: SYPartners. https://madeby.sypartners.com/products/ superpowers-card-deck. Accessed 30 Nov 2019 21. Drivers of change digital app for iOS and Android. http://www.driversofchange.com/. Accessed 30 Nov 2019 22. IDC Design Method Card (2019). In: SUTD-MIT International Design Centre. https://idc. sutd.edu.sg/design-contributions/creations/idc-design-method-card. Accessed 30 Nov 2019 23. Deng Y, Antle AN, Neustaedter C (2014) Tango cards: a card-based design tool for informing the design of tangible learning games. Proceedings of the conference on designing interactive systems. ACM, New York, pp 695–704 24. Brandt E, Messeter J (2004) Facilitating collaboration through design games. In: Proceedings of the eighth conference on participatory design: artful integration: interweaving media, materials and practices, vol 1. ACM, New York, pp 121–131 25. Halskov K, Dalsgaard P (2006) Inspiration card workshops. In: Proceedings of the 6th conference on designing interactive systems. ACM, New York, pp 2–11 26. Wölfel C, Merritt T (2013) Method card design dimensions: a survey of card-based design tools. In: Kotzé P, Marsden G, Lindgaard G, Wesson J, Winckler M (eds) Human-computer interaction – INTERACT 2013. Springer, Heidelberg, pp 479–486. https://doi.org/10.1007/ 978-3-642-40483-2_34

366

V. Rao et al.

27. Lauff CA, Perez KB, Camburn BA, Wood KL (2019) Design principle cards: toolset to support innovations with additive manufacturing. In: Proceedings of the 24th design for manufacturing and the life cycle conference, vol 4. American Society of Mechanical Engineers, Anaheim, p V003T05A005 28. Yilmaz S, Seifert CM (2011) Creativity through design heuristics: a case study of expert product design. Des Stud 32:384–415 29. Hasling KM, Ræbild U (2017) Sustainability cards: design for longevity. In: PLATE 2017 product lifetimes environment 30. Ræbild U, Hasling KM (2018) Sustainable design cards: a learning tool for supporting sustainable design strategies. In: Sustainable fashion in a circular economy. Aalto University, Espoo, pp 128–151 31. Lucero A, Arrasvuori J (2010) PLEX cards: a source of inspiration when designing for playfulness. In: Proceedings of the 3rd international conference on fun games. ACM, New York, pp 28–37 32. Kim E, Jensen MB, Poreh D, Agogino AM (2018) Novice designers’ lack of awareness to cybersecurity and data vulnerability in new concept development of mobile sensing devices. In: 92 Proceedings of the design 2018 15th international design conference. https://doi.org/10. 21278/idc.2018.0461 33. Kim E, Kwon J, Yoon J, Agogino AM (2019) Embedding cybersecurity into design education: increasing designers’ awareness of cybersecurity throughout the design process. ASME 2019 international design engineering technical conferences and computers and information in engineering conference 34. Mouheb D, Abbas S, Merabti M (2019) Cybersecurity curriculum design: a survey. In: Pan Z, Cheok AD, Müller W, Zhang M, El Rhalibi A, Kifayat K (eds) Transactions on edutainment XV. Springer, Heidelberg, pp 93–107. https://doi.org/10.1007/978-3-662-59351-6_9 35. Luburić N, Sladić G, Slivka J, Milosavljević B (2019) A framework for teaching security design analysis using case studies and the hybrid flipped classroom. ACM Trans Comput Educ TOCE 19:21 36. Huynh D, Luong P, Iida H, Beuran R (2017) Design and evaluation of a cybersecurity awareness training game. In: Munekata N, Kunita I, Hoshino J (eds) Proceedings of the entertainment computing – ICEC 2017: 16th IFIP TC 14 international conference, Tsukuba City, Japan, 18-21 September 2017. Springer, Cham, pp 183–188. https://doi.org/10.1007/ 978-3-319-66715-7_19 37. Cards. In: Cyber realm. https://gencybercards.com/cards. Accessed 30 Nov 2019 38. Jin G, Tu M, Kim T-H, Heffron J, White J (2018) Evaluation of game-based learning in cybersecurity education for high school students. J Educ Learn EduLearn 12:150 39. Chu ES, Payne A, Seo JH, Chakravorty D, McMullen D (2019) Data center physical security training VR to support procedural memory tasks. In: Stephanidis C (ed) HCI international 2019 - posters. Springer, Cham, pp 353–358 40. Coles-Kemp L (2009) Information security management: an entangled research challenge. Inf Secur Tech Rep 14:181–185 41. Coles-Kemp L, Ashenden A (2012) Community-centric engagement: lessons learned from privacy awareness intervention design. In: 26th BCS conference on human computer interaction. pp 1–4 42. Coles-Kemp L, Theoharidou M (2010) Insider threat and information security management. In: Probst CW, Hunker J, Gollmann D, Bishop M (eds) Insider threats in cyber security. Springer, Boston, pp 45–71. https://doi.org/10.1007/978-1-4419-7133-3_3 43. Lewis MM, Coles-Kemp L (2014) Who says personas can’t dance? The use of comic strips to design information security personas. In: CHI14 extended abstracts on human factors in computing systems. pp 2485–2490 44. Denning T, Shostack A, Kohno T (2014) Practical lessons from creating the control-alt-hack card game and research challenges for games in education and research. 2014 USENIX summit gaming games gamification security education 3GSE 14

Design for Cybersecurity (DfC) Cards: A Creativity-Based Approach …

367

45. Denning T, Lerner A, Shostack A, Kohno T (2013) Control-Alt-Hack: the design and evaluation of a card game for computer security awareness and education. In: Proceedings of the 2013 ACM SIGSAC conference on computer and communications security - CCS 13. ACM Press, Berlin, pp 915–928 46. Home. The security cards: a security threat brainstorming kit. http://securitycards.cs. washington.edu/index.html. Accessed 30 Nov 2019 47. Nick Merrill. https://cosmopol.is/adversary-personas/. Accessed 30 Nov 2019 48. Howard T, Culley SJ, Dekoninck E (2007) Creativity in the engineering design process. Proceedings of ICED 2007, the 16th international conference on engineering design (DS 42), Paris, France, 28-31072007 49. Howard TJ, Culley SJ, Dekoninck E (2008) Describing the creative design process by the integration of engineering design and cognitive psychology literature. Des Stud 29:160–180 50. Gero JS, Kannengiesser U (2004) The situated function–behaviour–structure framework. Des Stud 25:373–391 51. Sultan, Ahmad (2019) Report: improving cybersecurity awareness in underserved populations - CLTC UC Berkeley center for long-term cybersecurity 52. Ceschin F, Gaziulusoy I (2016) Evolution of design for sustainability: from product design to design for system innovations and transitions. Des Stud 47:118–163 53. Reniers G, Khakzad N, Gelder PV (2017) Security risk assessment: in the chemical and process industry. Walter de Gruyter GmbH & Co KG

Towards a Virtual Librarian for Biologically Inspired Design Ashok Goel, Kaylin Hagopian, Shimin Zhang, and Spencer Rugaber

Abstract In biologically inspired design, designers typically search for natural language documents describing biological systems relevant to their problems. Then they construct an understanding of the biological systems described in the documents for transfer to a given problem. These are difficult, labor intensive and time consuming processes. Thus, we are constructing a virtual librarian called IBID for supporting designers in locating and understanding biology articles relevant to their design problems. IBID first extracts knowledge of the function, the structure, and portions of the causal mechanisms of biological systems from their natural language descriptions. Then, it organizes this knowledge as a Structure-Behavior-Function (SBF) model. Finally, it uses the SBF annotations to retrieve biology articles relevant to design queries. To extract causal mechanisms, IBID uses machine learning techniques to identify portions of a document that describe causal processes.

1 Introduction Biologically inspired design is a well-known paradigm that uses nature as a source of practical, efficient and sustainable designs to stimulate design of technological systems [1, 2]. However, not all architects, engineers, and designers are experts at biology [3]. Thus, not all designers have knowledge of a large number of biological systems stored in their internal memories, or a deep understanding of the biological systems available in their memories. Instead, in practice, given a design problem, many designers search for natural language documents describing biological systems and then construct an understanding of the retrieved systems for potential transfer to the design problem. In [4], we found that often this process is a labor intensive and time consuming because of the difficulty of finding articles relevant to a design problem, recognizing the relevance of a biological article to a design A. Goel (&)
K. Hagopian
S. Zhang
S. Rugaber Design and Intelligence Laboratory, School of Interactive Computing, Georgia Institute of Technology, Atlanta, GA 30308, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. S. Gero (ed.), Design Computing and Cognition'20, https://doi.org/10.1007/978-3-030-90625-2_21

369

370

A. Goel et al.

problem, and understanding the biological article deeply enough to enable analogical transfer of the biological knowledge to the design problem. In earlier times, a human librarian might help designers in navigating a physical library and locating biology articles relevant to their design problems. In the current era, when increasingly large amount of biology literature is available online, there is a similar need for virtual librarians for finding relevant biology articles. In addition to locating relevant articles, we also want the virtual librarian to help designers in constructing a deep enough understanding of the biological systems described in an article to support analogical transfer. In this paper, we briefly outline the IBID interactive system for supporting designers in locating and understanding biology articles relevant to their design problems. IBID (for Interactive Biologically Inspired Design) first extracts knowledge of the function, structure, and causal mechanisms of biological systems from their natural language descriptions. Then, it organizes this knowledge as a Structure-Behavior-Function (SBF) model [5]. Finally, it uses the SBF annotations to retrieve biology articles relevant to new design queries. Below we first present the conceptual design of IBID. Then, we briefly describe IBID’s knowledge-based method for extracting portions of a causal process from text. A fully automated solution to the general problem of extracting causal processes from text is not yet available. Thus, next we describe IBID’s use of statistical machine learning techniques to identify portions of a document that describe causal processes. The identified portions can potentially be examined by the human designer for deeper analysis.

2 Conceptual Architecture of IBID Let us consider an expert designer who, like many designers, is a novice in biology. Let us suppose that the designer is interested in designing a system for transporting water to remote regions in her country. Given a large corpus of biology articles, how may we design a virtual librarian to help the designer locate and understand biology articles relevant to the design problem? There are three stages in building a virtual librarian, starting with knowledge representation. AI research has developed precise languages for representing many kinds of knowledge. In the context of the current work, Julian Vincent in the United Kingdom has developed a detailed language for capturing knowledge of biological systems [6]. Our research laboratory has developed a more abstract language called Structure-Behavior-Function (SBF; [5, 7]) for expressing design problems as well as design patterns and principles. Briefly, an SBF model of a biological or a technological system explicitly specifies the structure of the system (the components and the connections among them), the functions of the system (the outcomes of the system), and the causal mechanisms (the system’s behaviors) that explain how the structures of the system achieve its functions. The SBF model derives from Chandrasekaran’s Functional Representation scheme [8, 9] and are similar to but

Towards a Virtual Librarian for Biologically Inspired Design

371

distinct from the Function-Behavior-Structure model [10] and Function-BehaviorState model [11]. In previous work [12], we found that SBF models help a designer understand a complex biological system more deeply so as to better answer questions about its functioning. This led us to develop the Biologue system [13] for manually annotating biology articles based on SBF models, accessing biology articles relevant to a design problem based on the annotations, and using the annotations to help the designer understand the article in terms of SBF models. Experiments with Biologue showed that if biology articles are annotated with SBF models of the biological systems described in the articles, then many designers are better able to both locate biology articles relevant to their design problem and understand how the biological systems work. Thus, we posit that it may be productive if the virtual librarian for biologically inspired design too uses the SBF model as the knowledge representation scheme for capturing knowledge of biological systems. Second, the design of a virtual librarian requires a scheme for organizing knowledge of biological systems in a digital library and methods for accessing the knowledge as and when needed. AI research has developed detailed schemes for organizing knowledge into digital libraries and accessing it from the libraries. In the context of the present work, The Biomimicry Institute has developed a library called AskNature ([14]; https://asknature.org/) containing hundreds of biological strategies and their application to biomimetic design. We have developed a digital library called DANE ([15]; http://dilab.cc.gatech.edu/dane/) that captures an understanding of a biological system in the SBF language, as well as a digital library named DSL of searchable natural language documents describing case studies of biological inspired design [16]. These libraries enable a designer to locate a biological system relevant to a design problem; they also scaffold the designer’s comprehension of the biological system. Our design of IBID is specifically targeted towards acquiring knowledge of DANE’s SBF models. Third, the design of a virtual librarian requires methods for automatically acquiring knowledge of biological systems or automating the process as much as possible. AI research has developed several computational methods for automatically or semi-automatically acquiring knowledge for populating digital libraries. For example, in the context of this work, AI researchers have developed methods for acquiring biological knowledge from natural language documents on the Web [17]); [18] summarizes early attempts to develop computational techniques for literature-based discovery in biologically inspired design. More recent efforts include use of AI techniques for accessing and classifying natural language documents describing biological systems [19, 20], and discovering structure in patent databases [21]. Mueller et al. [22] have proposed acquiring biological design knowledge directly from animal fossils through machine vision. In past work, our laboratory developed a virtual librarian for biologically inspired design based on IBM’s Watson tool: it accessed biology articles relevant to a design query and answered questions based on the retrieved articles [23].

372

A. Goel et al.

The IBID system in the current discussion focuses on extracting SBF models of biological systems from natural language documents, and using the SBF annotations to locate biology relevant to a function specified in a controlled vocabulary. IBID (http://dilab.gatech.edu/ibid/) operates in two modes. First, it extracts SBF models of biology articles in a given corpus and annotates the articles with structural, behavioral and functional terms. Given a research article describing a biological system from a journal such as Chrispeels & Maurel [24] article in Plant Physiology, IBID extracts the function, the structure, and parts of the causal behaviors of the system. Second, given a design query, IBID locates biology articles relevant to the query based on the structural, behavioral and functional annotations. Figure 1 shows the full functionality of IBID for its three use cases: (1) End users such as engineers and designers looking for biology articles relevant to their design problems, (2) Knowledge engineers extending IBID’s knowledge representation vocabulary, and (3) System administrators adding to its repository of analyzed papers. Figure 1 also specifies the actions available to each user type; the arrows in the figure indicate progression of steps and/or access to/from the database.

Fig. 1 The conceptual architecture of IBID

Towards a Virtual Librarian for Biologically Inspired Design

373

3 Extant Tools Before we describe the conceptual architecture of IBID in more detail, we note that it uses several extant tools including the following:

3.1

Stanford Natural Language Parser

The Stanford Natural Language Parser (http://nlp.stanford.edu/software/lex-parser. shtml) generates parse trees of input sentences. IBID uses the core Stanford Natural Language Parser tool to construct the parse trees for sentences in biology articles (and natural language design queries). IBID uses the parse tree of a sentence to help identify if a part of the sentences refers to the structure, behavior or function of the biological system described in the article.

3.2

WordNet

WordNet (https://wordnet.princeton.edu) is a large lexical database of the English language in which different parts of speech are grouped into sets of synonyms (synsets), each expressing a distinct concept. Synsets are interlinked by means of conceptual-semantic and lexical relations. Given a design query expressed as an English language sentence, IBID uses WordNet to widen the set of search terms.

3.3

VerbNet

VerbNet (https://verbs.colorado.edu/*mpalmer/projects/verbnet.html) is an on-line verb lexicon that includes specific syntactic information and indications of verb class membership. Each verb class in VerbNet is described by its frames, thematic roles, and arguments. IBID uses VerbNet to identify and extract function terms from articles and expand its vocabulary of functions.

3.4

Vincent’s Vocabulary for Structure of Biological Structures

Julian Vincent has developed a detailed vocabulary for describing the structure of biological systems (Vincent [6]). IBID uses a small part of his vocabulary as a domain-specific controlled vocabulary of biological structural components.

374

3.5

A. Goel et al.

Domain-Independent Vocabularies for Structure, Behavior, and Function

We have developed domain-independent vocabularies of the structure, the behaviors, and the function of complex systems that build in part on the extant SBF vocabulary (Goel et al. [5]). Rugaber et al. [25] describes IBID’s functional vocabulary. IBID uses these controlled vocabularies to capture the structural, behavioral, and functional concepts and relationships in the description of a biological system and in design queries.

4 Extraction of Structure, Behavior and Function from Text The current version of IBID extracts functions, structure, and parts of the causal behaviors of a system from its natural language description. For each sentence in a biology article, IBID uses the Stanford NLP parser to obtain its phrase structure grammar representation in the form of a tree. Each valid phrase’s start token in the tree represents the root node of a subtree whose leaf words are combined to create a logical sentence component. For example, one component of “Minute water droplets from fog gather on its wings; there the droplets stick to…” is “Minute water droplets from the fog gather on its wings”.

4.1

Function Extraction

As indicated above, IBID uses a domain-independent controlled vocabulary of function terms [25]. Each term in this controlled vocabulary is expressed as a frame in VerbNet. The first step of function analysis is to generate a Stanford Dependency (SD) object for a given sentence component, the root of the SD tree is the predicate of the sentence and is then stemmed to produce the root verb. For example, “gather” is the root verb for the component “Minute water droplets from the fog gather on its wings”. SD also provides information on whether the root verb is passive by listing any passive nominal subjects. Root verbs for which there are VerbNet records will have their VerbNet syntactic frames matched against the parser’s Part-Of-Speech (POS) tags. For others, a custom algorithm uses WordNet to find the closest matching VerbNet word. The best matched predicate, its VerbNet syntactic frame, thematic relations mapping from our sentence component to the frame, and the sentence itself are saved in the database. IBID annotates the article with all this functional information.

Towards a Virtual Librarian for Biologically Inspired Design

4.2

375

Behavior Extraction

We have developed a domain-independent vocabulary for behavioral concepts and relations. Each behavioral term in this controlled vocabulary is expressed as a frame in VerbNet. As in function extraction, each sentence component is parsed for its predicate and then stemmed. Next, the root verbs are matched against causal verbs in our vocabulary of behavioral terms. If the system finds a causal verb, then IBID replaces it with a verb token and matches it against a list of predefined regular expressions capturing various forms of causal patterns. These causal regular expression patterns also delineate the sentence component’s cause and effect clauses. The causality record, which includes a stemmed predicate, its cause/effect clauses, and the original sentence component are then saved in the database. Finally, IBID annotates the article with this behavioral information.

4.3

Structure Extraction

In [26], we describe IBID’s extraction of structure of a biological system from its textual system. Briefly, IBID searches each sentence in the biology article for terms in Vincent’s domain-dependent structural vocabulary. If it identifies a structural term, it then searches for adjectives that describe the structure/nouns. In addition, IBID uses WordNet to find synonyms, hyponyms, and meronyms for each structural term identified. IBID performs this additional search to map the structural terms from the domain-specific vocabulary into our domain-independent structure vocabulary. IBID then annotates the article with all this structural information.

5 Search IBID is an interactive system intended to support complex human-AI interaction. Thus, it uses two kinds of search to locate biology articles in a corpus relevant to a design query: faceted search in which it uses domain-independent controlled vocabularies of structure, behavior and function terms; and search based on design queries stated as English language sentences. In the latter case, the current version of IBID does not yet use behavioral knowledge for locating biology articles. Faceted search is based on a controlled vocabulary for function, behavior and structure. The function facet’s controlled vocabulary has eight high-level function terms and multiple sub-level terms described in [25]. Given a designer’s selection of functional terms, IBID searches the functional annotations on articles for the

376

A. Goel et al.

selected verbs. The behavior facet’s controlled vocabulary is made up of a list of causal verbs. When a designer selects causal verbs of interest, IBID searches for articles possessing one or more behavior annotations with the selected verbs (or synonyms) as their foci. Cheong & Shu [27] describe a similar method for extracting causally-related functions. In IBID, the functional and behavioral terms derive from the SBF model and are also related to structural terms. Perhaps more importantly, these knowledge-based methods are able to extract only parts of the causal process. Thus, IBID seeks to combine them with machine learning techniques as described below. The structural facet’s controlled vocabulary consists of the domain-independent structural vocabulary we have developed. Recall that when IBID extracts structural terms from biology articles, the terms are domaindependent. In the current version of IBID, we manually map the domain-dependent structural annotations on the biology articles and the domain-independent terms in the faceted search. We intend to automate this process (and IBID already performs automated ontology alignment for natural language search). In addition to faceted search, IBID can search based on design queries expressed in English sentences. Consider the query: “I want to create a system for transporting liquid.”

IBID first identifies functional and structural terms in the design query (verbs and nouns, respectively) and lemmatizes/stems them. For the given input, IBID finds Function: [want, create, transport] and Structure: [system, liquid]. Next, IBID enlarges this query by adding domain-independent structure terms and high-level function terms. For the given input, this results in Function: [acquire, want, construct, create, move, transport] and Structure: [system, portion, liquid]. Finally, IBID uses the same mechanism as in its faceted search based on the structural and functional annotations on the biology articles.

6 Preliminary Testing of Structural Queries We have conducted preliminary evaluation of IBID’s ability to retrieve biology articles based on structural design queries [26]. We begin with a piece of text taken from a biology research article [24] that we have used for evaluating parts of IBID:

Towards a Virtual Librarian for Biologically Inspired Design

377

Bulk flow of water across a membrane occurs in response to an osmotic or hydrostatic gradient. Osmotic water permeability is readily measured in small vesicles or cells by the stopped-flow light-scattering technique, a method that relies on the dependence of light scattering on vesicle or cell volume, and is used to quantitate the time course of net water flow that occurs in response to transmembrane osmotic gradients. The osmotic gradients are established by adding an impermanent solute to the external solution. With the help of other chemical and physical methods to measure diffusional and osmotic water transport across biological membranes ...

We selected seven participants for our study, where the participants were not experts in biology (as with most biologically inspired designers). We asked the participants to list the structure terms in the above paragraph. Structure terms refer to the components, substances and connections of a system. For instance, in the following sentence: “Trees can transport water from the ground by their vascular system.”, “water”, and “vascular system” are the structure terms of the system “tree”. We gave exactly the same text and problem to IBID and compared the results with the human participants. We used the commonly used F1 metric for the comparison, as it captures both the fraction of relevant terms that were retrieved as well as the fraction of the retrieved terms that were relevant. We found F1 for identifying structural terms in the above experiment to be 79%. An interesting observation is the recall was higher than the precision, meaning that there were a larger number of false positives as compared to false negatives.

7 Machine Learning for Causal Relation Discrimination Current natural language processing techniques for extracting causal processes work only in limited contexts (e.g., [28]); a general AI technique for automatically extracting causal process from a natural language document is not yet known. Thus, IBID’s knowledge-based technique is able to extract only portions of the causal processes of a biological system from its natural language description. We posit that it would be useful to complement this top-down approach with a machine learning technique that performs bottom-up pre-processing on natural language documents to focus the knowledge-based extraction of causal behaviors on specific portions of a natural language document. To focus the text analysis process so that only potentially relevant parts of a biological document are analyzed deeply, we have prototyped a causal biological process discriminator. Since our definition of a biological process (BP) has

378

A. Goel et al.

causality as one of its key elements, this system aims to classify paragraphs in an article that describes a causal relation involving at least one biological entity. The working definition of a biological process for the algorithm has three main components: (i) a biological organism or an entity closely related to a biological entity, (ii) one or more causal relations relating to at least one entity from the previous list, and (iii) a function served by the previous causal relations.

7.1

Approach

Our approach for causal relations discrimination has four main components. First, we apply term frequency inverse document frequency (TF-IDF) on a biology article to pick up important entities within the document. The algorithm then passes the top candidates to a knowledge database to filter for the topic biological entity candidates. The entities are then combined with causal adverbs and causal patterns to form a causal chain within an article section, which is finally passed through to a classifier to produce a BP classification.

7.2

Biological Entity Detection

The first step of analyzing a biological document is picking out keywords that are important to an article. For example, an article that describes the properties of spider silk, some of these keywords may include spider, silk, fiber, spinneret, and strength. In order to accomplish this, the document is compared against a non-biological domain text corpus via TF-IDF, a popular weighting scheme for words used by search engines and document query tools. The term frequency portion of the metric measures how important a particular word is to the document, while the inverse document frequency portion is used to filter out pronouns and other common stop-words not of interest for the analysis. In our case, Reuters 21,578 text categorization text collection is used as a benchmark. The result of the TF-IDF process is a list of words ordered by their importance to the article. To differentiate words that are not just important to the document but also are related to biological entities, an outside knowledge source is required. This is accomplished by querying each word through DBpedia (https://wiki.dbpedia.org/), a database of structured content extracted from Wikipedia [29]. We leverage DBpedia’s keyword search functionality to specifically search for entities that match both the keyword found and belong to the query class species. This ensures that only organism-related entities are saved, even if the keyword in question is not an entity. For example, in the previous example spider would return results on species of spiders, silk would return both spiders and silkworms and is retrained as a biology related entity while strength would not return any result and is discarded.

Towards a Virtual Librarian for Biologically Inspired Design

7.3

379

Causal Knowledge Graph Construction

After the key biology related entities are found, the individual sections of a document are analyzed and converted into a knowledge graph before serving the graph’s key metrics as input to a classifier. To do this, each article paragraph is first tagged by their part of speech tags using the Python Natural Language Toolkit (NLTK; https://www.nltk.org/) library, then matched against a set of causal verbs and patterns as developed by Khoo et al. [30] for causality detection. The causal graph creation is accomplished in a two-step manner. During the first pass, for each causality match found in a sentence, the cause and effect noun phrases are extracted if they are also one of the biological entities found in the previous section. The entities are then extracted and stored in a knowledge graph, with an edge connecting the two. In the second pass, each additional causal pattern matched is added to the knowledge graph if at least one of the two entities is already in the graph. This is repeated until no more nodes are added to the graph after an entire pass through the paragraph. The rationale for the two-step approach is to ensure all entities in the graph have a path to a biological entity, reducing the chance of experiment-related and otherwise non biology related entities being included during graph construction. This can result in the creation of one or more causality graphs. Finally, the total number of nodes in the knowledge graphs, the number of biological and non-biological entities, the total number of edges, the total number of edges linking biological entities and the size of the largest graph are outputted. Because most classifiers cannot take a graph model as input, our feature set is engineered to contain the pertinent information about the causal relation in our knowledge graph. The features are used to both train a classifier with human annotated labels as well as to serve the role of input at prediction time.

7.4

Discriminator Testing

To test the performance of the discriminator algorithm, a collection of 25 articles was hand annotated (based on the definition of a biological process provided earlier as well as their perceived usefulness for a designer/engineer looking for inspiration from biology) on a paragraph level to be used as a testing/validation set. The articles were selected from the existing IBID article database, each describing either a study conducted on some biological entity or is a general-purpose scientific article describing a biological process or a species. The 421 total annotated paragraphs from them were used as both a training and validation set to test a wide range of classifiers using the Python Scikit-Learn library. Results using an 80/20 training/ validation set split and averaging all classifier results over 10 trials can be seen in Table 1. From the initial validation results, Gradient (Boosting) Trees were selected for testing on the testing set.

380

A. Goel et al.

Table 1 Classifier performance on validation data Classifier

Weighted accuracy

Precision

Recall

F-beta score

SVM Decision tree Gradient tree Random forest Logistic regression

72.8% 70.9% 86.4% 76.5% 69.1%

0.81 0.79 0.83 0.83 0.82

0.72 0.71 0.86 0.76 0.69

0.76 0.74 0.84 0.79 0.74

Fig. 2 ROC curve of algorithm performance on test data

As the paragraphs from both the training and validation data-sets came from the same set of articles, a new set of 5 articles was annotated in order to prevent the model from over-fitting. The new articles (130 paragraphs) were written by different authors and described different biological processes, which prevents the classifier from capturing the writing style of the training set authors and mimics the distribution of articles likely seen by an IBID-like tool in practice. The articles cover a range of topics, ranging from female frog calls to the vision system of cormorants. Over the new set of paragraphs, the classifier performed a little worse than it did on the previous test, achieving an accuracy of 81.5% with precision of 0.793, recall of 0.812 and F-beta Score of 0.777. To get a better understanding of classifier performance, the Receiver Operating Characteristic plot was generated to see the effect of varying the discrimination threshold, as we are more interested in lowering our rate of false negatives than false positives. From Fig. 2, we see that to increase our true positive rate from 0.6 to 0.8 would require a corresponding increase of false positives from 0.25 to 0.65, indicating the lowering of our classifier’s threshold may not be an effective method for reducing false negatives in our predictions. In order to further study the characteristics of the algorithm output, two additional papers were annotated, and classifier outputs studied in-depth. One of the most confident false positives was produced by the classifier on the following paragraph from [31]:

Towards a Virtual Librarian for Biologically Inspired Design

381

Despite the high abundance of mucus and mucus aggregates in coral reefs, their development over time in the reef water has not been investigated. An understanding of temporal changes in mucus aggregate composition is critical, because these aggregates potentially have an important function in the transfer of energy from the corals to other reef organisms, and in the trapping of organic matter and nutrients from water over passing the reef. The objectives of this study were (1) to assess whether dissolved mucus can cause mucus-particle aggregates, and (2) ...

Here the paragraph describes the central biological process investigated by the paper, that is, whether coral mucus can attract organic matters in the shallow coral reefs and form a key component of the nutrient cycle. The paragraph also contained key causal phrases such as cause, release, assess, and trapping as well as the key biological entities studied (coral, mucus, reef). However, the paragraph was not initially labelled as the description of a biological process because it does not talk about the production process of coral mucus and appears to be speculative in nature. On the other hand, the following is an excerpt from [32] of the most confident false negative paragraph predicted: The improvements in local buckling resistance under axial load or bending moment are mixed, with most species achieving some improvement in local buckling moment resistance (Fig. 8(b)). ... The short spines of the hedgehog (Erinaceus) and the spiny rat (Hemieehlnus) are required to act as shock absorbers as much as armour and protection to discourage predators, hence the high structural efficiency requirement and the need to delay local buckling until the internal stresses have almost reached ...

In this example, the biological process is described in a fleeting manner (spines acting as shock absorbers and armour). The causal relation is described in an indirect manner, (act as shock absorbers) instead of ‘spines absorb shock’ and ‘act … as much as amour and protection’ instead of ‘protect against predators’. Because the biological process is mentioned both briefly and in an indirect manner, it was not successfully picked up as a part of the knowledge graph and ultimately classified incorrectly with a very high confidence level.

382

A. Goel et al.

8 Discussion AI scientists have long dreamt of supporting human creativity, for example, creativity in scientific discovery and technical design. As early as the mid 1960s, just a decade after McCarthy and colleagues [33] first coined the term “artificial intelligence”, Feigenbaum and colleagues developed the DENDRAL knowledge system for abducing the structures of chemical molecules from mass spectroscopy data [34]. In the 1970s and the 1980s, knowledge systems such as R1 [35], AIR-CYL [36], PRIDE [37], VEXED [38] and VT [39] sought to capture design expertise in the form of design concepts, rules, constraints and plans. The hope at the time was that if AI systems could help capture large-scale expert knowledge, then the systems could support problem solving, design, discovery, and creativity at scale. However, as is well known now, experts at a given task in a given domain do not always agree, much of expert knowledge is tacit, it is difficult to elicit knowledge from experts, and it is also difficult to maintain it over time. In the second wave of AI research on supporting creativity in the 1980s and1990s, the focus expanded from knowledge to include experience. While expert knowledge may be tacit and difficult to elicit, the argument went, experts have external representations of their experiences: For example, most designers develop design briefs and many scientists keep research journals. If AI systems could capture these experiences in the form of case libraries, the systems could support problem solving, design, discovery, and creativity at scale. CYCLOPS [40], STRUPLES [41], ARGO [42], ARCHIE [43] and AskJef [44] were among the first case-based systems for supporting design creativity. While these interactive systems provided annotated digital libraries of design cases, they left the task of case adaptation to the designer. Although research in this paradigm continues, once again it has been difficult to acquire large libraries of well-documented cases, as well as difficult to maintain them over time. In this century, the notion of literature-based discovery has given rise to a third wave of AI research on supporting creativity. Literature-based discovery analyzes publicly available scientific literature to find connections among seemingly distant entities and analogies between seemingly different relationships and processes [45, 46]. Bruza & Weeber [47] compile an anthology of work on literature-based discovery; Henry & Mcinnes [48] provide a recent survey. The hope is that publicly available literature can ameliorate some of the difficulties of earlier AI attempts at creativity. In the context of AI research on creativity, Abgaz et al. [49] use natural language processing to find analogies between constructs in research papers on computer graphics, and Lavrac et al. [50] describe text mining techniques for detecting bridging concepts between seemingly unrelated terms such as migraine and magnesium. One important issue in AI research on literature-based discovery and design is the balance between the techniques of knowledge-based reasoning and statistical machine learning. Previous research has ranged from using mostly machine learning techniques [20, 50] for classification documents, to using mostly knowledge-based

Towards a Virtual Librarian for Biologically Inspired Design

383

techniques [17, 49] for analogical retrieval and mapping. As we described in the previous section, IBID combines knowledge-based reasoning and statistical machine learning: While IBID’s conceptual architecture is knowledge-based, it uses machine learning to accomplish specific tasks defined by its architecture. The results of the machine learning technique can then be processed by the knowledge-based method or by the human designer.

9 Conclusions IBID is an interactive AI system for helping biologically inspired designers locate and understand biology articles that describe biological systems relevant to a design query. Given a design problem, many designers typically search online for biology articles for inspiration. The analogical retrieval, mapping and transfer from biology to design often is mediated by functional models of the biological systems described in the articles. Thus, IBID first extracts structural, behavioral and functional terms in biology articles and annotates the articles with the terms. Then, given a design query, IBID locates the biology articles relevant to the query based on the articles’ annotations. IBID uses two kinds of search to locate biology articles: faceted search based on domain-independent controlled vocabularies of structures, behaviors and functions; and natural language query search for function and structure. The problem of extracting complex elements, such as the behaviors of a biological system in the form of casual processes, from natural language documents remains unsolved in AI. This problem had confounded our preliminary work on the IBID project. This work posits that addressing this problem requires a combination of knowledge-based and machine learning techniques. In general, techniques of statistical machine learning are computationally expensive, and require large amounts of labeled data but do not perform deep semantic analysis. A knowledge-based architecture for the overall task can help focus machine learning on specific subtasks thereby controlling the computational cost. The machine learning techniques can, then, make use of standardized datasets to produce preliminary results in the form of portions of natural language documents for further semantic analysis. Our experiments with the IBID system explore this approach. Thus, as described in this paper, IBID’s conceptual architecture spawns the subtask of behavior extraction and focuses the machine learning techniques to the behavioral processes. In addition, its SBF models provides the behavioral terms for searching the biology articles. It then uses machine learning techniques to classify specific portions of natural language documents that specify biological processes. While the machine learning techniques do not provide any guarantee of correctness, their output can be further analyzed for behavior extraction either by the knowledge-based methods or the designer.

384

A. Goel et al.

Acknowledgements We are grateful to Julian Vincent for sharing his domain-specific ontology for describing the structure of biological systems (Vincent [6]). We thank Swapnal Acharya, Kimisha Mody, Pablo Boserman, Daniel Diaz, and Ruth Petit-Bois for their contributions to this research.

References 1. Benyus J (1997) Biomimicry: innovation inspired by nature. William Morrow 2. Vincent J, Mann D (2002) Systematic technology transfer from biology to engineering. Phil Trans Roy Soc London A Math Phys Eng Sci 360(1791):159–173 3. Yen J, Helms M, Goel A, Tovey C, Weissburg M (2014) Adaptive evolution of teaching practices in biologically inspired design. In: Goel A, McAdams D, Stone R (eds) Biologically inspired design: computational methods and tools. Springer, London: pp 153–200 4. Vattam S, Goel A (2013) Seeking bioinspiration outline: a descriptive account. In: Proceedings 19th international conference on engineering design (ICED13), Seoul, Korea, August 2013, pp 517–526 5. Goel A, Rugaber S, Vattam S (2009) Structure, behavior and function models of complex systems: the structure-behavior-function modeling language. AIEDAM 23:23–35 6. Vincent J (2014) An ontology of biomimetics. In: Goel A, McAdams D, Stone R (eds) Biologically inspired design: computational methods and tools. Springer, London, pp 269–285 7. Goel A (2013) One thirty year long case study; fifteen principles: implications of an AI methodology for functional modeling. AIEDAM 27(3):203–215 8. Chandrasekaran B (1994) Functional representation: a brief historical perspective. Appl Artif Intell 8(2):173–197 9. Chandrasekaran B, Goel A, Iwasaki Y (1993) Functional representation as design rationale. IEEE Comput: 48–56 10. Gero J (1990) Design prototypes: a knowledge representation schema for design. AI Mag 11(4) 11. Umeda Y, Tomiyama T (1997) Functional reasoning in design. IEEE Expert 12(2):42–48 12. Helms M, Vattam S, Goel A (2010) The effects of functional modeling on understanding complex biological systems. In: Proceedings 2010 ASME conference on design theory and methods, Montreal, Canada, August 2010 13. Vattam S, Goel A (2011) Foraging for inspiration: understanding and supporting the information seeking practices of biologically inspired designers. In: Proceedings 2011 ASME DETC conference on design theory and methods, Washington DC, August 2011 14. Deldin J, Schuknecht M (2014) The AskNature database: enabling solutions in biomimetic design. In: Goel A, McAdams D, Stone R (eds) Biologically inspired design: computational methods and tools. Springer, London: pp 17–27 15. Goel A, Vattam S, Wiltgen B, Helms M (2012) Cognitive, collaborative, conceptual and creative - four characteristics of the next generation of knowledge-based CAD systems: a study in biologically inspired design. Comput Aided Des 44(10):879–900 16. Goel A, Zhang G, Wiltgen B, Zhang Y, Vattam S, Yen J (2015) On the benefits of digital libraries of analogical design: documentation, access, analysis and learning. AIEDAM 29(2) 17. Chiu I, Shu L (2007) Biomimetic design through natural language analysis to facilitate cross-domain information retrieval. AIEDAM 21(1):45–59 18. Shu L (2010) A Natural-language approach to biomimetic design. AIEDAM 24:507–519 19. Kruiper R, Vincent J, Chen-Burger J, Desmulliez M (2017) Towards identifying biological research articles in computer-aided biomimetics. In: Proceedings conference on biomimetic and biohybrid systems. Springer: 242–254 20. Vandevenne D, Verhaegen P, Dewulf S, Duflou J (2016) SEABIRD: scalable search for systematic biologically inspired design. AIEDAM 30(01):78–95

Towards a Virtual Librarian for Biologically Inspired Design

385

21. Fu K, Cagan J, Kotovesky K, Wood K (2013) Discovering structure in design databases through functional and surface based mapping. ASME J Mech Des 135 22. Mueller R et al (2018) Biodiversifying bioinspiration. Bioinspiration & Biomimetics 13(5) 23. Goel A et al (2016) Using watson for supporting design creativity. In: Proceedings fourth international conference on design creativity, Atlanta, Georgia, October 2016 24. Chrispeels M, Maurel C (1994) Aquaporins: the molecular basis of facilitated water movement through living plan cells? Plant Physiol 105(1):9–13 25. Rugaber S et al (2016) Knowledge extraction and annotation for cross-domain textual case-based reasoning in biologically inspired design. In: Proceedings 24th international conference in case-based reasoning (ICCBR 2016), Atlanta, USA, October 2016: pp 342–355 26. Goel A, Acharya S, Mody K, Hagopian K, Zhang S, Rugaber S (2018) Extracting structural knowledge from natural language documents to support biologically inspired design. In: Proceedings AAAI fall symposium on AI and natural systems. https://www.grc.nasa.gov/ vine/presentations-and-papers/ 27. Cheong H, Shu L (2014) Retrieving causally related functions from natural-language text for biomimetic design. ASME J Mech Des 136(8) 28. Berant J, Srikumar V, Chen P-C, Huang B, Manning C, Linden A, Harding B (2014) Modeling biological processes for reading comprehension. In: Proceedings 2014 conference on empirical methods in natural language processing (EMNLP 2014), pp 1499–1510 29. Bizer C, Lehmann J, Kobilarov G, Auer S, Becker C, Cyganiak R, Hellmann S (2009) DBpedia-a crystallization point for the web of data. Web Semantics Sci Serv Agents World Wide Web 7(3):154–165 30. Khoo C, Kornfilt J, Oddy R, Myaeng S-H (1998) Automatic extraction of cause-effect information from newspaper text without knowledge-based inferencing. Literary Linguist. Comput. 13(4):177–186 31. Huettel M, Wild C, Gonelli S (2006) Mucus trap in coral reefs: formation and temporal evolution of particle aggregates caused by coral mucus. Mar Ecol Prog Ser 307:69–84 32. Karam G, Gibson L (1994) Biomimicking of animal quills and plant stems: natural cylindrical shells with foam cores. Mater Sci Eng C 2(1–2):113–132 33. McCarthy J, Minsky M, Rochester N, Shannon C (1955) A proposal for a dartmouth summer project on artificial intelligence. www-formal.stanford.edu/jmc/history/dartmouth/dartmouth. html 34. Lindsay R, Buchanan B, Feigenbaum E, Lederberg J (1980) Applications of artificial intelligence for organic chemistry: the dendral project. McGraw-Hill Book Company 35. McDermott J (1982) R1: a rule-based configurer of computer systems. Artif Intell 19(1):39–88 36. Brown, D., & Chandrasekaran, B. (1989) Design problem solving: Knowledge structures and control strategies. San Mateo (Calif.): Morgan Kaufmann. 37. Mittal S, Dym C, Morjaria M (1986) PRIDE: an expert system for the design of paper handling systems. IEEE Comput 19(7):102–114 38. Steinberg L (1987) Design as refinement plus constraint propagation: the VEXED experience. In: Proceedings national conference on AI (AAAI-87), July 1987, Seattle 39. Marcus S, Stout J, McDermott J (1988) VT: an expert elevator designer that uses knowledge-based backtracking. AI Mag 9(1):95–111 40. Navinchandra D (1991) Exploration and Innovation in Design. Springer, New York 41. Zhao F, Maher M (1988) Using analogical reasoning to design buildings. Eng Comput 4:107–122 42. Huhns M, Acosta E (1988) Argo: a system for design by analogy. IEEE Expert 3(3):53–68 43. Barber J et al (1992) AskJef: integration of case-based and multimedia technologies for interface design support. In: Proceedings second international conference on artificial intelligence in design. Dordrecht: Kluwer: p 457–74 44. Pearce M, Goel A, Kolodner J, Zimring C, Sentosa L, Billington R (1992) Case-based decision support: a case study in architectural design. IEEE Expert 7(5):14–20

386

A. Goel et al.

45. Swanson D (1986) (1986) Undiscovered public knowledge. Libr Q 56(2):103–118 46. Swanson D (2008) Literature-based discovery? The very idea. In: Bruza P, Weeber M (eds) Literature-Based Discovery. Springer 47. Bruza P, Weeber M (eds) (2008) Literature-Based Discovery, Springer 48. Henry S, Mcinnes B (2017) Literature-based discovery: models, methods, and trends. J Biomed Inf 74 49. Abgaz Y, Chaudhry E, O’Donoghue D, Hurley D, Zhang J (2017) Characteristics of pro-c analogies and blends between research publications. In: Proceedings 8th international conference on computational creativity, pp 1–8 50. Lavrac N (2019) Bisociative knowledge discovery for cross-domain literature mining. In: Veale T, Cardoso A (eds) Computational Creativity. Springer

An Investigation into the Cognitive and Spatial Markers of Creativity and Efficiency in Architectural Design Kinda Al Sayed and Alan Penn

Abstract This paper presents a preliminary study into the spatial features that can be used to distinguish creativity and efficiency in design layouts, and the distinct pattern of cognitive activity that is associated with creative design. In a design experiment, a group of 12 architects were handed a design brief. Their drawing activity was recorded and they were required to externalize their thoughts during the design process. Both design solutions and verbal comments were analyzed and modeled. A separate group of mature architects used their expert knowledge to assign creativity and efficiency scores to the 12 design solutions. The design solutions were evaluated spatially. Protocol analysis studies including linkography and macroscopic analysis were used to discern distinctive patterns in the design processes that are marked with the highest and least creativity scores. Through this investigation, we suggest that expert knowledge can be used to assess creativity and efficiency in designs. Our findings indicate that efficient layouts have distinct spatial features, and that a cognitive activity that yields a highly creative outcome corresponds to higher frequencies of design actions and higher linkages between design moves. These linkages build up from local sequential design decisions to global design decisions linking the problem definition stage to the solution definition stage.

1 Significance At essence, architectural design is a creative activity. It is creative in the sense that design is a search for satisficing solutions that minimize conflicts between different design requirements [1]. In defining creativity, Bo-den differentiates between psychological creativity (P-creativity) and historical creativity (H-creativity) [2]. K. Al Sayed (&) University of Sussex, Brighton, UK e-mail: [email protected] A. Penn University College London, London, UK e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. S. Gero (ed.), Design Computing and Cognition'20, https://doi.org/10.1007/978-3-030-90625-2_22

387

388

K. Al Sayed and A. Penn

P-creativity is related to the designer’s frame of reference during the process of design. Yet, what is new to a designer might not be necessarily genuinely new to the design community. From that perspective, H-creativity depends on the frame of reference of a professional community through evaluating the historical discourse in the design domain of knowledge. The definition of H-creativity could be quantitatively defined through comparing generated spatial layouts to a large set of architectural styles [3], although such definition remains to be reductionist, as it does not attend to other qualities such as building material, the vertical dimension, the style of construction, and the finer grained description of designed layouts. It can also be argued that architects would have accumulated knowledge of architectural styles throughout their education and architectural practice, hence they posses tacit knowledge of the history of design progress. Designers’ expert knowledge can be used to assess how creative designs are compared to past designs. In linking a designer’s frame of reference with the community’s frame of reference, creativity attains social value. So there is a margin of subjectivity in designers’ expert judgment that can be reduced through attainting a level of agreement. When evaluating creativity, there are non-trivial challenges in defining the frame of reference, particularly in what concerns the quality or value that is being assessed; is it purely aesthetical or does it have to do with the mechanisms of operation in an artifact? In architecture, the manner in which building functions are programmed is an element of creativity. Yet, efficiency in the spatial distribution of functions is not necessarily correspondent to higher levels of creativity. It is probably difficult to define a set of benchmarks to evaluate creativity and efficiency in design solutions, considering that there is a large universe of design solutions for every architectural design problem. Creative designs could belong to the larger universe of probable designs, but efficient designs would belong to a smaller cluster of possible designs where the performance of building function is highly optimized. A design solution can be considered as an emergent product of a set of local actions that respond to problems both locally and globally. If we are to assume that the value of creativity is embedded in expert knowledge, and is difficult to quantify as a whole, then perhaps it is legitimate to ask expert designers to assign scores to designs measuring on how creative and efficient they appear to be. This paper builds on previous research [4, 5], by attempting to discern distinct spatial features that characterize creativity and efficiency in design solutions. Creativity and efficiency are assigned as scores to design solutions by a committee of expert architects. The design solutions are evaluated spatially to look for any correspondences between the scores assigned and the distribution and size of spaces in the designed layouts. Through externalizing and modeling designers’ cognitive activity during the process of design, we also attempt to discern differences in the frequencies and structural linkages between design actions/moves that lead to creative design outcomes [6, 7]. Due to technical and data limitations it is difficult to observe P-creativity, where episodes of creative activity could be correspondent to generating new ideas over the course of the design process. Rather we compare the cognitive activities in the two design processes that have led to designs marked with

An Investigation into the Cognitive and Spatial Markers …

389

the highest and least creativity scores. The assumption is that the cognitive activity that leads to a highly creative design solution can be distinguished from the cognitive activity that leads to the least creative design solution. We do not attend to efficiency in the design process itself, although few metrics of design process efficiency are discussed, including design productivity as termed by Goldschmidt [7], and the overall duration of design process.

2 Method In order to investigate whether creativity can leave traceable patterns or markers in the design outcomes and in the process of design, this paper will use a range of methods to; quantify and analyze design solutions, and represent and analyze cognitive activity during design processes (macroscopic analysis of verbal protocols and linkographs). A design task will be presented to a group of architects. The architects will be required to solve a well-defined design problem. A separate committee of expert designers will assess the creativity of design solutions. The tessellation in the spatial grid representing the design outcomes will be analyzed. Cognitive analysis will be applied to the design processes to find distinct patterns that differentiate between the most and least creative design solutions.

2.1

A Description of the Design Experiment

In this paper, 12 design cases -previously studied by Al_Sayed et al. [4, 5]—are reintroduced. Architects were asked to think aloud whilst designing and to develop design solutions intuitively using sketches within a duration of 15 min. A video camera recorded the drawing process and a microphone recorded the architect’s verbal expressions whilst describing his/her thoughts during the design process. The verbal comments were later transcribed in order to use them in protocol analysis. The protocol analysis considered semantic expressions without including physical acts. The design brief was limited to a set of functional spaces that form the basic requirements for an architect’s office. Accounting for the idea that architectural design problems are ill-defined [4, 5], the scope was to limit the variation on how the brief might be interpreted. The program that sets the narratives for the relationships between the functions listed in the brief is likely to have impact on the spatial attributes of the design outcome. In order to simplify the design task, architects were required to allocate the predefined functions in a given layout (see Table 1). The layout itself was restricted to one level rather than multi-levels. The layout was a hypothetical rectangular floor plan in a skyscraper with two access points from two cores [8]. There are some challenging problems with regards to the layout settings and its massive size, the number and pattern of columns, and the two cores that link it with the external environment.

390

K. Al Sayed and A. Penn

Table 1 Design task including a brief for an architect’s office, and an existing layout (the layout was cited in Shpuza [8]).

The design brief

Design task layout.

Head office and private secretary space Waiting area with small exhibition Two meeting rooms Management offices; (number: 3-4). Telecommunication offices (number: 2). Three spaces for consultants Five spaces for design teams. Two IT offices Two technical studies units One construction expertise unit Two service areas with kitchenette.

2.2

Modeling Cognitive Activity

In this part of the study, we attempt to establish a link between creative design solutions—as defined by expert knowledge—and characteristic pat-terns in the macroscopic analysis of design protocols and linkographs. The macroscopic analysis of verbal protocols is a content-based method that was proposed by Suwa, Purcell and Gero [6] to analyze design activity. In Suwa et al., the design process is segmented using protocol analysis of physical actions and semantic expressions. Considering the scope of our research, physical actions (e.g. hand gestures) were ignored, but semantic expressions were recorded during the design process. The semantic expressions were segmented into design actions following Suwa et al.’s model of categorizations (Table 2). Their description separates physical, perceptual, functional, and conceptual cognitive actions, and they provide detailed subcategories. For the purpose of our study, Suwa et al.’s model of categorization was applied only partially. The only physical action that was taken into consideration was (L-action), which represents the state when designers look at previous depictions and refer to them semantically. Perceptual, functional and conceptual actions will be fully considered as long as the subjects verbally express them. Perceptual actions (P-action) will be recorded whenever the architect refers to visual features or spatial relations. Functional actions (F-action) apply when an architect considers interactions between artifacts and people/nature, and account for the psychological reactions of people. Conceptual actions may occur during the process of knowledge retrieval (K-action), or whenever an architect makes preferential and aesthetical evaluations (E-action), or when an architect defines a goal (G-action). The segmentation model decodes every segment in relation to a corresponding reference. For instance, talking about cores defines one segment, whilst talking about design teams defines another segment. Further detailed segmentations refer to

An Investigation into the Cognitive and Spatial Markers …

391

Table 2 An adapted description of macroscopic design segmentations Suwa et al. [6], PP460 Category

Names

Description

Examples

Physical Perceptual

L-action P-action

Look at previous depictions Attend to visual features of elements Attend to spatial relations among elements Organize or compare elements

– Shapes, sizes, textures

Functional

F-action

Conceptual

E-action G-action K-action

Explore the issues of interactions between artifacts and people/ nature Consider psychological reactions of people Make preferential and aesthetic evaluations Set up goals Retrieve knowledge

Proximity, alignment, intersection Grouping, similarity, contrast Functions, circulation of people, views, lighting conditions Fascination, motivation, cheerfulness Like-dislike, good-bad, beautiful-ugly – –

different cognitive actions as defined by (Table 2). The resulting string of data has a time dimension (the number of design actions in 30 s units of time) as demonstrated in Fig. 1. In a linkograph model proposed by Goldschmidt [7], the cognitive activity is recorded, segmented and rebuilt into a relational structure that links design moves by matching their semantic meaning. The linkograph’s protocol is segmented into a set of ‘design moves’ with directed links. A ‘design move’ is explained as ‘an act of reasoning that presents a coherent proposition pertaining to an entity that is being designed’ [7]. Links among moves are determined arbitrarily by the observer, and are notated in a network. The design process is interpreted as a pattern of linked moves that comprise the graphic network of the linkograph. Goldschmidt identified two types of directed links: links connecting to preceding moves—‘backlinks’; and

Fig. 1 Segmenting semantic expressions into cognitive actions [4, 5]

392

K. Al Sayed and A. Penn

Fig. 2 A model of the linkograph’s segmentation scheme

links connecting to subsequent moves—‘forelinks’. Moves that have dense linkage connections; namely critical moves (CM) can be considered as indicators for design productivity. An example of a linkograph is represented in Fig. 2, where the transcribed verbal comments are segmented into design moves (moves 1–12). Design moves were linked by nodes whenever they exhibited some association in terms of content. In the original scheme of a linkograph, Goldschmidt referred to four main types. In Case 1, design moves are completely unrelated, indicating low potentials for idea development. In Case 2, design moves are completely interconnected, hinting to a fully integrated process in which successive ideas may suffer from fixation and lack of diversity; this leaves fewer chances for novel ideas. In Case 3, each design move is linked only to its subsequent move; this signifies a progression in the process with not much development in terms of ideas. In Case 4, design moves are partly interrelated, indicating a productive design process that provides plenty of opportunities for idea generation and development. In order to highlight differences in nodes’ clustering, a Nonparametric Density Estimation (NDE) feature was used to distinguish patterns in the nodes’ point density [9]. The bivariate density estimation projects a smooth surface that describes the density of nodes in a linkograph at each point in that surface. The nodes are mapped in a two dimensional space, and a set of contour lines are set at quantiles in 5% intervals. The contours are rendered to show the density of nodes in a linkograph. This means that 5% of the nodes are below the lowest contour, 10% are below the next contour, and so on. The highest contour has about 95% of the points representing the nodes below it indicating to clusters that contain the highest concentration of nodes within the contour boundary. These clusters may represent moments of ‘fixation’ in the cognitive activity where architects tend to focus on solving certain problems. The nonparametric density method is computed by dividing each axis into a fixed number of binning intervals. The number of points is then counted in each bin. Following that, a smoothing kernel standard deviation is

An Investigation into the Cognitive and Spatial Markers …

393

set. A bivariate normal kernel smoother is applied using a fast Fourier transform (FFT) algorithm and inverse FFT to do the convolution. Following this procedure, a contour map is created using a bilinear surface patch model. This method is explained in Rodriguez and Stokes [10] and applied in SAS software. In this paper, the Kernel Standard Deviation was set to 6 to enable a comparison between all linkographs. A statistical representation of clustering in the node densities was favored over a structural description of the linkographs [11, 12]. The latter was thought to present a wide range of variation in the structure subject to the representation of design moves.

3 Results The main criterions used to evaluate design solutions are creativity and efficiency. Six MSc SDAC students (raters)1 were to assess a set of design proposals2 for an architectural practice in terms of ‘creativity’ and ‘efficiency’. The judgment is based on their ‘expert knowledge’ as architects. Unfortunately, the ratings assigned to the layouts varied in their level of agreement. Based on measures associated with the Consensual Assessment Technique (CAT), a technique for measuring agreement between raters on assessing creative products [13], the overall Kappa value (produced in JMP statistical software) was slightly higher than 0, indicating an agreement between raters for a given layout that is slightly higher than chance (0.07 for creativity and 0.04 for efficiency). The rater’s agreement with him or herself and the other raters for a given layout varied between (11% and 20% in measuring creativity scores) and between (6% and 13% in measuring efficiency scores). The average creativity scores assigned to the layouts yielded the proposal (layout 7) made by [AB] as the most creative design proposal, marking the highest average ‘creativity’ score (C-score), whilst the design proposal (layout 2) made by [KS] was reported as the least creative. Average efficiency scores (E-score) yielded layout 3 as the most efficient design proposal, whilst layout 7 designed by [AB] was reported as the least efficient (Table 3). It is evidently difficult to establish what makes efficiency in the designers’ judgment. One physical metric might be the area of circulation compared to the area of the overall layout. A more efficient layout will render out as the layout that minimizes circulation area whilst connecting all spaces. This does not count the circulation area that is presumed to be enabling movement between desks within larger spaces. So the definition of a circulation space in this case is limited to those spaces that are physically defined as corridors or lobby 1

MSc Spatial Design: Architecture and Cities (SDAC), 2015/16 cohort, at the Bartlett School of Architecture, University College London. 2 The design proposals belong to a case study that was presented in Al_Sayed et al. [4]. A detailed description on the terms of the experiment, subjects, and data used and generated by the experiment is available in; https://discovery.ucl.ac.uk/id/eprint/4928/1/4928.pdf. The original layout belongs to Weyerhaeuser Company SOM - Sidney Rodgers & Associates Tacoma, WA, USA.

394

K. Al Sayed and A. Penn

Table 3 Average ‘creativity’ scores (C-scores) and ‘efficiency’ scores (E-scores) based on raters’ expert knowledge. The scores (1–12) are averaged based on 6 observations; where higher scores indicated lower creativity or efficiency. STD and Kappa values are included to show variability and agreement levels respectively. Ratio of circulation to layout area is included to inspect its relation to efficiency.

cscores STD Kappa escores STD Kappa Ratio of circulation

L1 5.33

L2 10.5

L3 7.00

L4 7.17

L5 6.83

L6 9.17

L7 2.00

L8 7.50

L9 5.50

L10 5.50

L11 5.33

L12 5.67

3.64 0.63 7.83

2.14 0.12 6.33

2.77 0.20 3.50

2.11 0.01 7.00

2.61 0 8.00

2.67 0 6.17

2.24 0 11.33

3.15 0 6.50

3.20 0.05 4.83

2.69 0.01 6.17

2.75 0 6.17

2.75 0.1 6.63

4.52 0.05 19%

2.43 0.01 17%

2.14 0.01 10%

2.71 0 13%

1.91 0.12 23%

3.02 0.20 11%

1.11 0 26%

3.69 0 24%

1.46 0 27%

4.34 0 16%

1.21 0.01 13%

3.44 0.34 12%

areas. The ratio between circulation spaces to overall layout area does correspond in some cases to the efficiency scores—as marked by expert raters. Layout 3, marked with the highest efficiency score has the least circulation area, and layout 7, marked with the lowest efficiency score, has one of the largest circulation areas. These distinctions do not apply to layouts 4, 6, 11, and 12, all appearing to have smaller circulation ratios, and are marked as average in terms of efficiency. Some layouts have a large number of small spaces and few large spaces (Fig. 3). The ratio of this distribution differs following variations on the design solutions. Measuring on the scores assigned and the distribution of space size in the designed layouts, it is difficult to establish whether higher creativity scores are related to the distribution of space size. Generally, there are no sharp distinctions in the density of smaller spaces up to 100 grid points and density of larger spaces above 100 grid points in relation to creativity and efficiency scores. Layout 3 scored as the most efficient appears to have a more regular pattern of change in the distribution of space size, with a large density of smaller spaces under 100 grid points, a smaller number of larger spaces between 100 and 150 grid points, and finally two clusters of larger spaces peaking at 230 and 290 grid points. Overall, it is difficult to make the case that we can recognize creativity or efficiency from the distribution of room/ space size in the designed layouts. There is no linear correlation between larger areas allocated for circulation and the degree of tessellation (number of spaces allocated for different functions in each layout). Instead, data seems to cluster in two groups (Fig. 4). Layouts (3, 4, 6) have smaller circulation areas and lower degree of tessellation. The rest of the layouts

An Investigation into the Cognitive and Spatial Markers …

500

100

300

500

500

100

300

500

0.020 Density 100

300

500

0

Layout 10

Layout 11

100

300

500

100

300

500

N = 44 Bandwidth = 20.48

Layout 12

Density

0.020

0.020

0

N = 44 Bandwidth = 17.33

0.010 0.000

0

N = 44 Bandwidth = 14.18

0.010

300

Density

Density

0

N = 44 Bandwidth = 13.55

0.010

Density 100

0.020

0.020

Layout 9 Density

0

N = 44 Bandwidth = 14.34

0.000 0

N = 44 Bandwidth = 10.71

0.000

500

0.010

300

Layout 6

0.020

0.020 100

0.000

0.000

300

0.000 0

N = 44 Bandwidth = 13.71

0.020 Density

0.020 0.010 0.000

100

Layout 5

0.000

500

0.010

300

Layout 8

0

Density

Density 100

Layout 7

N = 44 Bandwidth = 9.768

0.010 0.000

0

N = 44 Bandwidth = 9.217

0.000

500

0.010

300

Layout 4

0.020

0.020 Density

0.010 0.000

100

N = 44 Bandwidth = 13.55

0.010

Density

0.010 0.000 0

Density

Layout 3 0.010

Layout 2

0.020

Layout 1

395

0

100

300

500

N = 44 Bandwidth = 9.217

0

100

300

500

N = 44 Bandwidth = 9.453

Fig. 3 Distribution of areas defined by functions in the designed layouts. Areas measured by number of grid units per layout. Grid unit equals 1.4375 m
1.4375 m

Circulaon area

600

6.6365x + 167.88 R² = 0.13747

500 400 300 200 100 0

0

10

20

30

40

Tesselaon

Fig. 4 Correlation between circulation area and tessellation

have a range of circulation area between (200 and 500 grid units) and higher degree of tessellation. The circulation areas are those that are physically defined in the drawing, excluding circulation areas that are nested within larger rooms in the layout.

3.1

Modeling the Protocols of the Most and Least Creative Designs

In this section, we analyzed the design processes that resulted with the most and least creative designs using macroscopic analysis and linkography. The macroscopic analysis showed higher frequencies of perceptual, functional and aesthetically–driven actions in AB’s design process compared to KS (Fig. 5), despite the fact that the duration of both design processes were very close -AB consumed 38 min, whereas KS consumed 32 min. This suggests that a highly creative design

396

K. Al Sayed and A. Penn

Fig. 5 Protocol analyses of the most creative design process by [AB] and the least creative design process by [KS], based on Suwa et al.’s [6] categorizations

Fig. 6 Nonparametric Density Estimation of linkographs representing the design process by AB and KS. The X axis represents the sequential progress of design moves over the period of the design session. Diagrams produced using JMP, The Statistical Discovery Software, Version 5.1

is a product of a cognitive activity with higher frequencies of cognitive actions. The linkography analysis showed remarkable differences between AB and KS (Fig. 6). When setting the nonparametric density estimation models to similar kernel standard deviation levels, AB’s linkograph showed a larger number of clusters than KS’s linkograph. The clusters in AB’s case are distributed at different levels; one aligning the horizontal axis linking sequential design moves, one in the middle connecting problem-definition, drawing activity and solution-definition stages, and a cluster at the top of the linkograph linking problem-definition and solutiondefinition stages. KS’s linkograph showed a large cluster at the problem-definition stage, and a cluster connecting drawing actions and the solution-definition stage. Number of design moves, number of critical design moves (>8 links), and number

An Investigation into the Cognitive and Spatial Markers …

397

of original ideas pronounced verbally by AB are more than double the ones in KS’s linkograph. The link index is relatively higher in AB’s linkograph (2.82) compared to KS’s linkograph (2.6). Goldschmidt [7] had previously found a correlation between design productivity and link index (ratio of links/moves). This indicates that higher productivity during design may yield higher creativity in design outcomes. This finding needs to be generalized on a larger population before confirming it true.

4 Conclusions This paper reports an investigation into the markers that distinguish creativity in design protocols, and creativity and efficiency in design solutions. Creativity and efficiency in designs are assessed based on expert knowledge. The designed layouts are analyzed spatially to distinguish features that are associated with creative and efficient designs. A committee of experts was asked to assign creativity and efficiency scores to the designs. The verbal protocols of designers were modeled to check how the cognitive activities of architects who design highly creative solutions differ from those that design the least creative solutions. The spatial distribution of spaces in the designed layouts did not show considerable differences in size regardless of the scores assigned. It was possible to distinguish a relationship between efficiency and the ratio of circulation to layout area. Highly efficient designs had a smaller circulation area compared with least efficient designs. The analysis of cognitive protocols yields distinctive patterns that characterize the design processes that lead to creative designs. A creative design appears to be an outcome of a process that has higher frequencies of cognitive actions in macroscopic analysis and higher ratio of linkages between design moves in linkographs. Moreover, a design process that yields creative outcome shows systematic pattern of clustering that builds up hierarchically from the local scale of sequential design moves to the global scale, linking the problem-definition stage, the drawing activity stage and the solution-definition stage in a linkograph. These findings remain to be experimental. They are subject to designers’ interpretation of what makes a creative and efficient design solution. The numbers of cases to compare are also very limited, and the circumstances underlying the original experiment—which was intended to compare two groups of architects with different types of expertise- may have influenced the dataset and the results of the analysis. Future studies will re-examine the methods of assessment by introducing more robust settings and metrics of evaluation to the case study including Creative Product Semantic Scale (CPSS) and Consensual Assessment Technique (CAT) [13] methods to support the judgment criteria and measures set by the committee of experts.

398

K. Al Sayed and A. Penn

References 1. Simon H (1957) Models of man: social and rational. John Wiley, London 2. Boden M (1990) The creative mind: myths and mechanisms. Weidenfeld and Niicolson, London 3. Hanna S (2007) Automated representation of style by feature space archetypes: distinguishing spatial styles from generative rules. Int J Archit Comput 5(1):1–23 4. Al-Sayed K, Dalton RC, Hölscher C (2008) Discursive and non-discursive design processes. In: Gero JS, Goel AK (eds) Design computing and cognition 2008. Springer, Dordrecht, pp 635–654 5. Al-Sayed K, Dalton RC, Hölscher C (2010) Discursive design thinking: the role of explicit knowledge in creative architectural design reasoning. AI EDAM 24(2):211–230 6. Suwa M, Purcell T, Gero J (1998) Macroscopic analysis of design processes based on a scheme for coding designer’s cognitive actions. Des Stud 19(4):455–483 7. Goldschmidt G (1992) Criteria for design evaluation: a process-oriented paradigm. In: Kalay YE (ed) Evaluating and predicting design performance. Wiley, New York, pp 67–79 8. Shpuza E (2006) Floorplate shapes and office layouts: a model of the effect of floorplate shape on circulation integration. Doctoral dissertation, Georgia Institute of Technology 9. Kan WT, Gero JS (2008) Acquiring information from linkography in protocol studies of designing. Des Stud 29(4):315–337 10. Rodriguez RN, Stokes ME (1998) Recent enhancements and new directions in SAS/STAT software, Part II. Nonparametric modeling procedures. In: Proceedings of the 23rd SAS users group international conference, pp 1262–1270 11. Gong Y, Xu S, Zhang S (2009) Structure of the design thinking network. In: IEEE 10th international conference on computer-aided industrial design & conceptual design, CAID & CD 2009. IEEE, pp 531–535 12. El-Khouly T, Penn A (2012) Order, structure and disorder in space syntax and linkography: intelligibility, entropy, and complexity measures. In: Proceedings of 8th Space Syntax Symposium, Santiago, De Chile, p 8242 13. Lee JH, Gu N, Sherratt S (2011). Developing a framework for evaluating creativity in parametric design. From principles to practice in architectural science-ANZAScA2011. The University of Sydney, Abstracts Book, 17

Conversational Co-creativity with Deep Reinforcement Learning Agent in Kitchen Layout Poyen Hsieh, Deborah Benros, and Timur Dogan

Abstract Architects traditionally take decades to train on both reasoning and sensibility, yet machines can now master the former. The functional goals are often more quantifiable and standardized, therefore, perfect to automate. This paper frames the layout design problem as a reinforcement learning problem to train a virtual design assistant that gives step-by-step suggestions based on a partial solution. The dialogue between the user and the assistant are interdependent, thus called conversational co-creativity. There are three stages to build the system: building an environment, collecting data, and training a deep neural network. Kitchen layout is chosen as the subject for its richness in functional requirements. As a result, the trained assistant successfully finds optimal and sub-optimal solutions in the test maps with only a handful of searches. The application of the proposed framework is potentially broader.

1 Introduction In architectural design, there are countless conditioning factors. It has become harder for architects to satisfy all the existing standards and be innovative at the same time. In the past two decades, automation has been gaining traction. From merely academic research to corporations’ endeavors, it may open the door to better architecture [1]. Different approaches to automation have their pros and cons. A rule-based system is reliable when generating designs, but requires expert domain knowledge, thus, are not flexible for re-adaptation of the code [2, 3]. Generative adversarial networks P. Hsieh (&)
D. Benros
T. Dogan Cornell University, Ithaca, NY, USA e-mail: [email protected] D. Benros e-mail: [email protected] T. Dogan e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. S. Gero (ed.), Design Computing and Cognition'20, https://doi.org/10.1007/978-3-030-90625-2_23

399

400

P. Hsieh et al.

(GANs) for floor plan generation make good use of the existing image data [4], but is confined by human’s wit and operates like a ‘black box,’ Generative design is capable of finding novel solutions that were previously unreachable by a human, but the computation is relatively slow when the solution space is large. This paper was inspired by the recent breakthrough in deep reinforcement learning, Alpha Zero defeating world champion programs in chess, shogi, and Go [5]. The paper re-frames a layout design problem as a board game problem. It runs on the core concepts of reinforcement learning, namely the self-play mechanism and the reward system, with additional components from the automation approaches mentioned above. The rules found in the rule-based system become the rules of the game, and the evaluation metrics found in the generative design method constitute the reward system. The evaluation metrics reduce the required amount of expert knowledge, and offers a way to directly teach a virtual agent, unlocking the ‘black box.‘ In addition, the self-play mechanism, with the help of the metrics, can expand the solution pool beyond human capability. Finally, the deep neural network learns from the massive amount of data and can predict the best movement in a relatively short time. The goal of automation should be to reduce the ever-increasing burden on the architects’ shoulders. Instead of replacing the architect, automation should allow them to focus on what the machines cannot do. The sensibility that humans have, which is to perceive and feel as a user in space, might never be replaced. Therefore, automation should first focus on the quantifiable aspects of design as they are what machines are good at, as well as the standardized goals for that they are widely accepted. For the above reasons, the kitchen layout design is chosen as the subject as it is rich in functional requirements that are more quantifiable and standardized.

2 Literature Review and Background The scope of the paper covers areas of design automation, human–computer co-creativity, deep reinforcement learning, and kitchen design.

2.1

Design Automation

Previous studies in design automation have mostly focused on expert systems, namely rule-based systems that follow the rules to solve problems. Shape grammar [6] is one of its branches and has been used to capture the interaction of spatial elements and reproduce architectural design styles [7, 8]. In practice, rule-based systems are now used by companies to generate floor plans for businesses, architects, developers, and consumers [9–12]. Generative design utilizes a rule-based system with parameters that can be optimized by a genetic algorithm based on specified goals, exploring a more extensive design space than manually could [13]. This approach has also been used in layout planning [14].

Conversational Co-creativity with Deep Reinforcement Learning …

401

In contrast, a study showed that a generative adversarial network (GAN) could skip the distillation of design rules entirely and learn to generate design directly from labeled images of apartment floor plans [4].

2.2

Human–Computer Co-creativity

Human–computer co-creativity is the process of cooperative problem solving between designers and computers. It can start even before the design begins. A study used IBM’s Watson to make the acquisition of the necessary knowledge for a design problem more intuitive and effective [15]. On the other hand, in the design process, critics are intelligent support systems that can detect and criticize partial solutions constructed by the designer based on knowledge of design principles. A critic system has been used to support co-creativity by telling the users what looks wrong and display criticism with knowledgeable arguments [16]. Such critics have been shown to be an effective mechanism for providing feedback to users [17]. Other studies have worked on design suggestions. One ran on specificationlinking rules, for example, suggesting to use a single-bowl-sink after the user specifies the design is a single person household [18]. Another tried to learn the scheme of a partial solution and generate complete solutions that meet preset goals for users to choose in the game design [19]. And another focused on misinterpreting what the user has drawn on a canvas, with an algorithm responding with its own strokes, stimulating creativity by suggesting a misinterpreted scheme in the sketching activity [20]. Our system sports a set of evaluation metrics that works like a critic system and provide directly actionable suggestions down to the level of a single design step. The Google Smart Compose bears great resemblance to this approach. It offers sentence completion suggestions as user types, and whether to accept them is entirely up to the users [21]. This approach contrast to the more conventional ‘solutions-to-select-from’ scenarios where a complete solution is presented, a human user only needs to select and perhaps edit. The comparable example in text generation would be OpenAI’s GPT-2, which generates coherent paragraphs with random seeds from some starter text [22]. Being able to participate in every step of the process implies the ability to steer directions for the end solution. We argue that with a conversational assistant like ours, a designer can be perceived as the true author of the design, only enhanced by the computational reasoning of the assistant.

2.3

Deep Reinforcement Learning

Reinforcement Learning (RL) is a paradigm of machine learning. It focuses on rewarding positive actions and punishing negative actions that an agent takes in an

402

P. Hsieh et al.

environment [23]. Combining it with Deep Learning (DL) technique, DeepMind’s Alpha Zero is a deep reinforcement learning (DRL) agent that needs only the basic rules of chess, shogi, and Go to achieve superhuman performance [5]. DRL is the key to enabling conversational co-creativity workflow as it is the brain of our system.

2.4

Kitchen Design

Kitchen design has gone through more of aesthetic and technological innovation than any other room in the development of modern dwelling. The Frankfurt Kitchen, the step-saving kitchen, and the Cornell kitchen are well-known examples that pushed for rationalization, efficiency, and modularization in kitchen design [24–26]. Kitchen-planning tools were once physical miniatures in 1950 [27]. It has evolved into the digital world and equipped with an actively supportive design knowledge system [18]. In recent years, modularized kitchen planning and visualization tools are widely available online. Some provide minimum tips and advice [28].

3 Methodologies There are two participants in the proposed framework, a system builder and a system user (see Fig. 1). The former builds a virtual environment with a tree search agent, some formal rules, and evaluation metrics. The agent explores the solution space and collects data to train a deep neural network. This neural network then becomes the brain of a virtual assistant who gives action suggestions. The system user interacts with the assistant by accepting the suggestions or executing their own moves in the conversational co-creativity workflow.

3.1

Building the Assistant: Three Steps

The methodology to produce a conversational kitchen design assistant consists of three main steps: (1) Build an environment. (2) Collect data with the self-play mechanism. (3) Train the neural network.

3.1.1

Environment: Board, Rules, Actions, and Evaluation Metrics

The board represents the state of the kitchen design environment. It is a 15’ by 15’ square matrix with numbers representing different elements: (−4) door, (−3) window, (−2) wall, (−1) floor, (0) legal cell, (1) fridge, (2) pantry, (3) sink, (4) dishwasher, (5) stove, (10) legal cell for only short appliances, (11) mark to the living

Conversational Co-creativity with Deep Reinforcement Learning …

403

room, (12) mark to the dining room. The beginning state of an episode, a map, is filled with doors, windows, walls, floors, legal tiles, and marks. Thirty-four handcrafted maps are used for training including U-shaped, L-shaped, galley, and strip kitchen with or without island, or peninsula, and five test maps with variations and different features for performance validation. Theoretically, an appliance can be placed anywhere on the board. Thus the action space is 5 (appliances)  225 (board size, 15′ by 15′) = 1125. However, not all actions make design sense, e.g., placing a fridge in the middle of the room. Rules are implemented to define illegal actions that lead to absurd and useless states. The fewer legal actions remain, the smaller the solution space gets. The kitchen layout environment includes four rules: (1) Appliances can only be placed against a wall, on an island, or a peninsula, (2) Appliances automatically face in the right direction, (3) Tall appliances, such as pantry and fridge, are not allowed in front of a window, (4) No appliances can get in the way of another appliance’s clearance zone.

Fig. 1 The framework of the system: the builder and the user: the system builder builds a virtual assistant that can co-create with the system user like in a conversation

404

P. Hsieh et al.

After an agent reaches an end state, the human devised metrics evaluate the design and produce a score. However, the parameters and their priorities are subjective to personal preference. For example, in kitchen design, a good workflow might not be achievable in any configuration when a sink is placed in front of a window in a given map. In other words, a person who cooks a lot may be annoyed by extra footsteps in everyday cooking; yet another who cooks occasionally might not mind so if he gets to enjoy the window view while washing dishes. Therefore, a trainer’s preference is a subjective cause for tuning priorities and weights of the metrics. After the iterative process, the agent eventually becomes an extension of its trainer’s mind. The metrics used in this paper are inspired by the following. The step-saving U-kitchen suggests: (1) A cooking workflow from fridge to sink to stove in geometric location, (2) A mixing counter 3′ to 4′ wide. Ikea Kitchen Planner suggests: (1) Worktop area on each side of the cooking unit should be at least 15″ wide, (2) Worktop area in between a cooking unit and a sink should be at least 30″ wide, (3) Sink, stove, or oven should not locate next to a wall or in a corner, (4) The dishwasher should locate close to a sink. Others come out from discussions with interior designers: (1) Fridge should be easily accessible, (2) Sink and stove should not locate at countertop ends.

3.1.2

Self-Play and Data Collection

A self-play tree search algorithm is implemented to explore the state values from the kitchen environment. A state value is the state’s best possible outcome score of the end states it can reach (see Fig. 2). A board configuration, if mirrored and rotated, can turn into eight different data points with the same state value, thus eliminates directional biases.

Fig. 2 (Left) The self-play mechanism is a tree search algorithm that explores the solution space. (Right) At the end of an episode, the evaluation score becomes the state values that backpropagates to the parent nodes

Conversational Co-creativity with Deep Reinforcement Learning …

3.1.3

405

Neural Network

A deep neural network allows the agent to gain generalized knowledge and apply beyond the training maps. Its architecture was obtained from Surag Nair’s alpha-zero-general [29], containing four convolutional layers and four fully connected layers. The network is trained from scratch and retrained whenever the rules, metrics, or appliances change to correctly reflect the intrinsic state values of the new environment.

3.2

Using the Assistant: Conversational Co-creativity Interface

A graphic user interface is implemented to enable the conversational co-creativity workflow (see Fig. 3). The assistant can suggest locations for any appliance with a degree of confidence, namely the projected final score. The user has options to (1) accept the proposed move, (2) view another suggestion, (3) execute their own move (4) undo, remove, or clear to try another design trajectory. Supportive evaluation metrics are displayed both as scores and a star diagram in the left column.

Fig. 3 Interface screenshot—Instructions for the user interface. An indicator shows the assistant suggests placing the pantry at (1, 7) with a confidence level of 99

406

P. Hsieh et al.

4 Results The assistant was trained on thirty-four handcrafted maps. It is capable of finding optimal solutions in both the training maps and the test maps (see Fig. 4). A breadth-first tree search algorithm is used. The trained neural network predicts the values of all possible next states and investigates only the best three at the first level and the best two for the following levels. This approach searches 48 (3  2  2  2  2) design solutions, whose values are confirmed by evaluation metrics and used to rank the immediate action suggestions. Each map takes around 3 to 8 s, depending on the size of the action space. A solution is validated the best by the failure of manually finding a better design score.

Fig. 4 Optimal design solutions found by the assistant for the 34 training maps and the 5 test maps

Fig. 5 In the test map #4 (a) The assistant is capable of finding the optimal solution, scoring 99 points. (b1) The empty map. (b2) The user decides to place the stove in front of the window, disagreeing with metric # 9: Putting a stove at a window can cause condensation, the hood blocking the window view, or curtains to catch fire. (b3 to b7) The assistant continues to give suggestions and find the optimal solution given the location of the stove, scoring 87 points, thus a sub-optimal solution. (b8) A procedural algorithm generates cabinetry that fills the remaining space

Conversational Co-creativity with Deep Reinforcement Learning …

407

The assistant is also capable of finding a sub-optimal solution, which means if the user decides to violate a metric, it can still find and suggest the best moves to optimize for other metrics. (see Fig. 5).

4.1

Training Time

About 82,000,000 training examples are collected from self-play for the 34 training maps, taking around 150 h on one core of the CPU, the Intel Core i7-9700 K at 3.6 GHz. Training the neural network for one epoch takes around 22 h, and it took two epochs to converge to the presented result. The GPU used for training is a single NVIDIA GeForce RTX2080.

5 Conclusion This paper attempts to create a conversational co-creativity interaction between humans and machines by first building an intelligent virtual assistant. The paper proposes a divided responsibility where a computer focuses on the standardized and quantifiable goals of design, while a designer focuses on the personalized and less measurable goals. The proposed system re-frames the layout design problem as a single-player board game problem and uses the DRL approaches that originated in game-playing agents. The benefits of the system are as follows: The builder of our system, in contrast to the commonly used rule-based system, can train and teach a neural network agent through the means of evaluation metrics and more straightforward rules, and as a result, spending less time. The user of our system has the flexibility to explore their personal and creative goals while getting action suggestions optimized for the standardized and quantifiable metrics. Moreover, this paper argues that the perceived authorship of the final solution will be human’s for two reasons: (1) A user has control over every design step, in contrast to the “solution-to-chose-from” approach. (2) The assistant is merely an enhancement for the standardized and quantifiable goals and is the extended mind of its human trainer. Finally, this helps to establish the relationship between the user and the assistant as a conversational one. A foreseeable problem of the proposed framework is a scale problem. When the size of the map or the number of appliances increase, computational time increases exponentially. The current framework, which uses an exhaustive search in the environment, will eventually become too costly. To tackle this, techniques from reinforcement learning that deals with exploration and exploitation will become useful, and more research needs to be done. The paper demonstrates the framework using kitchen layout as an example, but the applications are potentially broader. As long as a goal can be quantified, it can

408

P. Hsieh et al.

turn into a metric for the neural network to learn. We hope this approach responds to the ‘black box’ issue commonly associated with deep learning agents and presents a different way to train and interact with a virtual agent.

References 1. Kolarevic B (ed) (2005) Architecture in the digital age: design and manufacturing, 1st edn. Taylor & Francis, New York 2. Benfer R, Brent E, Furbee L (1991) Expert systems. SAGE Publications Inc, London 3. Yen C-C, Tang H-L (1989) Inside an expert system: strengths, weaknesses, and trends. J. Comput. Inf. Syst. 30(1):34–39. https://doi.org/10.1080/08874417.1989.11646943 4. Chaillou S (2018) AI + architecture | towards a new approach. Harvard University, Cambridge 5. Silver D et al (2018) A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science 362(6419):1140–1144. https://doi.org/10.1126/science. aar6404 6. Kotsopoulos S (2008) Shape grammar workshop MIT. SIGRAPH 7. Duarte JP (2005) Towards the mass customization of housing: the grammar of siza’s houses at malagueira. Environ Plann B Plann Des 32(3):347–380. https://doi.org/10.1068/b31124 8. Benros D (2018) A generic housing grammar for the generation of different housing languages 9. Anderson C, Bailey C, Heumann A, Davis D (2018) Augmented space planning: using procedural generation to automate desk layouts. Int J Archit Comput 16(2):164–177. https:// doi.org/10.1177/1478077118778586 10. Finch. https://finch3d.com/. Accessed 19 Dec 2019 11. TestFit. https://blog.testfit.io/. Accessed 19 Dec 2019 12. Higharc. https://higharc.com/. Accessed 19 Dec 2019 13. Singh V, Gu N (2012) Towards an integrated generative design framework. Des Stud. https:// doi.org/10.1016/j.destud.2011.06.001 14. Nagy D et al (2017) Project discover: an application of generative design for architectural space planning. In: Proceedings of the 2017 symposium on simulation for architecture and urban design. https://doi.org/10.22360/SimAUD.2017.SimAUD.007 15. Goel A et al (2015) Using Watson for enhancing human-computer co-creativity. In: Association for the advancement of artificial intelligence 16. Fischer G, Mccall R, Mørch A (1989) Design environments for constructive and argumentative design. ACM SIGCHI Bulletin 20:269–275. https://doi.org/10.1145/67450. 67501 17. Mohd Ali N, Hosking J, Grundy J (2013) A taxonomy and mapping of computer-based critiquing tools. Softw Eng IEEE Trans 39:1494–1520. https://doi.org/10.1109/TSE.2013.32 18. Nakakoji K, Fischer G (1993) Knowledge delivery: facilitating human-computer collaboration in integrated design environments. In: AAAI Technical Report FS-93-05 19. Yannakakis GN, Liapis A, Alexopoulos C (2014) Mixed-initiative co-creativity. FDG 20. Karimi P, Grace K, Davis N, Maher ML (2019) Creative sketching apprentice: supporting conceptual shifts in sketch ideation. In: Gero JS (ed) Design computing and cognition 2018. Springer, Cham, pp 721–738. https://doi.org/10.1007/978-3-030-05363-5_39 21. Write Emails Faster with Smart Compose in Gmail. Google. https://blog.google/products/ gmail/subject-write-emails-faster-smart-compose-gmail/. Accessed 24 Nov 2019 22. Radford A, et al (2019) Language models are unsupervised multitask learners 23. Sutton RS, Barto AG (1998) Reinforcement learning: an introduction, 2nd edn. MIT Press, New York

Conversational Co-creativity with Deep Reinforcement Learning …

409

24. Kinchin J, O’Connor A (2011) Counter space: design and the modern kitchen. The Museum of Modern Art 25. United States. Bureau of Human Nutrition and Home Economics (1949) A step-saving U kitchen. U.S. Dept. of Agriculture, Washington, D.C. 26. Beyer GH (1955) The Cornell kitchen, product design through research. New York State College of Home Economics in association with the Cornell University Housing Research Center 27. Randl C (2014) Look who’s designing kitchens’: personalization, gender, and design authority in the postwar remodeled kitchen. Build. Landsc. J. Vernacul. Archit. Forum 21 (2):57. https://doi.org/10.5749/buildland.21.2.0057 28. IKEA Home Planner. https://kitchenplanner.ikea.com/GB/UI/Pages/VPUI.htm. Accessed 26 Nov 2019 29. Nair S (2017) Alpha-zero-general. https://github.com/suragnair/alpha-zero-general. Accessed 02 Dec 2019

Assessing the Novelty of Design Outcomes: Using a Perceptual Kernel in a Crowd-Sourced Setting Dongwook Hwang and Kristin Lee Wood

Abstract The novelty of design concepts is often measured by experts’ judgment or indirect quantitative measures. Such measurement may have caused some issues of under or overestimation of humanjudgment due to their limited memory or experience. As an alternative to direct human judgment, crowdsourcing has been used for the evaluation of designs. While crowds offer diversity and a wide experience base, crowds may bring different aspects of viewpoints on evaluating novelty and have an inconsistency between them. To address this problem, we develop a new novelty assessment system using a perceptual kernel which is a distance matrix derived from aggregate perceptual judgments in a crowd-sourced setting. In order to demonstrate the impact of the assessment system, we compared the results with expert subjective ratings on novelty.

1 Introduction Designers are interested in enhancing their creative capabilities through ideation. Much research on concept generation focuses on how to enhance designers’ creative capabilities for their problem-solving activities. However, in order to identify their creative capabilities, we need to explicitly evaluate their ideation performance. One of the key aspects of quantifying creativity in concept generation is the novelty of idea outcomes. Currently, while indirect quantitative measures exist [1, 2], the most frequent measurement of novelty is conducted by human judges. In such evaluation tasks, humans are required to process idea outcomes, identify similar ideas from their own knowledge sets, make mental connections and compare ideas based on the uniqueness. However, a small set of humans are not capable D. Hwang
K. L. Wood (&) SUTD-MIT International Design Centre, Singapore University of Technology and Design, Singapore, Singapore e-mail: [email protected] D. Hwang School of Media and Communication, Kwangwoon University, Seoul, Republic of Korea © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. S. Gero (ed.), Design Computing and Cognition'20, https://doi.org/10.1007/978-3-030-90625-2_24

411

412

D. Hwang and K. L. Wood

of knowing all existing design alternatives similar to those being considered. It is also possible that they underestimate true novelty of ideas due to their limited memory and experiences. Even experts with substantial knowledge of a specific field will experience the same behaviors. It is often challenging to maintain their own evaluation strategies throughout the entire evaluation task. As an alternative to direct human judgment, especially obtained from expensive and inconsistent experts, crowdsourcing has been used for the evaluation of ideas. While crowds offer diversity and a wide experience base, crowds may bring different aspects or viewpoints on evaluating novelty and there may exist inconsistency across the population. To address this problem, we develop a new novelty assessment system using a perceptual kernel and compute novelty from this kernel. Based on the assumption that humans are better at assessing similarities and comparing items in pairs rather than making absolute assessment [3], we can identify distance matrices derived from aggregated perceptual dissimilarity judgments in a crowd-sourced setting and further quantify the crowds’ perceived dissimilarity between ideas. The perceptual kernels can help us create visualizations that better reflect the relationship between ideas in a two-dimensional space and further compute the novelty of an idea by aggregating the distances from each other [4]. In this study, two case studies of graphic design and product design were conducted to demonstrate the impact of crowd-estimated perceptual kernel approach. As an illustrative purpose, this study considers design tasks of evaluating the novelty of the book cover and end-of-life urn designs. In both case studies, we compute the novelty of designs by utilizing a crowd-estimated perceptual kernel approach and compared the results with expert evaluation.

2 Related Work There exist a number of methods for assessing novelty of ideation outcomes. The Consensual Assessment Technique (CAT) [5] suggests an approach of assessing novelty through the subjective evaluation by expert judges. In addition, Shah et al. [2] proposed a set of metrics to measure the effectiveness of idea generation methods for conceptual design. The further refinements and variations to the work of Shah et al. [2] were proposed in various other literature [6–12]. Moreover, Sarkar and Chakrabarti [13] also developed a method to assess novelty of an artefact by incorporating the concept of function–behavior–structure (FBS) [14] and SAPPhIRE (state change, action, parts, phenomenon, input, organs, and effect) models [15]. The four modifications of this model were suggested by the work of Jagtap [16]. Overall, several novelty assessment methods have been proposed; however, they are mainly dependent on human judgment in choosing attributes underlying the ideas, judgment on idea categorizing and even require direct evaluation of subjective ratings for novelty. To consider alternative or complementary approaches to subjective judgment, other methods use rarity as one factor of evaluating novelty. Jansson and Smith [17]

Assessing the Novelty of Design Outcomes …

413

suggest the novelty computation as the sum of the novelty scores for an individual’s ideas divided by the number of ideas for that subject. Linsey et al. [18] assess the novelty of each product solution as a function of the number of similar product solutions (i.e., number of product solutions in that particular bin) relative to the total number of product solutions. Chan et al. [19] suggest a novelty metric which measures the quantity of non-similar or non-related concepts with respect to the total of concept ideas generated. One of the disadvantages of measuring novelty based on rarity is that this approach is limited if many non-repetitive ideas are produced. Those ideas which are slightly different from more common solutions to the problem may be assessed as highly rare and novel ideas. The novelty assessment based on rarity alone may be insufficient. Apart from the human judgment on assessing novelty, crowdsourcing approaches for deriving the level of novelty of ideation outcomes have also been introduced. Green, Seepersad, and Hölttä-Otto [20] investigate the feasibility of using nonexperts in a crowdsourcing environment to evaluate engineering creativity. The results of this study show that non-expert student raters with excellent inter-rater agreement amongst themselves and fully trained can evaluate the originality measure reliably. Luther and their co-authors [21] created CrowdCrit, a web-based system that allows designers to receive design critiques from non-expert crowd workers, demonstrate it with non-expert crowd workers and compare the results with experts. This study shows that the aggregated critiques from non-expert crowd workers approaches expert critique. Overall, crowdsourcing approaches are gaining interest in the research community, with many recommendations for future work and further studies. An illustration of this viewpoint is shown in the work of Burnap et al. [22], where the authors show limitations of crowdsourcing, especially in evaluation tasks.

3 Method A perceptual kernel is the distance matrix of aggregated pairwise perceptual distances [4]. The perceptual distances can be achieved from the subjectively judged similarities between items. To construct the perceptual kernel, judged similarities, such as Likert ratings among pairs, ordinal triplet comparisons and manual spatial arrangement, have been utilized [4]. In this study, the ultimate goal in introducing perceptual kernels is to utilize simple similarity judgments in assessing the novelty of ideation outcomes. In order to perform simple similarity judgments, we employ a pairwise dissimilarity comparison with a 5-point Likert scale. Based on the results of judgment, a perceptual kernel for the entire set of ideas is constructed. Then, we transform the perceptual kernel into two-dimensional projections and plot the ideas on the projections to compute the novelty as the sum of distance from other ideas. The workflow of the entire procedure for assessing novelty using a perceptual kernel is shown in Fig. 1.

414

Fig. 1

3.1

D. Hwang and K. L. Wood

Workflow of a perceptual kernel approach

Pairwise Dissimilarity Comparison

We employ pairwise dissimilarity comparison using a 5-Point Likert Scale. In this task, participants are presented pairs of design alternatives and asked to rate the similarity of each pair on a 5-point Likert scale. Each participant is given a randomized order for pairwise comparison.

3.2

Novelty Computation

Regarding the aspect of novelty, novel ideas can be interpreted as how unique or dissimilar an idea is [23]. It can be regarded as a similar task to finding ideas which are distant from all other ideas on a two-dimensional projection. As nearby ideas on the projection denote similarity with each other, the idea furthest away from all other ideas will be the most unique to the set [24]. In this study, we employ the same novelty metric used in Ahmed et al. [24]. The novelty score of idea i is defined as: NoveltyðiÞ ¼

N X

d i;j

j¼1

where di, j is distance of idea i from idea j in the two-dimensional projection. The novelty of an idea is computed by adding distances from the idea to all other ideas. Ideas with a high score are expected to be further apart from other ideas on a two-dimensional projection, while ideas with a low score are be located near many others.

4 Example Illustrative Studies Two case studies are performed for an illustrative purpose. The perceptual kernels are constructed based on crowd-estimation for book cover and urn designs, and each perceptual kernel is projected into a two-dimensional space, where the distances between ideas were computed for novelty. The results are compared with experts’ judgment of novelty.

Assessing the Novelty of Design Outcomes …

415

Fig. 2 Alternative designs for a book cover and b urn

As shown in Fig. 2(a, b), 10 book cover and urn designs are randomly selected in this study. As a basic test case and pilot study, eleven and eight participants were recruited for a pairwise comparison task for book cover and urn designs, respectively. Each group of participants performed a pairwise dissimilarity comparison task through an online Google survey.

4.1

Crowd-Estimated Perceptual Kernel Approach for Computing Novelty

Participants are asked to evaluate two randomly selected book cover and urn designs based on how different the pair looks to them (Fig. 3). Each participant had all the pairs (45) of both the book cover and urn designs for evaluation. The pairwise dissimilarity comparison task directly produces a dissimilarity matrix among the designs. The rating results obtained from all participants are aggregated into two perceptual kernels for book cover and urn designs. In Fig. 4(a, b), each cell of the perceptual kernels was highlighted with color depending on the average scores of the dissimilarity between the designs—the darker black color indicates more favored similarity while the white color means the pair looks very different.

Fig. 3 The pairwise dissimilarity comparison task for book cover designs

416

D. Hwang and K. L. Wood

Fig. 4 Perceptual kernels for a the book cover and b urn designs

The resulting perceptual kernel as a matrix diagram can be transformed into a two-dimensional projection by using multidimensional scaling (MDS). The two-dimensional projection intends to provide a more intuitive visualization of the perceptual kernel. In this way, the perceptual distances between ideas can be approximated in the two-dimension space. As for the book cover design, a perceptual kernel (Fig. 4a) can be mapped into the two-dimensional space by using MDS (Fig. 5). With the dissimilarity scores in the kernel, we can compute the novelty by adding relative distances between one idea and the others. As nearby ideas on the map denote similarity with each other, the idea furthest away from all others becomes the most novel idea from the entire

Fig. 5 The two-dimensional projection of the perceptual kernel for the book cover designs

Assessing the Novelty of Design Outcomes …

417

Fig. 6 An example of relative dissimilarity scores between the book cover design #10 (yellow coffee design) and others; the number in each cell indicates average pairwise dissimilarity comparison scores from crowds

idea set. For example, the crowd-estimated novelty score for book cover design #10 was computed by adding all relative dissimilarity scores between the book cover design #10 and the other book covers (Fig. 6). Figure 7 presents the two-dimensional projection of the perceptual kernel for the urn designs. There were five traditional urns which are clustered closely together, and the other five urns are spread out as shown in Fig. 7. The novelty score for urn design #5 was computed by adding all relative dissimilarity scores between the urn design #5 and the other urns. The same procedure was applied to compute the novelty score for urn design #6. Urn design #5 represents some similarity with the urns #1–4 as they are considered traditional while the urn design #6 is quite dissimilar to all other urn designs (Fig. 8). This result shows that the relatively similar and traditionally well-known urns with minor changes would be assessed with low novelty scores as compared with some newly generated, individualized urn designs (urn design #6–10).

Fig. 7 The two-dimensional projection of the perceptual kernel for the urn designs

418

D. Hwang and K. L. Wood

Fig. 8 The examples of relative dissimilarity scores between urn designs (#5: blue urn with bird design and #6: start-shaped design) and others; the number in each cell indicates average pairwise dissimilarity comparison scores from crowds

4.2

Experts’ Ratings on Novelty

Two experts subjectively evaluated the direct perceived novelty scores of the entire set of book cover and urn designs. Both experts all had degrees in design and had worked full-time as professional designers. The experts used 5-point rating scales (one: extremely low, five: extremely high) to subjectively determine the novelty scores for each design, where a score of one denotes the lowest possible rating and a score of five indicates the highest possible rating. For each metric, the average of the ratings of the two experts was computed and used in subsequent analyses. They were both blind to the conditions of the experiment and the hypothesis. For the purpose of assessing inter-rater reliability, we employed the Intraclass Correlation Coefficient (ICC). The average ICCs for book cover and urn designs were 0.816 with a 95% confidence interval from 0.325 to 0.953 (F(9) = 5.762, p = 0.008); and 0.909 with a 95% confidence interval from 0.631 to 0.978 (F (9) = 10.159, p = 0.001), respectively.

4.3

Comparisons Between Crowd-Estimated and Expert Evaluation Approaches

In this study, we hypothesize that there would be no significant difference on utilizing two approaches of evaluating novelty: the use of crowd-estimated perceptual kernel and experts’ judgment on novelty scores. For book cover design, a Wilcoxon signed-rank test was conducted to compare two approaches on novelty evaluation. The results indicated that experts’ novelty evaluation was not significantly different with the crowd-estimated perceptual kernel approach, W = 97.00, Z = −0.611, p > 0.05 (Fig. 9). As for urn designs, another Wilcoxon signed-rank test was also conducted and the results indicated that experts’ novelty evaluation was not significantly different with the crowd-estimated perceptual kernel approach, W = 101.5, Z = −0.269, p > 0.05 (Fig. 10). This finding shows, at least for this

Assessing the Novelty of Design Outcomes …

419

Fig. 9 The novelty ranks of book cover designs assessed with expert rating and crowd-estimated perceptual kernel approach (one: highest, ten: lowest in rank)

Fig. 10 The novelty ranks of urn designs assessed with expert rating and crowd-estimated perceptual kernel approach (one: highest, ten: lowest in rank)

pilot study, that crowd-estimated relative similarity comparison transforming into novelty measures matches, within statistical significance, the result achieved from expert’s judgment on novelty evaluation. As an alternative way of validating crowd-estimated novelty assessment with the experts’ rating scores, we employed Spearman’s rank correlation coefficient to quantify the degree of similarity between crowd-estimated perceptual kernel approach and experts’ subjective rating approach. The results showed that there was a strong but negative correlation for book cover designs between ranks of crowd-estimated novelty scores and experts’ subjective rated novelty scores, which was statistically significant (rs(8) = −0.782, p = 0.008). As for urn designs, a strong, positive correlation between ranks of crowd-estimated novelty scores and experts’ subjective rated novelty scores was observed (rs(8) = 0.766, p = 0.010).

5 Discussion and Conclusion The concept of the perceptual kernel is introduced to compute the novelty of graphic and product designs. Within the crowdsourcing environment, the use of a perceptual kernel may help us to evaluate the novelty of an idea as compared with

420

D. Hwang and K. L. Wood

many other ideas. The novelty computed by a crowd-estimated perceptual kernel does not show a significant difference but does show a strong relationship with the novelty judged by expert designers. This finding illustrates that a simple pairwise comparison is well matched with expert ratings for novelty of graphic and product designs. A strong but negative correlation exists between crowd-estimated and expert evaluation approaches for book cover designs. This result may be attributed to the fact that experts can assess the novelty of ideas based on the aspect of ‘surprise’. As a strong indicator of creativity, surprise is the degree to which confident expectations about a design are violated by observing it [25, 26]. Experts may have an ability to identify the expectations from a set of their past experiences [27]. A few book covers, such as fourth, eighth and ninth cover designs in Fig. 9, are considered to be surprising to experts; however, less experienced crowds may disagree since they only consider the structural similarity for their evaluation. Limitations of the current study are acknowledged here, along with some future research directions. There could be more approaches of making similarity judgment other than pairwise comparison, such as triplet matching and spatial arrangement. Incorporating other approaches will provide us a better understanding on how perceptual kernels can be structured and performed more efficiently in computing novelty. Additionally, a comparison to other algorithmic approaches of assessing novelty might be necessary to strongly demonstrate the crowd-estimated approach. There could be other approaches using distance to nearest cluster or computing a similarity-based reconstruction loss. Moreover, future research may consider additional variable related to the degree of experience of crowds with an assumption that the degree of experience may be one factor that might affect the pairwise comparison judgment. We might think of assigning a higher weight on one’s assessment if one has seen many artifacts before. Further, since we have not demonstrated our approach with crowds but as pilot studies with a relatively small number of participant evaluators, future research should extend the perceptual kernel approach of evaluating novelty to larger crowdsourcing contexts using the existing platforms like Amazon MTurk. Finally, future work will consider how different approaches for evaluating novelty may be used together, in aggregate, or as complements. Acknowledgements This work was supported by the SUTD-MIT International Design Centre (IDC, https://idc.sutd.edu.sg/) and the CU Denver Comcast Media and Technology Center. Any opinions, results, or conclusions in the paper are those of the authors, and do not necessarily reflect the views of the sponsors. We acknowledge the fabulous professional design work of Swee Ping En Amanda and Tan Wei Hua in developing contemporary urn designs.

Assessing the Novelty of Design Outcomes …

421

References 1. Chakrabarti A, Khadilkar P (2003) A measure for assessing product novelty. In: DS 31: Proceedings of ICED 03, the 14th international conference on engineering design, Stockholm, pp 159–160 2. Shah JJ, Smith SM, Vargas-Hernandez N (2003) Metrics for measuring ideation effectiveness. Des Stud 24(2):111–134 3. Mantiuk RK, Tomaszewska A, Mantiuk R (2012) Comparison of four subjective methods for image quality assessment. Computer graphics forum, vol 31, no 8. Blackwell Publishing Ltd, Oxford, pp 2478–2491 4. Demiralp Ç, Bernstein MS, Heer J (2014) Learning perceptual kernels for visualization design. IEEE Trans Visual Comput Graph 20(12):1933–1942 5. Amabile TM (1983) The social psychology of creativity: a componential conceptualization. J Pers Soc Psychol 45(2):357 6. Fiorineschi L, Frillici FS, Rotini F (2018) A-posteriori novelty assessments for sequential design sessions. Int Des Conf Des 2018:1079–1090 7. Hernandez NV, Shah JJ, Smith SM (2010) Understanding design ideation mechanisms through multilevel aligned empirical studies. Des Stud 31(4):382–410 8. Lopez-Mesa B, Mulet E, Vidal R, Thompson G (2011) Effects of additional stimuli on idea-finding in design teams. J Eng Des 22(1):31–54 9. Nelson BA, Wilson JO, Rosen D, Yen J (2009) Refined metrics for measuring ideation effectiveness. Des Stud 30(6):737–743 10. Peeters J, Verhaegen PA, Vandevenne D, Duflou JR (2010) Refined metrics for measuring novelty in ideation. In: IDMME virtual concept research in interaction design, 20–22 October 2010 11. Verhaegen PA, Peeters J, Vandevenne D, Dewulf S, Duflou JR (2011) Effectiveness of the PAnDA ideation tool. Procedia Eng 9:63–76 12. Verhaegen PA, Vandevenne D, Peeters J, Duflou JR (2013) Refinements to the variety metric for idea evaluation. Des Stud 34(2):243–263 13. Sarkar P (2007) Development of a support for effective concept exploration to enhance creativity of engineering designers. Doctoral dissertation, PhD Thesis. Indian Institute of Science 14. Gero JS (1990) Design prototypes: a knowledge representation schema for design. AI Mag 11 (4):26–26 15. Srinivasan V, Chakrabarti A (2009) SAPPhIRE – an approach to analysis and synthesis. In: 17th International conference on engineering design (ICED09), Stanford, 23–27 August 2009 16. Jagtap S (2016) Assessing design creativity: refinements to the novelty assessment method. In: DS 84: Proceedings of the DESIGN 2016 14th international design conference, pp 1045– 1054 17. Jansson DG, Smith SM (1991) Design fixation. Des Stud 12(1):3–11 18. Linsey JS, Clauss EF, Kurtoglu T, Murphy JT, Wood KL, Markman AB (2011) An experimental study of group idea generation techniques: understanding the roles of idea representation and viewing methods. J Mech Des 133(3):031008 19. Chan J, Katherine F, Schunn C, Cagan J, Wood K, Kotovsky K (2011) On the benefits and pitfalls of analogies for innovative design: ideation performance based on analogical distance, commonness, and modality of examples. J Mech Des 133(8):081004 20. Green M, Seepersad CC, Hölttä-Otto K (2014) Crowd-sourcing the evaluation of creativity in conceptual design: a pilot study. In: ASME 2014 international design engineering technical conferences and computers and information in engineering conference. American Society of Mechanical Engineers Digital Collection 21. Luther K, Tolentino JL, Wu W, Pavel A, Bailey BP, Dow SP (2015) Structuring, aggregating, and evaluating crowdsourced design critique. In: Proceedings of the 18th ACM conference on computer supported cooperative work & social computing, pp 473–485

422

D. Hwang and K. L. Wood

22. Burnap A, Ren Y, Gerth R, Papazoglou G, Gonzalez R, Papalambros PY (2015) When crowdsourcing fails: a study of expertise on crowdsourced design evaluation. J Mech Des 137 (3):031101 23. Verhaegen PA, Vandevenne D, Duflou JR (2012) Originality and novelty: a different universe. In: DS 70: Proceedings of DESIGN 2012, the 12th international design conference, Dubrovnik, Croatia 24. Ahmed F, Fuge M, Hunter S, Miller S (2018) Unpacking subjective creativity ratings: using embeddings to explain and measure idea novelty. In: ASME 2018 international design engineering technical conferences and computers and information in engineering conference. American Society of Mechanical Engineers Digital Collection 25. Grace K, Maher ML, Fisher D, Brady K (2015) Modeling expectation for evaluating surprise in design creativity. In: Gero JS, Hanna S (eds) Design computing and cognition 2014. Springer, Cham, pp 189–206 26. Brown DC (2012) Creativity, surprise & design: an introduction and investigation. In: DS 73-1 proceedings of the 2nd international conference on design creativity 27. Grace K, Maher ML, Fisher D, Brady K (2015) Data-intensive evaluation of design creativity using novelty, value, and surprise. Int J Des Creativ Innov 3(3–4):125–147

Serendipitous Explorations in Distributed Work in Parametric Design Livanur Erbil Altintas, Altug Kasali, and Fehmi Dogan

Abstract This study reports a case involving computational practices in architectural design to understand designers’ explorations using parametric modeling tools in collaborative work environment. The objective is to understand how parametric modeling tools enable serendipitous design solutions in a distributed cognitive system. Following an ethnographic approach, we have investigated the nature of the interactions among team members as they produce and propagate a series of design representations. We have focused on how architectural design teams manage design development processes and how teams manage serendipity as the system creates multiple alternatives in an explorative setting. The episodes presented in this paper support the argument that parametric modeling practices facilitate serendipitous discoveries at various steps by offering multiple alternatives among which designers identify solutions in an opportunistic manner.

1 Introduction This study presents preliminary findings from an ethnographic research on multidisciplinary architectural design teams which operate as distributed cognitive systems. Within the scope of this paper, we present episodes of collaborative practices of design teams that adopted parametric modeling tools in their processes. Our aim is to develop an understanding of the series of strategies in a synchronized design exploration involving individuals, tools and representations. The research investigates how architectural design teams manage design development processes in a distributed cognitive system and how teams manage serendipity as the systems create multiple alternatives in an explorative setting. By design exploration, we refer to the process of creating multiple design alternatives to be represented in various representational modes [1]. Within this form of exploration, we distinguish serendipity as a distinct search process that leads to unintended discovery within L. Erbil Altintas (&)
A. Kasali
F. Dogan Izmir Institute of Technology, Izmir, Turkey e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. S. Gero (ed.), Design Computing and Cognition'20, https://doi.org/10.1007/978-3-030-90625-2_25

423

424

L. Erbil Altintas et al.

complex design situations [2]. These types of explorations have been recognized as “the driving force for the invention of issues or requirements” in situated design environments [3]. In approaching our case study, we have adopted the distributed cognition framework to account for the interactions, computations, and communications within a system consisting of human and non-human agents [4–7]. Individuals interact with others, artifacts, technologies, tools, surfaces, and the things that are represented within the system. According to the distributed cognitive system framework, ‘embedded’ individuals interact with artifacts, technologies, and tools to coordinate their internal cognitive tasks with external tools [8]. Thus, tracking the generation, manipulation, and propagation of representations is critical in understanding the mechanisms within the system. In this complex system, we consider design as an explorative act between sets of rules and constraints to explore alternatives [9]. According to Schön [9], explorative settings are created by a designer who “… constructs the design world within which he/she sets the dimensions of his/her problem space, and invents the moves by which he/she attempts to find solutions” [9]. Designers explore alternatives by setting rules, requirements, or adapting principles in the design process. Exploration occurs through iterative moves between problem space and solution space [10]. Within this search, serendipity is recognized as a form of exploration which is a result of deliberate and persistent search to satisfy major design intentions and constraints. To explore corresponding solutions to a design problem, being open to variety of alternatives and being prepared to arrive at a solution is essential [11–13]. In such explorative settings, Goldschmidt [12] refers to the importance of a “prepared eye” related to expertise to identify and adapt serendipitous solutions in a design process. In this research, we observed explorative settings where architects utilize parametric modeling tools to support generation of alternatives which is traditionally related to expertise in design [14]. Following programming codes created by participants, parametric modeling enables the production of multiple alternatives that meet design constraints considered. The variety of alternatives gives designers the ability to explore a larger solution space. Designing is ill-defined and it provides ambiguity with variety that gives richer situations of exploration. Diversity can be produced by means of computational languages, which might provide the designer an open space for opportunities, just like the ambiguity provided by any other deign media. In the case presented, we have observed a series of acts of sense-making by assessing the diversity of solutions offered by computational tools. We focus on the instances of serendipity as designers, individually or collectively, evaluate the set of solutions generated by the system, and claim that the computer is not only a very powerful calculator, but also a very important tool for learning [15] which can be evaluated as making interpretations from serendipitous explorations.

Serendipitous Explorations in Distributed Work in Parametric Design

425

2 Methods and the Cases This research is a qualitative study consisting of ethnographic observations and interviews with two professional architectural teams at Global Architectural Development (GAD), which is one of the pioneering architectural offices in Turkey, especially specialized in the use of parametric modeling tools in architectural design processes. In the first case, -a library project (P1) for İstanbul Technical University (ITU) - data was not collected real-time. The firm has a notable archive in the office to keep all project documents. Our study in the archive revealed particular documents for P1 including the alternatives which were eliminated in the design process. The project team for P1 consisted of one architect office leader (OL), one architect as coder (C1), and one architect as Job Captain (JC). In order to investigate the details of the library project, we conducted face to face semi-structured interviews with participants to provide a way to explore feelings, opinions, and behaviors [16]. The interviews highlighted the teams’ work flow in the process and the strategies for computation, representation and communication. In the second case (P2) -a large scale civic building in Istanbul-, data collection was real-time as the team put together a proposal for an invited competition. The case focused on the accommodation and hotel sections of the project, which were designed as towers. The focused team consisted of one architect office leader (OL), one architect as coder (C2), and one architect as Job Captain (JC). We conducted in-situ observations to understand the daily practices in the office [17], and semi-structured interviews with the participants. For both of the cases, the data set included video- and audio-recordings, photographs, sketches and notes. Digital files of the projects were not shared by the team but we were allowed to collect screenshots. The analysis of qualitative data followed the procedures of ‘grounded theory’. The qualitative data emerging from our interviews, recorded observations and the corpus of field notes was coded through open, axial and selective processes [18]. The inductive analytical process has led to a set of categories emerging out of the coding protocols. Then, the categories were listed according to the phenomena under focus. To create relationships between the categories and to build a ‘story’ or an episode, the researcher connected the categories considering their chronological order [18].

3 Exploration P1: Public Library Computational design tools enable a search space for a wide range of solutions [19]. In the first case involving a college library design (P1), the office leader and the coder worked closely during the early phases of design. The team was able to come

426

L. Erbil Altintas et al.

up with three different alternatives in order to convince the college administration. Each alternative was about exploring the formal qualities of the facility, rather than covering the functional program. The office leader’s intention was to push for the third option which displayed unusual formal qualities with paraboloid surfaces. The formal idea for the preferred option which was based on an arcade system and associated explorations through parametric tools had been on the agenda of the office, and the library project was seen as an opportunity to further explore and implement the idea in an architectural project. Having three alternatives allowed the office leader to push for their preferred scheme which required the utilization of particular tools in design processes. He was clearly enthusiastic of the arcade system because of its extraordinary and stylistic aspects. In order to obtain the desired qualities, a continuous formal exploration was required right from the beginning. For this favored alternative, the team used Wolfram Mathematica Software to generate the formal schema. It is a sophisticated computing system to solve problems in various domains including neural networks, geometry and visualization. The team considered this computing system as key to generate the series of paraboloid forms. Initially the use of Mathematica, obviously, limited the search space to obtain the final solution. However, the exploration space was still vast, and it allowed opportunities for various paths of serendipity in design. The coder in this project was an architect, working with coding languages and knowledgeable of Wolfram Mathematica and Rhino Software which were synchronously employed in the process. The Mathematica software provided the possibility to create paraboloid surfaces through processing a formula and facilitated a search space for designers by means of a series of parameters. The Mathematica software was used as a particular design tool which provided instant visualizations of geometries as participants manipulate the formula (Hyperbolic Paraboloid). The design team worked with Mathematica in three stages: in Stage 1, the designer adopted the ready-made formula to visualize idea; in Stage 2, the designer manipulated the parameters while observing the 2D geometric modifications on screen; in Stage 3, the team identified the potential geometric shapes among the set of available ones and transferred the set into Rhino software for further manipulation based on visualizations. In the interviews, the coder’s tendency was to explain the process in simple terms without mentioning the details of how the formula was used to generate geometries. Following her explanation, in the analysis phase we were able to partially reconstruct how the formula was used to obtain edges, and then surfaces which were interpreted as architectural enclosures by the team members. The formula creates curvilinear continuous lines and when the coder changes the parameters within the code, the lines on edges re-shape and the system generates and adapts the surfaces between the lines. Each alteration in numerical values generates

Serendipitous Explorations in Distributed Work in Parametric Design

427

a new geometry on screen which helps the coder to reason with these “non-ordinary forms” on screen. 00:01:04 C1: Mathematica. Here we acquire forms like these. Then we take these forms… But non-ordinary forms… We see them, for instance [points at the screen], that moves here, I mirror this one, deform, and I can form something out of it. Then we export them to Rhino. In Rhino, according to architectural inputs, heights, slope, and solid-void…Say I don’t want to see that much facade… We develop the forms.

The coder searches and observes possible alternatives and save them for the next step involving formal manipulations of the digital model in Rhino Software. Hence, it actually gives another level of exploration opportunity to the designer while working on Rhino. Through this explorative process, it is not fully explicit when or how a designer identifies a certain iteration and further develop the formal qualities at a different platform; namely in Rhino. Following our observations, we claim that the architectural design criteria, involving structural and programmatic issues are also in play when designers observe and study the set of emerging geometries. Available geometries generated by the Mathematica Software were quickly assessed on the spot based on the hollow created through paraboloids. The assessment involved the opportunities for daylight penetration, generation of an architectural space, and affordances for a structural element to form an arcade system. The geometries created by the Mathematica Software (Fig. 1) were further assessed, qualified or eliminated by the coder before bringing the set into the larger team’s attention. Fig. 1 Alternative forms that are created in Mathematica. (Generated by the coder, C1)

428

L. Erbil Altintas et al.

4 Exploration P2: The towers The second case (P2: Tower Project) was different from the first one in the way the designers created the search space which was again explored through parametric modeling tools. The coder in this case (C2) started with developing an algorithm in Grasshopper. The coder created two circles as a base and a top and oriented his efforts to develop a surface between the base and top. In this episode, we have observed the generation and manipulation of the surface. The coder’s strategy was to create the surface with objects that are arrayed in an order. In order to control the array, Grasshopper’s plug-in Graph Mapper was used to generate the surfaces of the facade design. Graph mapper has been defined as “a two-dimensional interface with which we can modify numerical values by plotting the input along the Graph’s X Axis and outputting the corresponding value along the Y Axis at the X value intersection of the Graph” [20]. It is used for enabling the control of multiple objects’ rotations following a value range. Graph Mapper component is a function that can decreases output numbers proportionally and also modifies and cut-offs distance data according to the graph shape and range [21]. We have observed the coder in this case to follow five distinct steps to compute the facade design (Fig. 2). The coder conducts trials in order to develop the formula separately, to be applied on the building facade later on. Firstly, the coder draws circles which are narrowing from bottom to top (Step 1). Then the coder draws lines from the center to create the guides (Step 2), inserts regular grids to create a pattern (Step 3). Then the coder puts facade elements on the grids (left) and rotate each element (right) with Orient command in Grasshopper (Step 4). Then the coder multiplies the facade elements for each row as staggered with Pattern Stagger or Staggered Pattern; an operation by using related components in Grasshopper (Step 5). Within Step 4, the coder used the Orient command in Graph Mapper which required a value and range for the rotation of objects. In Step 5, the Graph Mapper was used to control the arrays of the objects on each row. Staggering the object by sliding the slider provided an exploratory setting for the designers. While using the slider tool, Graph Mapper provides bouncing the object according to sensibility that

Fig. 2 The coder’s formula generation about facade design

Serendipitous Explorations in Distributed Work in Parametric Design

429

is defined by the coder. For instance, if the coder defines the graph mapper sensibility between 0 to 1, the tool divides the path that the objects were arrayed along into 123 sections. However, the coder was trying to find more sensitive version of the slider’s movement by adjusting the range. Inserting the value and modifying the parameter through the sliders provides instant visualizations of the geometries. The exploratory setup was made available by the functionality and capacity of the software.

5 Key Steps in Exploration In the both cases, the tools enabled designers to work concurrently with the visualizations of the formulas. Hence, the office leader and the team leader had the opportunity to observe the visual translations of the formulas, and monitor the design development process. Another exploratory step for the P1 is transferring the object that is created in Mathematica. The P1 team transferred the geometry and initiated another manipulation process to generate a column by means of stretching the edges of the existing geometry. The curvilinear surfaces that were transferred into Rhino Software were further refined through manipulating the geometry on the digital model. The modifications aimed to generate a shell that covers the library to house the architectural program and to develop a structural system for the facility. The team’s strategy was to refine a single geometry in isolation, then multiplying the forms by adding, rotating, and mirroring in Rhino. The team preferred to 3D print all potential alternatives to keep track of their exploration space, and keep available alternatives in sight. The intention was to create a tangible archive for the project that they defined as the ‘research project’. By generating a 3D printed catalogue, the possibilities of serendipitous exploration were enhanced. The search space of alternatives in P2: The Tower Project was created in Grasshopper and visualized in Rhino as similar to P1 but the team of P2 modified and refined the design work in Grasshopper (Fig. 3). While changing the parameters in Grasshopper, the coder shifts the slider button as he observes the visual on the Rhino screen for instant manipulations. The team leader and the coder observe

Fig. 3 Facade trials with the pattern (elevation) (left). Towers without facade and towers with pattern applied facade (right). (Generated by the coder, C2)

430

L. Erbil Altintas et al.

the monitor and when they notice a potential alternative the coder pauses the slider and save the alternative available on screen. Eventually, the integrated system comprising of human and non-human members- generated a number of alternatives to be further assessed through other forms of representations. Using parametric modeling tools obviously supports designers to have multiple alternatives to be represented in various representational forms. These representational forms act as visual stimuli that are key elements in creative processes [12], and they can be considered as basis for serendipitous discovery. Expertise is related with number of alternatives a designer produce [14] but, the novice coders might create even more with parametric modeling tools than experts. Parametric modeling tools re-form, reproduce, and transform the alternative. The office leader evaluated the tools and stated: 00:01:47 OL: Now... this is not an imaginable thing. It is not a thing that happen with human mind. Maybe we can imagine, but this [computational design tools] develops it, reproduces it... that’s why we are using digital technology here.

Designers have a purpose in their mind while engaging in design problem whether ill-defined or well-defined [12]. Even the purpose is ambiguous or clear, designers intuitively search a solution back and forth between associative and analytic thinking which is related to creativity [22]. Although many alternatives are produced through computational tools, a ‘prepared mind’ [12] can evaluate alternatives and turn them into a creative serendipitous moments. The office leader in this case was in a position to monitor each alternative while the coder worked with parametric modeling tools. As the coder manipulates the parameters of the formula, the tool generates alternatives continuously and synchronously. The office leader needed to review every alternative that would occur in this process, so the coder started to record screenshots continuously.

6 Discussions and Conclusions This research establishes the significance of parametric modeling tools in propagation of knowledge in a distributed system, and in supporting serendipitous discovery. The emerging question in this research is that in the way parametric modeling tools are used in current architectural offices, design exploration proceeds by mutually supportive contributions of experts, who provide informed judgement, and novices, who are knowledgeable about coding and can thus produce many alternatives. Our claim is that parametric tools enable the production of multiple alternatives to facilitate serendipitous exploration. The parametric design tools ensure the production of many related alternatives almost simultaneously. A parametric definition determines a possible set of alternatives, therefore it could be considered as a generic design representation. The cases observed in this study offer the capacity of these tools in generating and visualizing the set of alternatives. Each visualized alternative on screen is a token of

Serendipitous Explorations in Distributed Work in Parametric Design

431

this generic representation. Once visualized, however, it becomes subject to further visual modifications which might trigger alterations in the parametric definition. The use of parametric tools makes the process transparent for all parties engaged, expert and novices, and enables instant input by participants as they observe and assess alternatives that are laid out as discrete representations. Having a number of visualizations on screen, then, allows the expert to select -in an opportunistic manner- visually satisficing alternatives to be further developed with architectural and structural constraints in mind. The cases presented in this paper offer situations of serendipitous explorations at two levels. The parametric tool employed in the first episode allowed the emergence of generic forms during the very early stages of design. The forms –tied to a ready-made formula– were created through a particular software. The geometries were instantly created on screen and intuitively assessed by the coder in-charge. We argue that the manipulation of numerical values and associated visuals facilitated the situations of serendipitous exploration in which the coder intended to discover “non-ordinary” formal articulations of the future architectural structures. The parametric tool –Mathematica in this case allowed easy and coordinated manipulation, visualization and propagation of the ready-made parametric equations. The firm could have gone further to modify the equation itself; however, we did not observe any instances of such manipulations. In the second case, on the other hand, the situations of exploration were observed mainly during the collective evaluation phase in which the team assessed the variety of alternatives on screen simultaneously. The team’s intention was to digitally visualize all available alternatives which were produced through a productive system comprising of human and non-human participants. The existence of numerous alternatives provided a large solution set which, again, ensured the situations of discovery through observing and assessing formal qualities of the evolving structure. The creation of the alternatives is held by the novice. The generation of alternatives has been said to be related to expertise [12], but knowing how to evaluate alternatives and being ‘prepared’ [11, 12] leads to a successful outcome. The novice and the parametric design tool together enable a more ambiguous exploratory setting in which each alternative is quickly produced and archived through the visualization capacity of the parametric tools employed. Designing with parametric design tools requires planning through generated algorithms in opposition to designing with sketching. Parametric design tools require a visual translation of the outcomes of the algorithm whereas sketching is already a visual representation. Parametric design provides enormous numbers of visual alternatives. Visual stimuli enhance creativity in architectural design [12, 23] so, parametric design tools might boost creativity just like sketches do but differently. Sketches primarily enable multiple interpretations through their ambiguous, and syntactically and semantically dense ordering [24]. Parametric modeling tools, on the other hand, enable designers to produce and evaluate multiple alternatives among which one is opportunistically picked by the designer. Parametric design presents an exploration space which opens a path for serendipitous discoveries.

432

L. Erbil Altintas et al.

In the distributed work setting, designers and tools worked collaboratively. The parametric tool has taken a participant role in the system as evoking creative ideas by exporting multiple variations as visual in the explorative setting. Although designers generate the parameters behind the design idea, they might not easily visualize the outcomes in their mind. Parameters can be defined in significant values but para-metric design tools give the opportunity of searching within the values. Parametric tools can become exploratory tools that might produce serendipitous outcomes more than designers by themselves. Acknowledgements We would like to express our thanks to Gokhan Avcioglu who accepted our request to conduct research at Global Architectural Development (GAD). We are also grateful to the design team members who agreed to participate in this study.

References 1. Navinchandra D (1991) Exploration and innovation in design: towards a computational model. Springer, New York 2. Kamprath M, Tassilo H (2019) Serendipity and innovation: beyond planning and experimental-driven exploration. In: Chen ABJ, Viardot E, Wong PK (eds) The Routledge companion to innovation management. Routledge, Abingdon, pp 343–360 3. Suwa M, Gero J, Purcell T (2000) Unexpected discoveries and S-invention of design requirements: important vehicles for a design process. Des Stud 21(6):539–567 4. Hutchins E (1995) Cognition in the Wild. MIT Press, Cambridge 5. Cross N (2006) Understanding design cognition. In: Cross N (ed) Designerly ways of knowing, pp 77–93 6. Lyon E (2011) Emergence and convergence of knowledge in building production: knowledge-based design and digital manufacturing. In: Kocatürk T, Medjdoub B (eds) Distributed intelligence in design: kocatürk/distributed intelligence in design. Wiley-Blackwell, Oxford, pp 71–98 7. Lyon E (2005) Autopoiesis and digital design theory: CAD systems as cognitive instruments. Int. J. Archit. Comput. 3(3):317–333 8. Kirsh D (2008) Distributed cognition: a methodological note. In: Dror IE, Harnad S (eds) Cognition distributed: how cognitive technology extends our minds. John Benjamins Publishing Company, Amsterdam, pp 57–69 9. Schön DA (1991) The reflective practitioner: how professionals think in action. Ashgate Publishing Limited, Great Britain 10. Maher ML, Poon J, Boulanger S (1996) Formalising design exploration as co-evolution. In: Gero JS, Sudweeks F (eds) Advances in formal design methods for CAD. Springer, Boston, pp 3–30 11. Martin K, Tassilo H (2019) Serendipity and innovation: beyond planning and experimental-driven exploration. In: Chen ABJ, Viardot E, Wong PK (eds) The Routledge companion to innovation management. Routledge, Abingdon, pp 343–360 12. Goldschmidt G (2014) Ubiquitous serendipity: potential visual design stimuli are everywhere. In: Gero JS (ed) Studying visual and spatial reasoning for design creativity. Springer, Virginia, pp 205–214 13. Martin D (2012) The cooperative use of material resources and contextual features in graphic design work. Des Stud 33(6):589–610

Serendipitous Explorations in Distributed Work in Parametric Design

433

14. Akin O (2001) Variants in design cognition. In: Eastman C, Newstetter W, McCracken M (eds) Design knowing and learning: cognition in design education. Elsevier, Oxford, pp 105– 124 15. Coates P (2010) Programming architecture. Routledge, New York 16. Sommer B, Sommer R (1997) Observation. In: Sommer B, Sommer R (eds) A practical guide to behavioral research tools and techniques. Oxford University Press, New York 17. Emerson RM, Fretz RI, Shaw LL (1995) Writing ethnographic fieldnotes. The University of Chicago Press, Chicago 18. Strauss A, Corbin JM (1998) Basics of qualitative research: techniques and procedures for developing grounded theory. SAGE Publications, London 19. Olsen C, Namara SM (2014) Collaborations in architecture and engineering. Taylor & Francis, New York 20. McNeel B, Davidson S (2015) The grasshopper primer. GitBook 21. Davidson S. Grasshopper: algorithmic modeling for rhino. https://www.grasshopper3d.com 22. Gabora L (2010) Revenge of the ‘neurds’: characterizing creative thought in terms of the structure and dynamics of memory. Creat Res J 22(1):1–13 23. Akin Ö (1990) Necessary conditions for design expertise and creativity. Des Stud 11(2):107– 113 24. Goel V (1995) Sketches of thought. The MIT Press, Cambridge

Design Cognition – 2

Does Empathy Beget Creativity? Investigating the Role of Trait Empathy in Idea Generation and Selection Mohammad Alsager Alzayed, Scarlett R. Miller, and Christopher McComb

Abstract The ability to understand and feel the needs and circumstances of others, also known as empathy, has been found to help engineering designers develop a deeper understanding of the design problems they solve. While prior work has examined the utility of empathic design experiences on driving creative concept generation, little is known about the role of a designer’s empathic tendencies in driving creative idea generation and selection in an engineering design project. Without this knowledge, we cannot be sure if, when, or how empathy influences the design process. Thus, the main goal of this paper was to identify the role of trait empathy in creative concept generation and selection in a humanitarian engineering design student project. In order to achieve this, a study was conducted with 103 first-year engineering students during three design stages of an 8-week design project (problem formulation, concept generation, and concept selection). The results from this research highlighted that empathic concern tendencies predicted the generation of more ideas. In addition, perspective-taking and fantasy tendencies negatively predicted the generation of more ideas. During concept selection, personal distress predicted participants’ propensity for the selection of useful ideas while empathic concern negatively predicted the selection of useful ideas. These results present some of the first evidence on the relationship between trait empathy and creativity in the concept generation and selection stages of the design process. Keywords Empathy


Idea generation


Creativity

S. R. Miller The Pennsylvania State University, University Park, PA, USA M. A. Alzayed Kuwait University, Kuwait City, Kuwait C. McComb (&) Carnegie Mellon University, PA Pittsburgh, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. S. Gero (ed.), Design Computing and Cognition'20, https://doi.org/10.1007/978-3-030-90625-2_26

437

438

M. A. Alzayed et al.

1 Introduction Over the past decade, engineering design research has seen a significant surge of the discussion of empathy [1], or one’s ability to understand and feel the needs and circumstances of others [2], due to its ability to help engineering designers develop a deeper understanding of the design problem [3]. Empathy may be particularly important in the early conceptual stages of the design process (i.e. problem formulation, concept generation and selection [4]) as it involves a designer’s attempt to “relate to [the user] and understand the situations and why certain experiences are meaningful to these [users]” ([5], pg. 67). Investing in these earlier conceptual stages can save costs and effort [6], as the success of a product can be linked to the early conceptual stages of the idea’s emergence [7]. Using design effectiveness measures, Genco et al. [8] and Johnson et al. [9] found that empathetic design experiences were an effective method to drive creative outcomes (i.e., originality and quality). In the same line of research, simulating extraordinary user scenarios was effective in enhancing engineering students’ empathic self-efficacy as well as the novelty, quantity, and variety of ideas generated by students [10]. While this prior work discussed the promising role of empathy in concept generation, studying the role of empathy in driving creative concept selection has been scarcely examined. This is problematic since researchers have identified concept selection as one of the most critical stages that determine successful engineering design [11]. While this prior research indicates empathy may be a potential driver of successful engineering design processes, other work [12] warns that empathic design techniques might place designers in the “empathy trap” by triggering popular directed reflections from the users instead of providing radical innovations to the existing problems [12]. Additionally, recent research has also identified that engineering faculty may feel that empathy was “a plus but … not what is really necessary to be a good engineer” ([13], pg. 149). In the same line of research, engineering students discussed the irrelevance of empathy in engineering due to the technical and analytical nature of the engineering discipline [14]. Taken as a whole, prior research provides conflicting interpretations on the role of empathy in design and the scarcity of research on the role of empathy in concept selection. Without this knowledge, we cannot be sure if, when, or how empathy is important in the design process. Therefore, the main goal of this paper was to identify the role of trait empathy in creative concept generation and selection in an engineering design student project. The results from this research provide some of the first evidence that establishes the relationship between trait empathy and creativity in the concept generation and selection stages of the design process.

Does Empathy Beget Creativity? …

439

2 Related Work In order to establish the framework for the current investigation, this section highlights prior work on (1) the role of empathy in the design process, and (2) measuring trait empathy.

2.1

The Role of Empathy in the Design Process

Over the past decade, empathy has been found to help engineering designers better understand the needs of users that are different from themselves [15, 16]. Specifically, prior work has shown that developing empathy can help develop a deeper understanding of the design problem [3] and the stakeholders [15] and encouraged an employment of a more targeted user research [16] during the problem formulation stage. Through semi-structured interviews with engineering students, Fila and Hess [14] found empathy to be related to problem contextualization and design inspiration. Using design effectiveness measures, Genco et al. [8] and Johnson et al. [9] found that empathetic design experiences have been found to be an effective method to drive creative outcomes (originality and quality) in the conceptual design stages. While that prior work found a relationship between empathy and creativity, researchers have found that the creativity of solutions generated by a designer can hinge on the nature of the design task [17], and the designer’s personal connection with the end-user [10]. Similarly, Hess and Fila [18] highlighted the impact of the context of the design problem in impacting design outcomes, which this work controlled for. While previous research has highlighted the effectives of empathic design techniques in the concept generation stages, engagement in empathic design experiences have also received criticism in the literature. For example, Mattelmäki, Vaajakallio, and Koskinen [12] were concerned that designers engaged in empathic design techniques might end up in the “empathy trap”; their attempt to be empathic might trigger popular directed reflections from the users instead of providing radical innovations to the existing problems [12]. This has been studied by Chung and Joo [19] that found that engaging designers with an empathic instruction task (watching a video on the end-user) decreased their concept evaluation scores, suggesting a “dark” side to empathy. Breithaupt [20] discusses some of the dark sides of empathy, empathic vampirism [21], where individuals might over-identify with others. In the context of design, that line of research suggests that the designer would end up designing for themselves if they over empathize [21]. While this prior work provides conflicting interpretations on the role of empathy in concept generation, studying the role of empathy in driving creative concept selection has been scarcely examined. This is problematic since researchers have identified concept selection as one of the most critical stages that determine successful engineering design [11, 22]. During this stage, designers narrow down the

440

M. A. Alzayed et al.

ideas generated during concept generation [4]. Studying designers’ creativity during concept generation solely is not representative of the designers’ creativity since generating creative ideas does not necessarily guarantee the final design’s creativity [22]. One way of assessing designers’ creativity in the concept selection stage is through their propensity for selecting creative ideas [23, 24]. Prior research by Toh and Miller [4, 25] identified that the cognitive skills used in concept selection are very different from the skills used during concept generation. For example, designers’ risk tolerance and team centrality have been found to impact designers’ creative concept selection [26], but not necessarily their creative concept generation. In the same line of research, Hay et al. [27] found that different design activities might require different working memory operators and reasoning processes based on the specific design goals [28]. While concept selection has been found to be an important component of creativity of the design process [29] that requires a different cognitive skillset that concept generation [27, 28], the relationship between empathic tendencies and concept selection has not been established. This existing research provides conflicting interpretations on the role of empathy in design and the scarcity of research on the role of empathy in concept selection. Thus, formalizing the role of an individual’s trait empathy in driving design outcomes in the concept generation and selection stages of the design process could bring great clarity to the existing research.

2.2

Measuring Trait Empathy

Trait empathy is “a social and emotional skill that helps us feel and understand the emotions, circumstances, intentions, thoughts, and needs of others such that we can offer sensitive, perceptive, and appropriate communication and support” [30]. Trait empathy can further be broken into a cognitive component and an affective component [31]. The cognitive component defines one’s empathy as dependent on the situation, while the affective component characterizes one’s empathy by the emotional response [31]. One of the widely used measures of trait empathy is Davis’ Interpersonal Reactivity Index (IRI) [32]. The IRI defines trait empathy with four empathic tendencies: (1) perspective taking measures the ability “to adopt the perspectives of other people and see things from their point of view ([32]) pg. 12; (2) fantasy measures “the tendency to transpose themselves imaginatively into the feelings and actions of fictitious characters in books, movies, and plays” ([32], pg. 12); (3) empathic concern measures “the degree to which the respondent experiences feelings of warmth, compassion and concern for the observed individual” ([32], pg. 12); and (4) personal distress measures an “individual's own feelings of fear, apprehension and discomfort at witnessing the negative experiences of others” ([32], pg. 12). While there are numerous methodologies for assessing trait empathy [32, 33], IRI is one of the few measures in the literature that encompasses both cognitive and affective components of empathy [31]. In engineering design, Hess and Fila [18]

Does Empathy Beget Creativity? …

441

argue that both components are needed to allow designers to better understand the end-users’ needs. While IRI has been used in prior work to assess the empathic tendencies of engineering students [34, 35], it has not been used in relation to creative concept generation and selection. Due to its rigorous development and acceptance in diverse communities of research, this study used IRI [32] to model designers’ trait empathy and examine its relationship with driving designers’ creative design outcomes.

3 Research Design and Methodology In light of this prior work, the main goal of this study was to determine if or how engineering student trait empathy impacts their ability to generate and select creative concepts in a humanitarian engineering design project, see Fig. 1 for a summary of the factors investigated. Specifically, the following research questions (RQs) were devised: 1. Can trait empathy be used to predict the number of ideas generated and the elegance, usefulness, and uniqueness of those ideas? It was hypothesized that higher trait empathy would be related to the number of ideas generated and the elegance, usefulness, and uniqueness of those ideas due to prior work with engineering graduate students that found that trait empathy was related to innovative self-efficacy [34]. 2. Can trait empathy be used to predict the propensity for selecting elegant, useful, and unique ideas? It was hypothesized that trait empathy would predict the propensity for selecting elegant, useful, and unique ideas due to prior work with engineering graduate students that found that trait empathy was related to their innovative self-efficacy [34].

Fig. 1 Factors studied in the current investigation

442

M. A. Alzayed et al.

The remainder of this section highlights the experimental procedure aimed at addressing those research questions.

4 Participants Participants were recruited from four different classroom sections of a first-year undergraduate engineering design course at a large Northeastern university. In all, 103 first-year engineering students (73 men and 30 women) participated in the study.

5 Procedure The study was completed over the course of an 8-week design project. Thus, the data presented here is part of a larger data collection effort geared at understanding the role of empathy in engineering design [36]. However, only the aspects of the study pertinent to the current investigation are described here, see Fig. 2. At the start of the semester, the researchers presented the study to each of the four sections of the course according to the Institutional Review Board guidelines set forth at the university. Participation in the study was voluntary, and informed consent was gathered prior to the start of the study. Participants were then divided into 3–4 member design teams by the course instructor in their respective sections, and they were assigned the eight-week design project. The project focused on addressing the United Nation’s Sustainable Development Goal 3 [37], which aims at “ensuring healthy lives and promoting well-being for all at all ages.” Specifically, teams were asked to select between four different design challenges, see [38] for details on the problem statements. While participants in all four sections were allowed to select from these four design challenges, the design context of these challenges varied across the sections. Specifically, two of the sections focused on designing for the developed world (n = 50 participants) while the remaining two sections were tasked with designing for the developing world (n = 53 participants).

Fig. 2 Timeline of the project

Does Empathy Beget Creativity? …

443

After introducing the project, the participants completed a survey that included their demographics and trait empathy. The participants then continued to work on the project per the timeline presented in Fig. 1. Specifically, in weeks 1–2, the participants were asked to conduct user research, formulate a problem statement, and create an empathy map. Additionally, the participants were asked to generate a set of customer needs and conduct an external benchmarking activity in Weeks 3–4. During the concept generation stage (Week 4), participants were involved in two brainstorming sessions: reverse brainstorming [39], where they were given 15 min to individually brainstorm bad ideas that would make the problem worse, and then individual brainstorming where they were asked to individually generate concepts for 20 min. During the concept selection stage (Week 5), participants were asked to individually select concepts using a concept screening matrix. Finally, in weeks 6– 8, participants were asked to prototype their solutions and report their final conceptual design. Of importance to the current study, participants were asked to complete the same Trait Empathy survey completed in week 1 of the study at the end of Week 4, immediately after the concept generation activity, and at the beginning of Week 5, immediately after the concept selection activity. Three measures were taken in order to avoid experimental biases: throughout the eight-week project, we did not explicitly use the word empathy in the classroom instruction or in any of the survey materials; the survey that assesses participants’ trait empathy was embedded with other surveys; and the surveys have not been labelled with the name of the scales [40].

6 Data Collection Instruments and Metrics This section summarizes the metrics used to explore the factors critical to achieving the research objectives.

6.1

Trait Empathy

Participants’ trait empathy was measured using the IRI, a 28-item survey answered on a 5-point Likert scale ranging from “does not describe me well” to “describes me very well”. The IRI includes the following 4 subscales, each made up of 7 different items: perspective taking, fantasy, empathic concern, and personal distress. Due to previous research that shows that trait empathy changes between the design stages (problem formulation, concept generation, concept selection) [41], we have tested the hypotheses with participants’ empathy at those different time points. The four-factor structure of the IRI has been validated [42] and has been implemented to assess individuals’ empathic tendencies [43, 44], including engineering students [34, 35]. A reliability analysis was conducted to evaluate the

444

M. A. Alzayed et al.

internal reliability of the subscales of the IRI, and a high Cronbach's a was observed [45] as 0.76 to 0.91 between all cases.

6.2

Number of Ideas Generated

The number of ideas was calculated for each participant by counting the number of idea sheets completed by each participant during the individual brainstorming session. This aligns with the quantity metric from the work of Shah, Vargas-Hernandez, and Smith [46].

6.3

Consensual Assessment Technique

In order to identify the effectiveness of the ideas generated, the Consensual Assessment Technique [47] (CAT) was used. CAT has been identified as a global measure of creativity [48] and has been widely used in prior research in the social sciences [49], education [50], and engineering design [51]. The method defines that an idea is creative when judges independently agree that it as creative [52]. The ideas were rated based on the following criteria: usefulness, uniqueness, and elegance using a six-point Likert Scale [53]. Specifically, (1) uniqueness relates to overall perceptions of how original and surprising the idea was [53], (2) usefulness relates to the overall perceptions of value, logic, and how understandable the ideas were, while (3) elegance refers to the idea’s “simplicity, insight shown, and conciseness of [the idea’s] presentation” ([53], pg. 288). Those metrics have been widely used in design research to assess ideation effectiveness in a design task [48, 54, 55]. In addition to the three metrics, experts’ ratings of the overall creativity of the idea was collected; however, this aspect was a focus of a later investigation, and hence we did not include this analysis in the current paper. The CAT utilizes experts to rate 20% of the complete idea set to provide a training set for quasi-experts to rate the remaining set based on the experts’ mindset in rating the ideas [48]. Specifically, two faculty members experienced in engineering design research were recruited to independently rate 20% of the ideas. Additionally, two quasi-experts (PhD candidate and third-year undergraduate student, both studying Industrial Engineering) were recruited to independently rate the 20% overlap of ideas to ensure agreement with the expert judges [56]. The average of the quasi-experts’ ratings had high agreement (a [ 0:75) [57] on each of the three metrics. Once inter-rater reliability was achieved, the two quasi-experts proceeded to rate the remaining 80% of the ideas independently and a high interrater reliability (a [ 0:75) [57] was achieved between the two quasi-expert raters for each of the three metrics. For each metric, an average of the scores from the two quasi-expert raters was calculated, as per recommendations by Silvia [58], see Figs. 3a and 3b for examples of CAT ratings.

Does Empathy Beget Creativity? …

445

Fig. 3a An idea from participant 61 that Fig. 3b An idea from participant 78 that received received a score of 1 on usefulness, 1 on a score of 4 on usefulness, 4.5 on uniqueness and 5 uniqueness and 1 on elegance on elegance

6.4

Propensity for Selecting Creative Ideas

To assess participants’ propensity for selecting creative concepts, we used the propensity toward creative concept selection metric, PC [23]. This metric was devised by Toh and Miller [23] and has been implemented in numerous design studies [24, 25]. In this metric, PC measures the “…tendency towards selecting (or filtering) creative concepts during the concept selection process” ([23], pg. 118). For example, the formula to calculate participants’ propensity towards selecting useful concepts (PUsefulness ) can be summarized as the following: PUsefulness ¼

average usefulness of selected concepts average usefulness of generated concepts

Similarly, participants’ propensity towards concept selection of ideas rated high in (1) uniqueness and (2) elegance was also assessed in the same manner. An individual can receive a value (PUsefulness ) greater than 1 if the average usefulness of the selected ideas is higher than the average usefulness of the available ideas, indicating a propensity for selecting useful ideas. A value on PUsefulness that is less than 1 indicated a participant’s aversion for selecting useful concepts [23]. Further details on the scoring methodology can be found in Toh and Miller’s work [23].

7 Results During the concept generation stage, participants developed a total of 806 ideas with an average of 8 ideas developed by each participant (SD = 3.53). In order to answer our research questions, statistical analyses were computed using SPSS 25.0, and a significance level of 0.05 was used in all analyses. The results are presented as mean ± standard deviation (SD), unless otherwise denoted.

446

M. A. Alzayed et al.

Prior to addressing our research questions, we first needed to identify if the design context impacted students creative concept generation and selection [10, 17] to determine whether all teams could be treated as a single group or if context-dependent treatments were necessary. Specifically, a set of independent samples t-test were computed with the independent variable being the number of ideas generated by each participant, average usefulness of ideas generated, average uniqueness of ideas generated, average elegance of ideas generated, propensity for selecting useful ideas, propensity for selecting unique ideas, propensity for selecting elegant ideas, and the dependent variable being the context of the design problem (developing, developed). Prior to running this analysis, assumptions were checked. Due to finding outliers, the analyses were conducted both with and without the outliers to identify their influence on the results. The outliers were found to have no significant impact on the significance of the results and therefore, the full analysis (with outliers) is presented here. In addition, normality was confirmed; however, the Levine’s Test for Equality of Variances revealed that the variety scores violated the assumption of homogeneity of variances, p < 0.05. Because of this, the Welch-Satterthwaite method was used to adjust the degrees of freedom [59]. The results of the t-test showed that the mean number of ideas produced by participants in developed world contexts (9.20 ± 4.050) was significantly higher than participants in developing world contexts (7.02 ± 2.454), a mean difference of 2.175 95% CI [0.845, 3.505], t (83.154) = -3.523, p = 0.002, Cohen’s d = 0.65. These results indicated that participants working on developed world projects generated more ideas than participants working on developing world projects. However, participants’ mean scores on usefulness, uniqueness and elegance were not significantly different between the two design contexts, p > 0.05, indicating that the design context did not impact the creativity of the ideas generated. Similarly, participants’ propensity for selecting useful, unique and elegant ideas were not significantly different between the two design contexts, p > 0.05, indicating that the design context did not impact participants’ propensity for selective creative ideas. Based on these results, subsequent analyses that address RQ1 and RQ2 do not combine these two groups when analyzing the number of ideas generated by participants, but do combine these two groups for the remaining analyses. RQ1: Can Trait Empathy be Used to Predict the Number of Ideas Generated and the Elegance, Usefulness, and Uniqueness of Those Ideas? The first research question was devised to assess whether participants’ trait empathy was related to the number of ideas generated and the elegance, usefulness, and uniqueness of those ideas. Based on prior work [34], it was hypothesized that higher trait empathy would be related to the number of ideas generated and the elegance, usefulness, and uniqueness of those ideas. Since we found that there were differences in the number of ideas between participants working on developing world sections compared to those working on developed world problems, separate statistical analyses were computed for participants in the two different design contexts.

Does Empathy Beget Creativity? …

447

First, in order understand the impact of trait empathy on the number of ideas generated by participants working on developing world projects, four multiple regression models were computed to predict number of ideas, from their empathic tendencies, perspective-taking, fantasy, empathic concern, and personal distress on the problem formulation and concept generation stages of the project respectively. This analysis was repeated for participants working on developed world projects. Prior to the analysis, statistical assumptions were checked. The results showed linearity of the independent variables as assessed by partial regression plots and a plot of studentized residuals against the predicted values. There was also independence of residuals, as evaluated by a Durbin-Watson statistic of 1.910. By visual inspection of a plot of studentized residuals, the assumption of homoscedasticity was met. There was no multicollinearity in the independent variables, as assessed by tolerance values greater than 0.1. There were no studentized deleted residuals greater than ± 3 standard deviations, no leverage values greater than 0.2, and no values for Cook’s distance above 1. Finally, normality was confirmed by visually inspecting the histograms and Q-Q plots. For participants working on developing world problems, the results showed that the number of ideas can be significantly predicted from participants’ trait empathy at concept generation, p = 0.008, R2 = 0.257 (see Table 1 for summary for contributing predictors), but not for problem formulation, p > 0.05. Specifically, this finding found that empathic concern tendencies encouraged the generation of more ideas while perspective-taking and fantasy tendencies discouraged the generation of more ideas for participants working on developing world contexts. Meanwhile, for participants working on developed world problems, the results showed that the number of ideas cannot be significantly predicted from participants’ trait empathy at problem formulation, p > 0.05, or concept generation, p > 0.05. Second, in order to understand the impact of participants’ trait empathy on their average scores on uniqueness, usefulness and elegance, 6 multiple regression models were computed to predict participants’ average scores on uniqueness, usefulness and elegance, from their empathic tendencies, perspective taking, fantasy, empathic concern, and personal distress on problem generation and concept generation stages of the study respectively. While we found that there were differences in the number of ideas based on the design context, there were no differences based on participants’ average usefulness, uniqueness, and elegance.

Table 1 Summary statistics of the regression model on the relationship between the number of ideas and trait empathy at problem formulation for participants working on developing world problems

Factor

B

SE

b

p

Perspective-taking −0.324 0.136 −0.376 0.021* Fantasy 0.012 0.102 0.017 0.906 Empathic concern 0.374 0.106 0.508 0.001* Personal distress −0.326 0.114 −0.420 0.007* Note: B represents the unstandardized coefficient; SE represents the standard error associated with that coefficient; b is the standardized coefficient; p is the significance value associated with each factor

448

M. A. Alzayed et al.

Therefore, separate models were not necessary. The results from the linear regression models did not significantly predict participants’ usefulness, uniqueness and elegance from their four empathic tendencies at problem formulation, p > 0.05, or concept generation, p > 0.05, indicating that participants’ empathic tendencies did not predict their creative concept generation. These results partially support our hypothesis; the results showed that trait empathy in our sample predicted the number of ideas generated, but it did not predict the creativity of those ideas as measured through the CAT. Specifically, the results showed that for participants working in a developing world context, empathic concern predicted the generation of more ideas while perspective-taking and fantasy negatively predicted the generation of more ideas. This finding corroborates a qualitative investigation with engineering students [14] that found that empathic concern tendencies motivated students to work harder on an engineering task. On the other hand, the results indicated that personal distress and perspective-taking negatively predicted participants’ number of ideas. This finding alludes to the discussion in the literature on the dark sides of empathy [20] (also known as the empathy trap [12]), where being empathic has been hypothesized to restrict a designer from coming up with creative innovations to a design problem [12]. RQ2: Can Trait Empathy be Used to Predict the Propensity for Selecting Elegant, Useful, and Unique Ideas? While the first research question explored the relationship between participants’ trait empathy and their creative concept generation, the second research question was devised to assess whether participants’ trait empathy was related to their creative concept selection. Based on prior work [34], it was hypothesized that trait empathy would predict the propensity for selecting elegant, useful, and unique ideas. In order to understand the impact of participants’ trait empathy on their propensity for selecting unique, useful, and elegant ideas, 9 multiple regression models were computed to predict participants’ propensity for selection ideas that are rated high in usefulness, uniqueness, and elegance from their empathic tendencies, perspective taking, fantasy, empathic concern, and personal distress from problem formulation, concept generation, and concept selection. The results from the regression models did not significantly predict participants’ propensity for selecting unique and elegant ideas from their four empathic tendencies at problem formulation, p > 0.05, concept generation, p > 0.05, or concept selection, p > 0.05. However, the results showed that participants’ propensity for selecting useful ideas can be predicted from their trait empathy from problem formulation, p = 0.026, R2 = 0.118 (see Table 2 for summary for contributing predictors), but not their trait empathy from concept generation, p > 0.05, or concept selection, p > 0.05. Specifically, the results indicated that personal distress positively predicted the selection of useful ideas while empathic concern negatively predicted the selection of useful ideas. While empathic concern tendencies were helpful in encouraging designers to generate more ideas, empathic concern actually hindered designers’ preference for highly useful ideas in the selection process. This dichotomy stands in support of

Does Empathy Beget Creativity? … Table 2 Summary statistics of the regression model on the relationship between the propensity of selecting useful ideas & trait empathy from problem formulation

Factor

449 B

SE

b

p

Perspective-taking −0.001 0.004 −0.376 0.889 Fantasy 0.004 0.003 0.017 0.181 Empathic concern −0.009 0.004 0.508 0.022* Personal distress 0.008 0.003 −0.420 0.024* Note: B represents the unstandardized coefficient; SE represents the standard error associated with that coefficient; b is the standardized coefficient; p is the significance value associated with the factor

prior research [4, 25] that identified that the cognitive skills used in concept selection are very different from the skills used during concept generation. The results confirm prior work that discussed varying points of views [8, 9, 20] on the role of empathy in design, whereby we find evidence that supports the notion of the utility of empathy and the negative impact of empathy in the design process.

8 Discussion The main goal of this paper was to identify the role of trait empathy in creative concept generation and selection in an engineering design student project. The first major finding from this study was that empathic concern tendencies positively predicted the generation of more ideas while perspective-taking and personal distress tendencies negatively predicted the generation of more ideas, for participants working on developing world projects. The second major finding was that personal distress tendencies positively predicted the selection of useful ideas while empathic concern tendencies negatively predicted the selection of useful ideas. These results are discussed in the following sections in relation to the research questions.

8.1

The Relationship Between Trait Empathy and Concept Generation

The first finding from the study indicated that trait empathy predicted the number of ideas generated by participants but not necessarily the creativity of those ideas. Specifically, for participants working on developing world contexts, empathic concern tendencies encouraged the generation of more ideas while perspectivetaking and fantasy tendencies discouraged the generation of more ideas. These findings indicated that empathic concern, “the degree to which the respondent experiences feelings of warmth, compassion and concern for the observed individual” ([32], pg. 12), might be of more utility in terms of generating more ideas. This finding corroborates a qualitative investigation with engineering students [14]

450

M. A. Alzayed et al.

that found that empathic concern tendencies motivated students to work harder on an engineering task. This finding warrants future research that could empirically assess the relationship between trait empathy and designers’ motivation. On the other hand, the findings from our study indicated that personal distress and perspective-taking tendencies negatively predicted participants’ number of ideas. This finding relates to findings in the literature that note how being empathic may restrict the designer from coming up with creative innovations to the existing problem [12]. This phenomenon is commonly referred to as the dark side of empathy [20] or the empathy trap [12].

8.2

The Relationship Between Trait Empathy and Concept Selection

While the first finding showed that the empathic concern encouraged designers to generate more ideas, the second finding from the study indicated that empathic concern discouraged the selection of ideas rated high in usefulness. Meanwhile, personal distress tendencies positively predicted participants’ propensity for selecting useful ideas while it was not helpful in concept generation. This dichotomy resonates with prior research by Toh and Miller [4, 25] that identified that the cognitive skills used in concept selection are different from the skills used during concept generation. Overall, the results from this study confirmed prior work that discussed varying points of views [8, 9, 20] on the role of empathy in engineering design, whereby we find evidence that supports the notion of the utility of empathy and the negative impact of empathy in both the concept generation and selection stages of the design process. Specifically, these results suggest that, while empathy may be useful throughout design, the utility of specific types of empathy vary depending upon the design stage. These findings confirm previous research [27] that found that different design activities might require different working memory operators and reasoning processes based on the specific design goals [28]. However, the results warrant future research that should assess the relationship of trait empathy with other design outcomes (e.g., the quality of the final design [46]).

8.3

Implications for Design Theory and Practice

The current literature is divided between discussing the positive [8, 9] and negative [12–14] impacts of empathy in design, in addition to being invested in devising empathy invoking interventions [8, 9]. This research adds to this body of

Does Empathy Beget Creativity? …

451

knowledge by suggesting that, while empathy may be useful throughout design, the utility of specific types of empathy vary depending upon the design stage. In other words, the design community should be invested in preparing specific interventions to trigger specific types of empathic tendencies (e.g. perspective-taking, fantasy, empathic concern, or personal distress) depending on the design stage (e.g. concept generation, concept selection) and the desired outcome (e.g. useful, unique, or elegant ideas). Overall, this study took the first step towards our goal of encouraging empirical investigations aimed at understanding the role of trait empathy across different stages of the design process.

9 Conclusions and Future Work The main goal of this study was to understand the role of trait empathy on creative concept generation and selection in an engineering design student project. The results from this research highlighted that empathic concern tendencies predicted the generation of more ideas while perspective-taking and fantasy tendencies negatively predicted the generation of more ideas. During concept selection, personal distress predicted participants’ propensity for the selection of useful ideas while empathic concern negatively predicted the selection of useful ideas. The results from this research suggest that, while empathy may be useful throughout design, the utility of specific types of empathy vary depending upon the design stage. However, there are several limitations that can lead to exciting avenues for future research. While this work began exploring the relationship between trait empathy and creative concept generation and selection, future research should assess the relationship of trait empathy with other design outcomes, such as the overall creativity of the ideas, or the quality of the final design. Moreover, while it is known that ideation patterns of first-year and senior-level students differ [60], this work only studied first-year students. Finally, while this research explored the utility of empathy in humanitarian engineering problems, future research is needed to extend these results with other engineering design tasks. Overall, the findings from this paper present some of the first evidence on the relationship between different components of trait empathy in driving creative design outcomes across the concept generation and selection stages of the design process. Acknowledgements We thank Kuwait University for funding the doctoral fellowship of Mohammad Alsager Alzayed. The authors are also grateful to Dr. Daryl Cameron for his help on the project. We would also like to acknowledge the help of undergraduate research assistants Abby O’Connell and Lois Jung.

452

M. A. Alzayed et al.

References 1. Tang X (2018) From’empathic design’to’empathic engineering’: toward a genealogy of empathy in engineering education 2. Ioannidou F, Konstantikaki V (2008) Empathy and emotional intelligence: What is it really about? Int J Caring Sci 1(3):118 3. Walther J, Miller SE, Kellam, NN (2012) Exploring the role of empathy in engineering communication through a transdisciplinary dialogue. In: Proc. 119th ASEE annual conference and exposition, American society for engineering education 4. Toh CA, Miller SR (2016) Choosing creativity: the role of individual risk and ambiguity aversion on creative concept selection in engineering design. Res Eng Design 27:195–219 5. Battarbee K (2004) Co-experience: understanding user experiences in interaction. Aalto University 6. Mattson CA, Messac A (2005) Pareto frontier based concept selection under uncertainty, with visualization. Optim Eng 6(1):85–115 7. Goldenberg J, Lehmann DR, Mazursky D (2001) The idea itself and the circumstances of its emergence as predictors of new product success. Manage Sci 47(1):69–84 8. Genco N, Johnson D, Holtta-Otto K, Seepersad CC (2011) A study of the effectiveness of the empathic experience design creativity technique. In: ASME IDETC design theory and methodology conference, Paper number: DETC2011–021711, Washington, DC 9. Johnson DG., Genco, N, Saunders, MN, Williams P, Seepersad, CC, Hölttä-Otto K (2014) An experimental investigation of the effectiveness of empathic experience design for innovative concept generation. J Mech Des, 136(5):051009 10. Raviselvam S, Sanaei R, Blessing L, Hölttä-Otto K, Wood KL (2007) Demographic factors and their influence on designer creativity and empathy evoked through user extreme conditions. In: Proc. ASME 2017 international design engineering technical conferences and computers and information in engineering conference, American society of mechanical engineers, pp. V007T006A011–V007T006A011 11. Pugh S (1996) Creating innovative products using total design. Addison-Wesley Longman Publishing Co., Inc, Boston, MA 12. Mattelmäki T, Vaajakallio K, Koskinen I (2014) What happened to empathic design? Des Issues 30(1):67–77 13. Strobel J, Hess J, Pan R, Wachter Morris CA (2013) Empathy and care within engineering: qualitative perspectives from engineering faculty and practicing engineers. Eng Stud 5 (2):137–159 14. Fila ND, Hess JL (2016) In their shoes: Student perspectives on the connection between empathy and engineering. American Society for Engineering Education 15. Schmitt E, Morkos B. Teaching Students Designer Empathy in Senior Capstone Design 16. Gray CM, Yilmaz S, Daly SR, Seifert CM, Gonzalez R (2015) Idea generation through empathy: Reimagining the ‘cognitive walkthrough’ 17. Starkey E, Toh CA. Miller SR (2016) Abandoning creativity: The evolution of creative ideas in engineering design course projects. Design Studies 18. Hess JL, Fila ND. The development and growth of empathy among engineering students. American Society for Engineering Education 19. Chung J, Joo J (2017) Effect of empathy on designers and non-designers in concept evaluation. Arch Des Res 30(3):57–70 20. Breithaupt F (2019) The dark sides of empathy. Cornell University Press 21. Breithaupt F (2018) The bad things we do because of empathy. Interdisc Sci Rev 43(2):166– 174 22. Rietzschel E, Nijstad BA, Stroebe W (2010) The selection of creative ideas after individual idea generation: choosing between creativity and impact. Br J Psychol 101(1):47–68 23. Toh CA, Miller SR (2015) How engineering teams select design concepts: a view through the lens of creativity. Des Stud 38:111–138

Does Empathy Beget Creativity? …

453

24. Zheng X, Ritter SC, Miller SR (2018) How concept selection tools impact the development of creative ideas in engineering design education. J Mech Des 140(5):052002 25. Toh CA, Miller SR (2016) Creativity in design teams: the influence of personality traits and risk attitudes on creative concept selection. Res Eng Des 27(1):73–89 26. Heininger K, Chen HE, Jablokow K, Miller SR (2018) How engineering design students’ creative preferences and cognitive styles impact their concept generation and screening. In: Proc. ASME 2018 international design engineering technical conferences and computers and information in engineering conference, American society of mechanical engineers, pp. V007T006A032–V007T006A032. 27. Hay L, Duffy AH, McTeague C, Pidgeon LM Vuletic T, Grealy, M (2017) A systematic review of protocol studies on conceptual design cognition: Design as search and exploration. Des. Sci. 3:1–36 28. Stauffer LA, Ullman DG (1991) Fundamental processes of mechanical designers based on empirical data. J Eng Des 2(2):113–125 29. Rietzschel EF, Nijstad BA, Stroebe W (2006) Productivity is not enough: a comparison of interactive and nominal brainstorming groups on idea generation and selection. J Exp Soc Psychol 42(2):244–251 30. McLaren K (2013) The art of empathy: a complete guide to life's most essential skill. Sounds True 31. Duan C, Hill CE (1996) The current state of empathy research. J Couns Psychol 43(3):261 32. Davis MH (1980) A multidimensional approach to individual differences in empathy 33. Baron-Cohen S, Wheelwright S (2004) The empathy quotient: an investigation of adults with Asperger syndrome or high functioning autism, and normal sex differences. J Autism Dev Disord 34(2):163–175 34. Surma-aho AO, Bj¨orklundKatja TA, Holtta-Otto K (2018) Assessing the development of empathy and innovation attitudes in a project-based engineering design course. In: 2018 ASEE annual conference & exposition 35. Hess JL, Fila ND, Purzer S (2016) The relationship between empathic and innovative tendencies among engineering students. Int J Eng Educ 32(3):1236–1249 36. Alzayed MA (2019) An exploration of the role of student empathy in engineering design education. In: Proc. 2019 IEEE frontiers in education conference (FIE), IEEE, pp. 1–2 37. Nam UV (2015) Transforming our world: The 2030 agenda for sustainable development. Division for Sustainable Development Goals: New York, NY, USA. 38. 2020 Problem Statements - Sustainable Development Goal 3. https://doi.org/10.6084/m9. figshare.11825748.v1 39. Hagen M, Bernard A, Grube E (2016) Do it all wrong! using reverse-brainstorming to generate ideas, improve discussions, and move students to action. Manag Teach Rev 1(2):85– 90 40. Cash P, Culley S (2015) The role of experimental studies in design research. In: The Routledge companion to design research, RoutledgeFalmer, pp. 175–189 41. Alsager Alzayed M, McComb C, Menold J, Huff J, Miller S, (2020) Are you feeling me? An exploration of empathy development in engineering design education. J Mech Des (In review) 42. Davis MH (1983) Measuring individual differences in empathy: evidence for a multidimensional approach. J Pers Soc Psychol 44(1):113 43. Gilet A-L, Mella N, Studer J, Grühn D, Labouvie-Vief G (2013) Assessing dispositional empathy in adults: a French validation of the Interpersonal Reactivity Index (IRI). Can J Behav Sci 45(1):42 44. Péloquin K, Lafontaine M-F (2010) Measuring empathy in couples: validity and reliability of the interpersonal reactivity index for couples. J Pers Assess 92(2):146–157 45. Cronbach LJ (1951) Coefficient alpha and the internal structure of tests. Psychometrika 16 (3):297–334 46. Shah J, Kulkarni S, Vargas-Hernandez N (2000) Evaluation of idea generation methods for conceptual design: effectiveness metrics and design of experiments. J Mech Des 122:377–384

454

M. A. Alzayed et al.

47. Amabile TM (1983) A consensual technique for creativity assessment, The social psychology of creativity. Springer, New York, pp 37–63 48. Cseh GM, Jeffries KK (2019) A scattered CAT: A critical evaluation of the consensual assessment technique for creativity research. Psychol Aesthet Creat Arts 13(2):159 49. Chen C, Kasof J, Himsel AJ, Greenberger E, Dong Q, Xue G (2002) Creativity in drawings of geometric shapes: a cross-cultural examination with the consensual assessment technique. J Cross Cult Psychol 33(2):171–187 50. Kaufman JC, Baer J, Cole JC, Sexton JD (2008) A comparison of expert and nonexpert raters using the consensual assessment technique. Creat Res J 20(2):171–178 51. Christiaans H, Venselaar K (2005) Creativity in design engineering and the role of knowledge: modelling the expert. Int J Technol Des Educ 15(3):217–236 52. Amabile T (1982) Social psychology of creativity: a consensual assessment technique. J Pers Soc Psychol 43(5):997–1013 53. Besemer SP, O’Quin K (1999) Confirming the three-factor creative product analysis matrix model in an American sample. Creat Res J 12(4):287–296 54. Klein C, DeRouin R, Salas E (2006) Uncovering workplace interpersonal skills: a review, framework, and research agenda. Int Rev Ind Organ Psychol 21(1):79–126 55. Buelin-Biesecker J, Wiebe E (2013) Can pedagogical strategies affect students’ creativity? Testing a choice-based approach to design and problem-solving in technology, design, and engineering education. In: Proc. Proceedings of the 2013 American society for engineering education annual conference & exposition 56. Landis JR, Koch GG (1977) The measurement of observer agreement for categorical data. Biometrics 159–174 57. Koo TK, Li MY (2016) A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 15(2):155–163 58. Silvia PJ (2011) Subjective scoring of divergent thinking: examining the reliability of unusual uses, instances, and consequences tasks. Think Skills Creat 6(1):24–30 59. Zimmerman DW, Zumbo BD (1992) Parametric alternatives to the Student t test under violation of normality and homogeneity of variance. Percept Mot Skills 74(3):835–844 60. Alzayed MA, McComb C, Hunter ST, Miller SR (2019) Expanding the solution space in engineering design education: a simulation-based investigation of product dissection. J Mech Des 141(3):032001

How Do Designers Think in Systems? – Empirical Insights from Protocol Studies of Experienced Practitioners Designing Product-Service Systems Abhijna Neramballi, Tomohiko Sakao, and John S. Gero

Abstract Conceptual Product-Service System (PSS) designing is expected to be inherently different and more complex than designing its discrete product or service elements. This paper presents empirical insights from 10 protocol studies of experienced practitioners, conceptually designing a PSS in a laboratory setting. A coding scheme based on function-behavior-structure ontology was used to capture the distributions of the designers’ cognitive effort on design issues and processes. Two further coding schemes were used to capture the cognitive effort expended on different levels of a systems hierarchy and in the cognitive design spaces of PSS, product and service, as well as the interactions between them. The results derived from the analyses indicate that the practitioners expend the most cognitive effort on the design issue of behavior and on the process of evaluation. They focus most on the element level of the systems hierarchy and spend the most time in the product design space.

1 Introduction In response to intensifying market competitiveness [1] and growing customer demands for personalised solutions [2], manufacturing companies are increasingly seeking to design and offer integrated bundles of products and services that together deliver value. Services in this context can include remote monitoring, maintenance and repair of products, user training and online customer support ([3–5]). The market proposition of service integrated jet engine solutions offered by Rolls-Royce for the aerospace industry ([3, 6]) is a well-known example of such an integrated offering. This type of integrated offering is widely known as product/service

A. Neramballi (&)
T. Sakao Linköping University, Linköping, Sweden e-mail: [email protected] J. S. Gero University of North Carolina at Charlotte, Charlotte, USA © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. S. Gero (ed.), Design Computing and Cognition'20, https://doi.org/10.1007/978-3-030-90625-2_27

455

456

A. Neramballi et al.

systems (PSSs), and are described as combinations of tangible products and intangible services that are designed to jointly address specific customer needs [7]. Previous research indicates that during the conventional development of PSSs in industry, product and service designing are temporally and spatially detached [8]. Consequently, the cognitive design spaces of product and service elements are separated during the conceptual design phase. As a result of this separation, designers cannot effectively consider the potential mutual interdependencies of product and service elements [5]. Thus, this way of sequentially designing products and services is considered to be unsuitable for the effective development of PSSs [ibid]. During PSS designing a broader systems perspective needs to be applied, considering the influences of various aspects such as business models, socio-cultural factors, users and other stakeholders ([9–12]). Furthermore, the constituent product and service elements are prescribed to be integrated from a systems perspective [10], based on their respective exchangeability [13]. For example, the durability of a physical product element can be enhanced by introducing sensors to accommodate the effective design of relevant service elements, such as remote monitoring and maintenance of the product. Their respective design spaces need to be integrated during conceptual PSS designing, as depicted in Fig. 1, to facilitate interactions between the product and service design elements. The integration of product and service design spaces containing inherently distinct elements with heterogenous characteristics such as tangibility and intangibility or longevity and perishability, respectively, is expected to increase the complexity of the design process ([9, 14]). Currently, there is limited empirical insight into how the expected increase in complexity of the PSS design space might influence the designers’ thinking. Thus, the focus of this paper is on investigating and describing the cognitive nature of conceptual PSS designing.

Fig. 1 Illustration of the PSS design space

How Do Designers Think in Systems? …

457

2 Past Research Past empirical studies have reported insights into PSS designing in industry from operational and organizational perspectives through analysing cases ([8, 15, 16]). Although these studies have contributed to qualitative insights, they do not provide reproducible results and quantitative insights into how conceptual PSS design is performed. Due to this lack of scientific knowledge, most of the available PSS design support tools and methods in the state of the art are based on the conventional approaches of designing products or services ([1, 10]). More empirical and quantitative research is required, for instance, to test the validity of existing theories or models of conceptual PSS design activities. Quantitative and commensurable approaches to analyse and characterise the cognitive nature of conceptual designing in general are available ([17, 18]). Several past studies have utilised these approaches to provide substantial insights into product designing ([19–22]). In contrast, only a handful of past studies have carried out quantitative investigations to analyse the cognitive nature of conceptual PSS designing ([23–27]). The approaches used in these studies are exclusively characteristic of PSS designing. Although the results of these studies are insightful, they are not commensurable with, for example, the results of conceptual product designing. Commensurable results are crucial to characterise the nature of conceptual PSS designing. A previous work [28] provided some empirical and quantitative insights into the cognitive nature of conceptual PSS designing, in terms of cognitive effort expended by designers on different design issues and processes. However, there is little information on the distributions of cognitive effort expended by designers during conceptual PSS designing, on the different levels of hierarchy of the applied systems perspective. Similarly, there is little information on how designers decompose and recompose design problems during conceptual PSS designing. Furthermore, there is limited scientific understanding regarding the interactions between the different design spaces of products, services and the PSS. Consequently, there is a lack of scientific knowledge concerning the effects on the designers’ cognition, introduced by the heterogeneity of the inherently distinct design elements within the design space during conceptual PSS designing. This missing knowledge is crucial to develop a comprehensive understanding of PSS designing and designing of other systems with similar complexity. This paper aims to address some of these gaps in our knowledge.

3 Aims This paper, to bridge these knowledge gaps, aims to provide empirical insights into the distributions of cognitive effort expended by designers during conceptual PSS designing, on different design issues, design processes, hierarchical levels of

458

A. Neramballi et al.

systems abstraction, problem decomposition and recomposition. This work also aims to investigate and describe how the design spaces of products, services and PSSs interact during conceptual PSS designing. This paper builds on previous work [28] with additional data and frames of analyses. The paper operationalizes the aim through the following research questions: • RQ1: What are the distributions of cognitive effort expended on different design issues and processes during conceptual PSS designing? • RQ2: What are the distributions of cognitive effort expended on the different hierarchical levels of systems perspective applied during conceptual PSS designing? • RQ3: How do designers decompose and recompose design problems during conceptual PSS designing? • RQ4: How do designers distribute their cognitive effort on and between the various design spaces of PSSs? The rest of this paper is laid out as follows. The next section briefly introduces the significance of this research before presenting the methods used to investigate and answer the research questions. Subsequently, the results of the research are presented before discussing the results and drawing conclusions.

4 Significance The industrial transition towards the design and propagation of PSSs is reported to introduce several benefits to a wide range of stakeholders such as consumers, providers and the society [29–31]. Also, it is expected to have the potential to extend the useful life of products and reduce the use of materials [32]. Thus, this transition is widely considered as an effective approach to enhance resource efficiency of society [33]. Consequently, there is an increasing demand from manufacturing companies for dedicated tools and methods that support PSS designing ([10, 34]). In order to develop the required support for effective PSS designing, especially during the conceptual phase, a scientific understanding of how experienced practitioners carry out such a design process is needed. Thus, the implications of the knowledge derived from this research include i) guidance for the development of effective design support methods or tools for conceptual PSS designing, and ii) support for the effective development of pedagogy for PSS designing.

How Do Designers Think in Systems? …

459

5 Research Design 5.1

Research Method

Protocol analysis is used to extract empirical data from verbal reports of thought sequences [35]. More specifically, the verbal reports are collected using the think aloud protocol. This protocol is widely utilized across different disciplines to model cognitive processes [36]. An approach to investigate conceptual design processes using protocol analysis was initially proposed by Gero and Mc Neill [17]. Since then, protocol analysis has been widely utilized to study conceptual design processes across different domains of designing [22]. This approach can provide both quantitative and qualitative insights into conceptual designing, thus justifying its use in this research. The empirical data for protocol analysis is derived from multiple design sessions involving experienced practitioners, conceptually designing a PSS in a laboratory setting. The features of the design sessions are identical to the previous work [28], and half of the data sets is derived from the same study.

5.2

Design of Experiment

The protocol data was collected from 10 design sessions, involving a cohort of 20 experienced practitioners. The participants worked for different Swedish manufacturing companies operating in multiple industrial sectors. The practitioners had an average working experience of 11.25 years with a standard deviation of 5.99 years. These participants have a background in product, and/or service development, and/or environmental engineering. Out of the 20 participants, 8 were female and 12 were male. The design sessions were carried out in a laboratory setting. This type of setting was chosen for the following reasons. The first was to circumvent the practical issues of shadowing the practitioners in real industrial settings, such as corporate confidentiality, organizational issues and the varying nature of the practitioners’ everyday work. Second, product and service design activities are often carried out sequentially and in silos in industry [8], and simultaneous and integrated product and service design is hindered significantly. Third, a laboratory setting can produce a sufficient number of sessions enabling statistical analysis. Therefore, a laboratory setting, despite its limitations regarding the ability to capture the complexities of industrial reality, was chosen over an industrial setting. The practitioners were from the companies in the consortium of the program for this research and participated in the experiments on a voluntary basis and were given feedback by the researchers regarding their analyzed activities, at a later date. The participants in the design sessions were grouped into pairs to form teams. Each team included one experienced practitioner playing the role of a product designer and the other the role of a service designer. These roles were assigned

460

A. Neramballi et al.

based on the nature of their work experience. The objective of the design task was to conceptually design a natural resource-efficient PSS offering, based on an existing offering. The chosen PSS offering is an office use coffee machine and related services. For more details regarding the design task, refer to [28]. Each design session used audio and visual recordings to collect the think aloud protocol data. One video camera and one audio recorder were used for each design session. The data collected was later transcribed as chunks of conversations between the designers known as utterances. The utterances from the transcription were later broken down into smaller units known as segments. This was done to ensure that each segment can be assigned only one code, from one or more coding schemes [37]. The segmented data was later coded by two individual coders and arbitrated by a third coder. The following frameworks for coding the collected protocol data were utilized.

6 Function – Behavior – Structure (FBS) Ontology A coding scheme based on the widely accepted function-behavior-structure (FBS) ontology [18] is applied to analyze the conceptual PSS design protocols. It is used to characterize the cognitive nature of conceptual PSS designing. More specifically, it is used to analyze the distributions of cognitive effort on various design issues and design processes. This coding scheme can produce commensurable results across different domains of designing, as the FBS ontology is not dependent on the nature of the design task, the design environment or the experience of the designers [38, 39]. Its applicability in the PSS context was illustrated in previous work [28]. The FBS ontology [18] is a widely used knowledge representation scheme for designing. It represents designing in terms of the following design issues (adopted from [18, 40]: Requirements (R) represents the (existing or potential) customer needs or demands from the artefact being designed that are explicitly communicated to the designers. Function (F) represents the teleological expressions (purposes) of the artefact being designed that are generated by the designers based on their interpretations of the design task. Expected behavior (Be) represents the characteristics expected by the designers from the interactions of the artefact with the environment. Behavior from Structure (Bs) represents the characteristics of the artefact that can be measured or derived from design solutions for the artefact and their interactions with the environment. Structure (S) represents the components of the artefacts and their relationships. Description (D) refers to any design-related writings or drawings produced by the designers. As the designers carry out their task, they move from one design issue to the other. The transitions between these issues are considered as design processes. The commonly occurring design processes are described in detail in [18]. This coding scheme is applied to investigate and answer the first research question.

How Do Designers Think in Systems? …

6.1

461

Levels of Systems Hierarchy

A coding scheme based on relevant previous works is utilized to capture the cognitive effort of designers on the different levels of systems hierarchy [17, 19]. This coding scheme has three levels of systems hierarchy. The first level is the systems level (Code 1) and represents the highest level of abstraction, in terms of the designer’s cognition. In this level, the designer considers the system as an integral whole. The second level is the subsystems level (Code 2) and represents an intermediate level of abstraction, in terms of the designer’s cognition. In this level, the designers focus on the various subsystems within the system. The third and lowest level of abstraction is the element level (Code 3). In this level, designers focus their cognitive effort only on individual elements of the entire system or subsystems. The different levels of this coding scheme are illustrated in Fig. 2. When the level of the design problems being discussed by the designers transitions from a higher level to a lower level (e.g., from Level 2 to Level 3), it is considered as problem decomposition, while transitions from lower levels to higher levels are considered as problem recomposition [19]. This coding scheme is applied to investigate and answer the second and third research questions.

Fig. 2 Illustration of the coding scheme to capture the different levels of hierarchies of the systems perspective applied by designers. Note: there can be multiple types of subsystems and elements

462

6.2

A. Neramballi et al.

Distribution of Cognitive Effort on Design Spaces of the PSS, Products, Services and the Interactions Between Products and Services

The following coding scheme is applied to understand the nature of the interactions between the design spaces of products, services and PSSs. If, in a segmented utterance of the protocol data, the designers focus only on a product-based subsystem or element (e.g., coffee machine, heating coil), then that segment is coded as P. If the designers focus only on a service-based subsystem or element (e.g., service technician, maintenance schedule), then that segment is coded as S, where context is used to disambiguate it from the FBS symbol for structure. If the designers focus on the interactions between product and service-based subsystems and or elements (e.g., repair of coffee machine), then the segment is coded as I. If the designers focus on the entire system in a segment, then it is coded as PSS e.g., resource optimization or business model proposition of the entire product and service offering. This coding scheme is applied to investigate and answer the fourth research question.

7 Results 7.1

Overview of FBS Coding

Initially, the segmented protocol data of the 10 design sessions were coded using the FBS coding scheme by two independent coders. They had an average inter-coder agreement of 72.4% with a standard deviation of 4.0%. This coding was subsequently arbitrated by a third independent coder. The segments that were not relevant to designing were coded as O and were removed. After the removal of segments coded as O, there was an average of 835 segments per session, with a standard deviation of 338 segments over the 10 sessions. Excerpts of segments and the assigned FBS codes are given in Table 1, as an illustration of FBS coding.

Table 1 Excerpt of segmented and FBS-coded protocol data Designer

Segment

Code

X Y Y Y X X

Resource efficiency is the priority Let us optimize the maintenance The product might deteriorate over time To reduce the deterioration, optimized maintenance could be key Then the useful life of the product increases Writes – optimizing maintenance

R S Be F Bs D

How Do Designers Think in Systems? …

7.2

463

Distribution of Cognitive Effort on FBS Design Issues

The average distributions of the designer’s cognitive effort and the respective standard deviations over 10 design sessions of the FBS design issues are shown in Table 2. Design issues related to behavior (both Be and Bs) share around 45.8% of the overall distribution of the designer’s cognitive effort. Bs as a design issue received the highest cognitive effort, with 26.7% of the overall distribution, while the design issue R received the lowest cognitive effort, with 0.7% of the overall distribution. The standard deviations of the distributions of the designer’s cognitive effort on each FBS design issues over the 10 sessions do not exceed 5%. The cumulative occurrences of the different design issues in Session 10 are illustrated in Fig. 3, as an example. This figure provides rich qualitative and quantitative insights into the distribution of designers’ cognitive effort on the different design issues, over time. The graphs indicate that the designers in this session focus more on issues related to Bs, closely followed by F and Be towards the end of the session. In order to check the level of consistency of the designers’ cognitive focus over segments on the various design issues, a linear approximation of the cumulative occurrence of each issue was carried out over the 10 sessions. Coefficients of determination (R2) of all the design issues over the 10 sessions were subsequently extracted from the linear approximations. The mean values and the respective standard deviations of the R2 of all the design issues over the 10 sessions are given in Table 3. The mean values of R2 for all the design issues except requirements (R) are greater than 0.95, as shown in Table 3. This indicates a high degree of linearity in the occurrence of these design issues over the 10 sessions. The standard deviations and corresponding coefficients of variation of the mean values are low. This indicates a high degree of consistency of the R2 values of the design issues over the 10 sessions. On average, Bs has the highest R2. The average value of R2 for R was lower than 0.95 and thus was not considered further.

Table 2 Mean values and respective standard deviations of design issue distribution from 10 design sessions, expressed as percentages along with their coefficients of variation

Design issues

Mean

r

CV

Function (F) 24.49 3.97 0.16 Expected behavior (Be) 19.14 3.36 0.17 Behavior of structure (Bs) 26.66 4.65 0.17 Structure (S) 21.44 3.13 0.16 Design description (D) 7.72 2.99 0.35 Requirements (R) 0.72 0.76 1.30 Note: r – standard deviation, CV – Coefficient of variation

A. Neramballi et al.

350 300 250 200 150 100 50 0 1 40 79 118 157 196 235 274 313 352 391 430 469 508 547 586 625 664 703 742 781 820 859 898 937

Cumulative occurence of design issues

464

Number of segments R (Requirement)

F (Function)

Be (Expected Behavior)

Bs (Behavior of structure)

S (Structure)

D (Design description)

Fig. 3 Cumulative cognitive effort expended on design issues in Session 10

Table 3 Mean values and respective standard deviations of coefficients of determination (R2) from a linear approximation of the transition per design issue

7.3

Mean F 0.9836 Be 0.9832 Bs 0.9913 S 0.9812 D 0.9625 Note: r – Standard deviation,

r 0.0152 0.0167 0.0049 0.0147 0.0311 CV – Coefficient

CV 0.0154 0.0169 0.0049 0.0149 0.0323 of variation

Distribution of Cognitive Effort on Design Processes for 10 Sessions

The transitions between the design issues are represented by eight syntactic design processes defined by the FBS ontology. The mean values of the cognitive effort expended on the 8 design processes over the 10 design sessions are shown in Table 4. Evaluation (22.74%), which represents the bi-directional transitions between Be and Bs, receives the highest share of the cognitive effort. It is closely followed by Reformulation 1 (17.14%), which represents the transitions within S. However, the coefficient of variation for Reformulation 1 is relatively high indicating a high variability in the design sessions for this design process.

How Do Designers Think in Systems? … Table 4 Mean values and respective standard deviations of syntactic design process distributions expressed as percentages, along with their coefficients of variation

Design processes

465 Mean

r

CV

Formulation (F ! Be) 11.96 3.00 0.25 Synthesis (Be ! S) 9.31 1.80 0.19 Analysis (S ! Bs) 15.07 2.25 0.14 Evaluation (Be $ Bs) 22.74 6.36 0.27 Documentation (S ! D) 5.10 1.54 0.30 Reformulation 1 (S ! S) 17.14 7.63 0.44 Reformulation 2 (S ! Be) 8.96 1.91 0.21 Reformulation 3 (S ! F) 9.68 3.17 0.32 r – Standard deviation, CV – Coefficient of variation, ! uni-directional transitions, $ bi-directional transitions

Table 5 Excerpt of segmented and systems hierarchy-coded protocol data Designer

Segment

Code

X Y

Resource efficiency of the entire offering needs to be enhanced To do so, we need to increase the durability of the coffee machine with regular maintenance We also need to optimize the service schedule to reduce the resources consumed by service personnel One way of solving that would be to introduce sensors in the machine to collect statistics regarding usage, power consumption, etc Great idea! Let us now focus on the sensor After that, we can look at the maintenance

1 2

Y X Y X

7.4

2 2 3 3

Overview of Systems Hierarchy Coding

The systems hierarchy coding was carried out using the same procedure as the FBS coding. Initially, two independent coders coded the segments individually. They had an inter-coder agreement of 76.9%, with a standard deviation of 1.4%. It was later arbitrated by a third independent coder. An excerpt of the segmentation and illustration of the respective codes are provided in Table 5.

7.5

Distribution of Cognitive Effort on Levels of Systems Hierarchy

The mean values of the distribution of the designers’ cognitive effort over the 10 design sessions, on the three levels of systems hierarchy is given in Table 6.

466

A. Neramballi et al.

Table 6 Mean values and respective standard deviations of distribution of cognitive effort on the three levels of systems hierarchy from 10 design sessions, expressed as percentages, along with their coefficients of variation r

Mean

CV

Level 1 20.50 4.62 Level 2 24.50 5.97 Level 3 55.20 6.68 Note: r – Standard deviation, CV – Coefficient of variation

Table 7 Mean values of the probabilities of the Markov transitions between three levels of systems hierarchies for 10 sessions

0.2253 0.2436 0.1210

From

To Level 1

Level 2

Level 3

Level 1 Level 2 Level 3

0.76 0.06 0.07

0.06 0.59 0.15

0.18 0.35 0.78

From Table 6, it is evident that more than half of the share of designers’ cognitive effort is focused on the third level. It represents the lowest level that focuses on elements of the system or of the constituent subsystems. The cognitive effort on the other two higher levels is balanced to some extent.

7.6

Temporal Distribution of Cognitive Effort on Problem Decomposition and Recomposition

The temporal distribution of cognitive effort on problem decomposition and recomposition is given by the mean values of Markov transitions of the designers’ cognitive effort on the three levels of systems hierarchy, for 10 sessions in Table 7. The results from Table 7 indicate that probability of intra-level transitions of the designers’ cognitive effort is higher than that of the inter-level transitions (problem decomposition and recomposition). The most probable transitions are within Level 3 with a transition probability of 0.78. It is closely followed by transitions within Level 1. The probabilities of both inter and intra level Markov transitions are depicted in Fig. 4.

7.7

Overview of Design Space Coding

The coding scheme to capture the distribution of cognitive effort on the various design spaces was applied to the segmented protocol data in the same way as the previous two phases. It was initially coded by two independent coders who had an

How Do Designers Think in Systems? …

467

inter-coder agreement of 78.9%, with a standard deviation of 1.98%. Subsequently, the codes were arbitrated by a third independent coder. An excerpt of the segmentation and respective design space codes are illustrated in Table 8.

Fig. 4 Mean values of the probabilities of inter and intra level Markov transitions over 10 sessions. Note: The areas of the bubbles enclosing the levels are relative to the share of designers’ cognitive effort on each level. The thickness of the arrows representing inter- (problem decomposition and recomposition) and intra-level transitions are relative to the respective mean values of Markov transitions

Table 8 Excerpt of segmented and design space coded protocol data Designer

Segment

Code

X Y

Resource efficiency of the entire offering needs to be enhanced To do so, we need to increase the durability of the coffee machine with regular maintenance We also need to optimize the service schedule to reduce the resources consumed by service personnel One way of solving that would be to introduce sensors in the machine to collect statistics regarding usage, power consumption, etc Great idea! Let us now focus on the sensor After that, we can look at the maintenance

PSS I

Y X Y X

S P P S

468

7.8

A. Neramballi et al.

Distribution of Cognitive Effort on the Design Spaces of Product, Service, Interactions and PSS

The mean values of the distribution of cognitive effort on the four design spaces over the 10 design sessions are given in Table 9. The results in Table 9 indicate that the majority of the distribution of cognitive effort is expended on the product design space, with a share of 64%. It is followed by PSS design space, with around 20% of the total share. The rest of the distribution is shared between the design spaces of services and interactions, with only around 15% of the total share. The mean values of the Markov transitions between these design spaces were also determined and are presented in Table 10. The results given in Table 10 indicate that the transitions within the various design spaces dominate inter-space transitions. Transitions within the product design space has the highest probability of occurrence of 0.90 closely followed by transitions within the PSS design space with a probability of 0.77.

Table 9 Mean values and respective standard deviations of the distribution of cognitive effort on design spaces of products (P), services (S), interactions (I) between P and S, and product-service system (PSS) from 10 design sessions, expressed as percentages, along with their coefficients of variation r

Mean

CV

P 64.00 6.41 S 8.70 4.03 I 6.70 1.57 PSS 20.50 4.55 Note: r – Standard deviation, CV – Coefficient of variation

Table 10 Mean values of the probabilities of the Markov transitions between the 3 levels of systems hierarchies for 10 sessions

0.10 0.46 0.23 0.22

From

To P

S

I

PSS

P S I PSS

0.90 0.10 0.26 0.17

0.21 0.65 0.13 0.04

0.26 0.11 0.54 0.02

0.53 0.09 0.07 0.77

How Do Designers Think in Systems? …

469

8 Discussion This section responds to the four research questions. RQ1: What are the distributions of cognitive effort expended on different design issues and processes during conceptual PSS designing? As shown in Table 2, Behavior of structure (Bs), Function (F), Structure (S), and Expected behavior (Be) accounted for 26.7%, 24.5%, 21.4% and 19.1%, respectively, on average. The standard deviation for each issue did not exceed 5% and the coefficients of variation were between 0.16 and 0.17, which indicates that the cognitive effort on each design issue did not differ substantially between the sessions. Design issues for behavior (both Be and Bs) accounted for 45.8%. This means behavior was dominant, but Function and Structure also received substantial effort. As shown in Table 3, the linear approximation of the effort on Behavior of structure was high (R2 value was 0.99). That for F, Be and S was also above 0.98. These show they were spread evenly over segments. As for the design processes, Table 4 showed that the highest effort was spent on Evaluation (Be $ Bs) (22.74%), Reformulation 1 (S ! S) (17.14%) and Analysis (S ! Bs) (15.07%). The effort for Evaluation is associated with the effort on the design issues for Be and Bs. It should be noted that the coefficient of variation for Reformulation 1 was high at 0.44 (Table 4). RQ2: What are the distributions of cognitive effort expended on the different hierarchical levels of systems perspective applied during conceptual PSS designing? As shown in Table 6, Level 3 received the highest effort (55%), followed by Level 2 (25%) and Level 1 (21%). This means that more than half of the designers’ effort was spent on the element level and the least effort on the system level. RQ3: How do designers decompose and recompose design problems during conceptual PSS designing? The majority of the destination levels of the Markov transitions from each level were within the same level i.e., designers stayed at the same level in two consecutive segments, as indicated by the results in Table 7. The probability of transition from Level 2 is more decomposition to Level 3 (0.35) than recomposition to Level 1 (0.06), which is associated with the highest cognitive focus on Level 3. Recomposition from Level 3 is more to Level 2 (0.15) than to Level 1 (0.07). Interestingly, the probability of transition from Level 1 is more to Level 3 (0.18) than to Level 2 (0.06): this means that the decomposition from the whole system was more often directly down to the element level. RQ4: How do designers distribute their cognitive effort on and between the various design spaces of PSSs? For designers in this study, the dominant space was P (64.00%), followed by PSS (20.50%), Table 9. The interactions between product elements and service elements (I) accounted for the fewest segments (6.70%). The transitions between the different spaces were dominated by their inter-transitions, Table 10.

470

A. Neramballi et al.

9 Conclusions Motivated by the need for further insights about the cognitive nature of conceptual PSS designing, the results of the protocol analyses of 10 conceptual PSS design laboratory sessions involving experienced practitioners from industry, are presented. The function-behavior-structure ontology, then a scheme based on a systems hierarchy and a scheme for design space were applied to the coding of the protocols. The results show that the designers’ effort was dominantly on behavior. In terms of the design processes, the largest effort was spent on evaluation. From a systems perspective, more than half of the designers’ effort was spent on the element level, and it was found that the designers are more likely to make transitions within each of the three levels, than between them. In addition, the transition from the sub-system level was more decomposition than recomposition. It was also found that designers are more likely to make transitions within the different design spaces (product, service, interaction, and PSS) than between them. These results enrich the empirically grounded understanding of conceptual PSS designing. These insights are expected to be part of the foundation for the development of dedicated tools, methods and pedagogy that support PSS designing. Limitations of the study include the characteristics of the PSS design task given to the sessions, where an existing product is built upon and PSSs with product-oriented services are designed. Other tasks without such a product might result in different characteristics of designing. Future work includes analysing the correlations between the results from the three coding schemes and investigating the effects of design support to PSS designers. Also, investigation of PSS design practice in industrial settings in comparison to the results of this paper will be useful to derive potential recommendations to industry. Further, comparison of PSS designing with other designing, such as product designing, would add another dimension to enrich our knowledge of designing and assist in determining whether tools for product designing may be useful for PSS designing. Acknowledgements This research is supported in part by the Mistra REES (Resource Efficient and Effective Solutions) program funded by Mistra (The Swedish Foundation for Strategic Environmental Research) (grant number DIA 2014/16). The third author is supported by grants from the US National Science Foundation, Grant Nos. CMMI‐1400466, 1463873 and 1762415.

References 1. Baines TS et al (2007) State-of-the-art in product-service systems. Proc Inst Mech Eng Part B J Eng Manuf 221:1543–1552 2. Boehm M, Thomas O (2013) Looking beyond the rim of one’s teacup: a multidisciplinary literature review of product-service systems in information systems, business management, and engineering & design. J Clean Prod 51:245–250 3. Smith DJ (2013) Power-by-the-hour: the role of technology in reshaping business strategy at Rolls-Royce. Technol Anal Strateg Manag 25:987–1007

How Do Designers Think in Systems? …

471

4. Sakao T, Napolitano N, Tronci M, Sundin E, Lindahl M (2008) How are product-service combined offers provided in Germany and Italy? Analysis with company sizes and countries. J Syst Sci Syst Eng 17:367–381 5. Aurich JC, Fuchs C, Wagenknecht C (2006) Life cycle oriented design of technical product-service systems. J Clean Prod 14:1480–1494 6. © Rolls-Royce plc (2018) CorporateCare for the BR710 7. Tischner U, Verkuijl M, Tukker A (2002) First draft PSS review. In: Econcept Col. p 133 8. Matschewsky J, Kambanou ML, Sakao T (2018) Designing and providing integrated product-service systems–challenges, opportunities and solutions resulting from prescriptive approaches in two industrial companies. Int J Prod Res 56:2150–2168 9. Maussang N, Zwolinski P, Brissaud D (2009) Product-service system design methodology: from the PSS architecture design to the products specifications. J Eng Des 20:349–366 10. Meier H, Roy R, Seliger G (2010) Industrial product-service systems-IPS2. CIRP Ann Manuf Technol 59:607–627 11. Morelli N (2003) Product-service systems, a perspective shift for designers: a case study - The design of a telecentre. Des Stud 24:73–99 12. Kim YS (2020) A representation framework of product-service systems. Des Sci 6:e3. https:// doi.org/10.1017/dsj.2019.30. Cambridge University Press 13. Sakao T, Lindahl M (2015) A method to improve integrated product service offerings based on life cycle costing. CIRP Ann - Manuf Technol 64:33–36 14. Shimomura Y, Nemoto Y, Kimita K (2015) A method for analysing conceptual design process of product-service systems. CIRP Ann - Manuf Technol 64:145–148 15. Isaksson O, Larsson TC, Öhrwall Rönnbäck A (2009) Development of product-service systems: challenges and opportunities for the manufacturing firm. J Eng Des 20:329–348 16. Clayton RJ, Backhouse CJ, Dani S (2012) Evaluating existing approaches to product-service system design: a comparison with industrial practice. J Manuf Technol Manag 23:272–298 17. Gero JS, Mc Neill T (1998) An approach to the analysis of design protocols. Des Stud 19:21– 61 18. Gero J (1990) Design prototypes: a knowledge representation scheme for design. AI Mag 11 (4):26–36. https://doi.org/10.1609/aimag.v11i4.854 19. Song T, Becker K, Gero J, Deberard S, Lawanto O, Reeve E (2016) Problem decomposition and recomposition in engineering design: a comparison of design behavior between professional engineers, engineering seniors, and engineering freshmen. J Technol Educ 27:37–56 20. Kannengiesser U, Gero JS (2015) Is designing independent of domain? Comparing models of engineering, software and service design. Res Eng Des 26:253–275 21. Mc Neill T, Gero JS, Warren J (1998) Understanding conceptual electronic design using protocol analysis. Res Eng Des 10:129–140. https://doi.org/10.1007/BF01607155 22. Hay L, Duffy AHB, McTeague C, Pidgeon LM, Vuletic T, Grealy M (2017) A systematic review of protocol studies on conceptual design cognition: Design as search and exploration. Des Sci 3:e10. https://doi.org/10.1017/dsj.2017.11. Cambridge University Press 23. Sakao T. Paulsson S. Mizuyama H. (2011) Inside a PSS design process: Insights through protocol analysis. In: 18th Int Conf Eng Des pp 365–376 24. Shimomura Y, Nemoto Y, Kimita K (2015) A method for analysing conceptual design process of product-service systems. CIRP Ann - Manuf Technol 64(1):145–148. https://doi. org/10.1016/j.cirp.2015.04.035 25. Bertoni A (2013) Analyzing product-service systems conceptual design: the effect of color-coded 3D representation. Des Stud. 34(6):763–793. https://doi.org/10.1016/j.destud. 2013.02.003 26. Won Lee S, Lee J, Jo N, Kim YS (2013) Design activity and team interaction characteristics: a case study of protocol analysis on team-based product-service systems design processes. In: Proc Int Conf Eng Des ICED, pp 349–358

472

A. Neramballi et al.

27. Neramballi A, Sakao T, Gero JS (2019) Effects of a design support on practitioners designing a Product/Service System – a case study. In: 2nd SIG Conf. Hum. Behav. Des. Institut für Technische Produktentwicklung, Universität der Bundeswehr München., Tutzing, Munich, Germany, pp 11–22 28. Neramballi A, Sakao T, Gero JS (2018) What do experienced practitioners discuss when designing product/service systems? In: Int Conf Des Comput Cogn Springer, pp 361–380. https://doi.org/10.1007/978-3-030-05363-5 29. Oliva R, Kallenberg R (2003) Managing the transition from products to services. Int J Serv Ind Manag 14:160–172 30. Lindahl M, Sundin E, Sakao T (2014) Environmental and economic benefits of integrated product service offerings quantified with real business cases. J Clean Prod 64:288–296 31. Beuren FH, Gomes Ferreira MG, Cauchick Miguel PA (2013) Product-service systems: a literature review on integrated products and services. J Clean Prod 47:222–231 32. Roy R (2000) Sustainable product-service systems. Futures 32:289–299 33. Tukker A (2015) Product services for a resource-efficient and circular economy - a review. J Clean Prod 97:76–91 34. Vasantha GVA, Roy R, Lelah A, Brissaud D (2012) A review of product-service systems design methodologies. J Eng Des 23:635–659 35. Ericsson KA, Simon HA (1993) Protocol analysis. MIT Press 36. van Someren MW, Barnard YF, Sandberg JA (1994) The think aloud method: a practical approach to modelling cognitive 11:208. London: Academic Press, 37. Gero JS, Tang HH (2001) The differences between retrospective and concurrent protocols in revealing the process-oriented aspects of the design process. Des Stud 22:283–295 38. Kan JWT, Gero JS (2017) Quantitative methods for studying design protocols. 1, IX:180. https://doi.org/10.1007/978-94-024-0984-0. Springer 39. Gero JS (2010) Generalizing design cognition research. dtrs8 interpret des thinking. DAB Doc 2010 Sydney 8:187–198 40. Gero JS, Kannengiesser U (2004) The situated function-behaviour-structure framework. Des Stud 25:373–391

Towards Modelling Interpretation of Structure as a Situated Activity: A Case Study of Japanese Rock Garden Designs Yuval Kahlon and Haruyuki Fujii

Abstract This research aims to contribute to our understanding of conceptual design activity, for the enhancement of current CAD support systems targeting conceptual design. In this paper, we systematically approach conceptual design, by relating empirical data from design activity with theoretical models for situatedness in design. We focus on interpretation of physical entities, in the context of Japanese rock gardens, which serve as a case study. We then identify key elements in formally discussing interpretation in-action, and consider their contribution to subsequent design activity. These can enrich computational design systems dealing with interpretation.

1 Introduction In recent years there is a growing interest in constructing autonomous agents for conducting design tasks [1, 2]. Considering the potential of CAD tools to support conceptual design, research is conducted to develop computational systems for aiding in ideation and exploration [3], encouraging conceptual shifts [4] and more. Utilizing empirical data for formalizing practices of conceptual design can significantly contribute to further development of such systems. In this research, we approach conceptual design, by integrating theoretical models of situated cognition [5, 6] with empirical data from design activity. Specifically, we focus on interpretation of structures of physical entities in spatial configuration design, in the context of Japanese rock gardens (JRG), as a case study (Fig. 1). Simple interpretations are formally represented and their contribution to design activity is considered, from the perspective of situatedness in design.

Y. Kahlon (&)
H. Fujii Tokyo Institute of Technology, Tokyo, Japan e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. S. Gero (ed.), Design Computing and Cognition'20, https://doi.org/10.1007/978-3-030-90625-2_28

473

474

Y. Kahlon and H. Fujii

Fig. 1 Spatial configuration & interpretation: (a) Mt. Horai in JRG composition at Ryogen-in, Kyoto & in a painting by T. Tessai; (b) miniature composition by a participant & a photo of Yellow Mountain (Huangshan), China

1.1

Spatial Configuration and Interpretation

The initial design phase of artifacts is often guided by a symbolic description, generated by the designer in attempt to mentally frame the design activity and give it coherence as a whole [7]. Such descriptions, which often take the form of metaphor or analogy [8], are the product of a continuous interaction between the designer and the artifact, as an act of interpretation in-action. The practice of producing and using these descriptions is fundamental to conceptual design activity [9]. In this paper, we focus on a form of such descriptions which we term “representational interpretation” (RI). A RI is an interpretation in which the physical formation of rocks is regarded as a symbol expression of a state of affairs, composed of the entities that the rocks stand for. Since RIs depend both on the designer’s subjective viewpoint and on the interpreted object itself, any formalization of this practice is expected to account for the relation between the two; i.e. between the physical structure and its symbolic description, given by the designer.

1.2

Situatedness a Basis for Approaching Structure Interpretation

Situated models for design [5], which are based on the situated approach to cognition [6], consider design activity as interaction between the external world and the internal world of the designer. Accordingly, they can potentially explain the emergence of conceptual interpretations from designers’ interaction with their environment. We utilize this approach, and specifically the situated function-behaviour-structure framework (situated FBS) [5], to analyze the manner in which designers employ RIs during design activity.

Towards Modelling Interpretation of Structure …

475

2 Aim, Objectives and Scope This research strives towards a formal model for interpretation in conceptual design, from the perspective of situatedness, to enable the construction of cognitivelyinspired computational design systems. In this study, we focus on interpretation as an activity grounded in empirical data concerning analogical reasoning, by studying the assignment of RIs in designing a miniature model of a JRG. Our main objectives are 1) document conceptual design activity, 2) identify how designers assign RIs consisting of symbolic descriptions to ambiguous formations of physical entities, 3) construct a basic vocabulary for describing RIs from the perspective of situated cognition, and 4) utilize it to relate RIs with subsequent courses of action in design.

3 Significance This research contributes to our understanding of visual interpretation in conceptual design, by relating RIs with their analogical basis, in the context of the design situation in which they emerged. Consequently, it may aid in 1) constructing design agents capable of actively supporting conceptual design activity, for implementation in future CAD systems, by interpreting design representations in-action; and 2) adapting current situated design models for describing visual interpretation activity.

4 Method In accordance with our aim and objectives, we 1) document an activity of designing a model of a miniature JRG; 2) extract analogies used to facilitate dependencies between design elements, and their underlying conceptual metaphors, by analyzing the design protocol; 3) identify key-elements in relating structure and interpretation across the three design worlds of the situated FBS framework [5], with the aid of other formal models for design, and 4) discuss the contribution of these key elements to subsequent design activity.

4.1

Task and Setup

We devised a task of spatial configuration using a table-sized model of a JRG (Fig. 2). Subjects were asked to design a “garden” to their liking by selecting and positioning rocks from a small collection, with the constraint of refraining from stacking rocks (as in traditional JRG). The main design challenge lay in 1)

476

Y. Kahlon and H. Fujii

Fig. 2 Experiment environment: setup and documentation

construing the task by defining the design theme (i.e. the meaning of “a garden”; as in Goldschmidt [9]); 2) coordinating the selection and placement of rocks with one’s interpretations. The task is situated in the sense that the designer develops all of the above on-the-fly, while interacting with the model. The task was deemed suitable for the purpose of this research for several reasons, and mainly for 1) JRGs’ frugality, which renders them as a manageable environment for effectively exploring interpretation activity, considering the limited amount of design elements etc., and 2) the abstract and representational character of JRGs, which naturally encourages interpretation. All participants were adult subjects trained in architectural design (a bachelor’s degree in Architecture, at minimum). We employ the think-aloud method, instructing participants to design a miniature garden while expressing their considerations and deliberations, by voicing out their internal dialogue. Sessions are capped at 30 min, to constrain the conceptual complexity involved in the design. Speech and actions are recorded using video cameras from two angles (top, front). Protocol analysis is then combined with the situated FBS framework, as well as other theoretical design models, to establish a structured approach for extracting, analyzing and formalizing interpretations.

5 Identifying Interpretations and Their Analogical Basis 5.1

Interpretation and Analogy

We approach interpretation as an activity of “seeing as”, originally proposed by Wittgenstein [10] to refer to the human ability to perceive one thing as another, and later discussed by Goldschmidt [11] as well as Schon and Wiggins [12] in the specific context of design activity. This notion is strongly linked with similarity between objects, and therefore with analogy. As previously mentioned, a powerful strategy by which designers structure their understanding of design tasks is via the usage of conceptual descriptions, such as RI. One basis for effectively assigning such descriptions is the existence of

Towards Modelling Interpretation of Structure …

477

similarities between attributes and relations among the design elements and those among other entities, belonging to a domain which the designer is familiar with (source). This enables to intuitively project these onto the existing structure (target), as a form of analogy-based constructive interpretation. Furthermore, as similarity-based mappings between objects or entities, analogies serve as the basis for analogical reasoning. Analogical reasoning enables us to extend our knowledge from a familiar domain (source) into a less-familiar one (target), via retrieving such mappings and utilizing them for making further inferences [13]. Accordingly, they can serve as drivers for reasoning in interpretation.

5.2

Three Levels of Analogical Mapping

Analogical mappings are generally classified into three types, based on the level at which similarity is identified: attribute mapping (AM), relational mapping (RM) and structure mapping (SM, also commonly referred to as system mapping) [13, 14]. We rely on these for extracting and analyzing analogical mappings from design sessions. Below is a brief explanation including a simple example for each. In AM, two things are deemed similar owing to a shared attribute, for example: an orange and a basketball are both round and thus similar in this respect. In RM, similarity is identified between relations, e.g. on a higher level, for example: a bear cub and a kitten are both younger than their parents, and therefore share this relation. In SM, similarity occurs on an even higher level of abstraction. Gentner [14] provides the example of similarity between a hydrogen atom and the solar system - both contain an element which revolves around another.

5.3

Extracting Interpretations from Design Sessions

On the basis of the above classification, RIs were extracted from design sessions via protocol analysis, combined with the visual documentation. We now present several interpretations at varying levels of complexity, each accompanied by the analogy employed at the time of formation (Fig. 3). These are then summarized in Table 1, which also provides a simplified formalization for possible analogical mappings corresponding with the RI. Interpretation 3(a) presents a rock referred to by a subject as “an old storyteller”. Old age was attributed to the rock owing to its rugged surface texture, resembling the rough texture of an elderly person’s wrinkled skin, i.e. a simple AM was employed. In 3(b) we see a slightly more complex RM, under which the larger rock is interpreted as “a background” for the smaller one, justified by their spatial relations of front and back. In the accompanying image, a large bush serves as a “background”, for a smaller one. While the analogical mapping here is established based

478

Y. Kahlon and H. Fujii

Fig. 3 Examples for interpretations during design activity and possible associated sources: (a) “old person” (b) “a background” (c) “journey of discovery”; see Table 1 for further details. Sources: (c) Honpukuji Temple, Tadao Ando, Awaji island, Japan (japan-magazine.jnto.go.jp)

Table 1 Complementary to Fig. 3; interpretations & their supporting analogies Item

Interpret.

Focused Aspect

Analogical Mapping

Type

(a)

“an old storyteller” “a background”

apparent age visual composition user experience

rough-texture(old-person) !rough-texture(rock-a) behind(background,foreground) !behind(rock-a,rock-b) journey(user,rock-sequence) !journey (user,space-sequence)

AM

(b) (c)

“a journey of discovery”

RM SM

on the existence of simple spatial relations, it may be further elaborated by finding additional similarities between the source and target. For example, the flatness of the larger element may suggest the flatness of a canvas, as a background for a painting etc. Finally, 3(c) relies on a SM in which the similarity between the target and its source occurs on the level of human experience. Visitors to Honpukuji Temple (explicitly referenced by the subject) begin by climbing a staircase bounded by a narrow hall, which then leads to a vast open space. This contrast elicits a feeling of surprise and discovery, which the subject tried to embed in the design by placing a row of similarly-sized rocks leading to a significantly larger one. Interpretations, however, are not merely passive labels attached by designers to the designed artifact on analogical (or any other) grounds. They are, in fact, powerful drivers for developing a conceptual understanding of the design activity, by helping to structure the relations between design elements [15]. This is highly noticeable in cases where the analogy exposes a hierarchical order, which guides the designer’s understanding of the current situation, as discussed in the following section.

Towards Modelling Interpretation of Structure …

479

6 Relating Structure and RI via Element Dependencies Interpretation by analogy often invites attributes and relations from external reality, into the designer’s internal world. By projecting these onto the design elements (whether explicitly or implicitly), the designer begins to conceptually understand the design as a unified construct, which corresponds in some manner to constructs in the external world. The internal construct may then be further interpreted and reasoned about, based on the dependencies between its elements. We first introduce the notion of design worlds in the situated FBS framework, as a basis for analyzing interpretation activity. This is followed by concrete examples of interpretations from design sessions which are structured hierarchically, as well as the contribution of these to the designer’s understanding of the design activity.

6.1

Situated FBS Design Worlds

Gero and Kannengiesser’s situated FBS framework [5] describes design as an activity which occurs in three nested worlds: external, interpreted and expected (Fig. 4). Roughly speaking, the first accounts for the representation of the state of the structure as it exists in external reality, the second of the designer’s conception of the former, and the third of the intended states to be brought about via design activity. The interpreted world can be seen as a kind of mediator between the plans and actions which the designers aims to execute in the future, and the concrete current state of the design. Therefore, the manner in which the interpreted world is mentally structured will greatly affect the course of the design. Analogies seem extremely useful in informing us regarding certain relations between design elements, existing in the interpreted world. A fundamental and therefore important relation discussed hereafter is that of dependency between design elements, induced by hierarchical RIs.

6.2

Hierarchical Relations in Analogies Used by Participants

We begin by extracting and analyzing simple analogies which include a notion of hierarchy. Several examples, used by participants during design sessions, are provided below in Fig. 5. Fig. 4 Design worlds in situated FBS; adapted from Gero and Kannengiesser [5]

480

Y. Kahlon and H. Fujii

Fig. 5 Interpretation via analogy containing a hierarchical component

In what sense are these of a hierarchical nature? While in each RI different relations occur between the design elements (for example, 5(a) entails an age difference, while 5(e) a part-to-whole relation), in all underlie conceptual superiority-inferiority relations, reflected in the dependencies of certain elements on others [16]. These can be identified from various perspectives: for example, in 5(a) children may depend on the old man for guidance (social), while in 5(e) the growth of petals depends on the existence of the body of the flower (part to whole). The identification and selection of some dependencies rather than others during RI activity is related with the notion of aspectual shape [17], and is beyond the scope of this paper. It is important to emphasize, however, that regardless of a specific aspect a designer may focus on, utilizing an analogy of a hierarchical nature results in the projection of roles upon the design elements. These roles help to further structure design activity, as explained in the following sub-section.

6.3

Structuring the Interpreted World via Roles

Since RIs (such as the above) determine the meaning of design elements for the designer, they also determine their potential contribution to the design activity, from the internal perspective of the designer. Consequently, they are essential building blocks of our interpreted world. Therefore, relating RIs with their grounding in the external world can aid in understanding and describing the interpreted world. This can be done on the basis of hierarchical dependencies, revealed by metaphoric roles. Before proceeding, however, it is important to further elaborate the notion of RI: 1) the complete symbolic description assigned to the composition, which generally frames the theme for the design, will be referred to hereafter as “RI”; and 2) the function description projected onto a physical design element, by relating it with a referent external to the design, will be referred to as “role” (see Table 2). Now, consider the example in Fig. 6 below, in which the designer had interpreted a two-rock composition as “rock with plants growing under it”. Under this RI, one rock (e1) is simply seen as a rock (r1), while the other (e2) as plantation (r2) growing under the former. The rocks in the external world thus correspond with roles in the interpreted world.

Towards Modelling Interpretation of Structure …

481

Fig. 6 RM; representation of the correspondence between entities in the external and interpreted worlds of the situated FBS framework

This simple correspondence, however, goes beyond mere denotation. Brereton [18] has pointed out the importance of “negotiation” between material and abstract representations in conceptual design, which provides designers with opportunities for further structuring current representations, i.e. their interpreted world. Each description assigns an element with a role, which is to be conceptually fulfilled by its bearer. Fulfillment of a role may be seen as maintaining certain relations with other elements, as to correspond with their external world referents. This can be understood on two different levels: first, the interpretation imposes certain spatial relations between the elements. For example, continuing with Fig. 6, once the small rock has been assigned with the role of “plantation growing under…”, it is expected to sit beneath the larger one, and perhaps close to it. More fundamentally, however, we can uncover a powerful conceptual metaphor, which underlies the hierarchical relations resulting from the assigned roles, and thus guides design activity. Hierarchy implies dependency, and dependency implies control. Accordingly, moving the larger rock for some reason will entail considerations regarding its subordinate “plant” rock. The dependency from the internal world is thus translated into dependency in the external world, and the two worlds are linked by metaphoric roles.

7 Modelling RI in-Action from a Situated Perspective Smith and Gero note that, when perceiving, agents interact with focused objects to construct interpretations regarding their potential, rather than their current identity or denotation [19]. This is, of course, highly related with the act of “seeing as”, which served as a basis for the discussion so far. Objects’ potentials, reflected by their interpretation, may be further described as their possible contribution to the design process and/or its product. This potential is determined by a combination of objective attributes and relations as well as subjective impressions of these, i.e. via interaction between the external and interpreted worlds. Therefore, RI can only be fully accounted for by representing both the former, the latter, and the relation between them. We propose a structured approach for representing and analyzing RI in design, based on empirical data from design activity, as well as theoretical models for situated design and analogy-based design [20]. We begin by defining key terms (Table 2) and providing an example for the graphical notation used in our analysis (Fig. 7 below).

482

Y. Kahlon and H. Fujii

Fig. 7 Notation used in session analysis. Left: a transition between two states in the three worlds of situated FBS; Right: session analysis legend

Table 2 Important terms for modelling RI activity and their definition Term

Definition

Structure Element

The complete set of physical entities placed in the design space A single indivisible physical part of the design, or its representation (similar to “primitive element” in [20]) A single property of an element (similar to attribute in [20]) A behavior which is to intended to be brought about by design activity (as in the situated FBS framework) The complete symbolic description assigned to the structure which denotes a set of referents that the structure stands for Interpretive function description of an element as defined by a RI Descriptive expression assigned to an element or its role in the course of design by the designer, in an explicit manner The domain of discourse which is implied by the protocol A segment of the design session after which change is observed

Attribute Expected behavior RI Role Feature Domain State

7.1

Situated Interpretation and Action

We now present and discuss several important aspects of interpretation activity, from the perspective of situated design. we focus here on three operations named: role assignment, constructive elaboration, and detachment. Role assignment consists of naming design elements as to denote referents external to the design. In Fig. 8 the subject began by considering size (“I will pick a smaller one”) with the goal of creating “a small place for people to walk around”, followed by the suggestion of a general domain of interest (“I really like modern cities”). The structure was thus assigned with the role of a city (r1), which prompted a size-based selection (“pick something bigger”) of a thin element (e2) to serve as a “skyscraper” (r2), also based on a proportion attribute (a1). Consequently, an element (e6) was added as a “smaller building” (r3) (Table 3). In Fig. 9 we provide an example of role assignment for multiple elements, which is induced by gestalt perception. The subject saw a “connecting line” (e4, existing

Towards Modelling Interpretation of Structure …

483

Fig. 8 Role assignment; e2 and e6 are interpreted as a skyscraper and a building

Table 3 Complementary to Fig. 8; selected states and their representation State

Utterance

Representation

t6

“I will pick a smaller one…I really like very modern cities like Tokyo and New York” “to symbolize the image of city… I would pick some bigger thing to help create…the framework” “it looks like…how to say…a building, a tall skyscraper” “and also…some smaller ones beside the skyscraper”

f1 = feature(e1,small) f4 = feature(X,metropolitan) r1 = role(S,city) f5 = feature(e2,bigger)

t10

t11 t15

r2 = role(e2,skyscraper) a1 = attribute(e2,proportion) r3 = role(e6,building)

only in the interpreted world), which then prompted the interpretation of each rock as a “dot” (e1,e2,e3), resulting in the selection and addition of an element to continue the perceived line (e5). After assigning roles to design elements, it is also possible (and in fact common) to extend the interpretation, by constructively elaborating roles for the other elements as well, as a basis for their selection and positioning. This can be seen in the short segment, given above in Fig. 10. The designer deemed the first rock (e1) as “old man” (r1), and expanded its description by adding that “it has some stories to tell” (f1) etc. These interpretative descriptions, grounded in the designer’s RI by the AM [rough(rock-surface)!rough(elderly-skin)], have directed the next course of

484

Y. Kahlon and H. Fujii

Fig. 9 Role assignment; simultaneous role projection from gestalt perception

Table 4 Complementary to Fig. 10; selected states and their representation State

Utterance

Representation

t0 t1 t1 t3 t5 t6–7

“like an old man” “has some stories to tell” “maybe he has suffered a lot” “we can see… small bumps” “and maybe some…” “…next generation around him… the children are listening to his stories…they are sitting around”

t8

“…they should be face to face”

r1 = role(e1,old-man) f1 = feature(r1,has-stories) f2 = feature(r1,suffered) b1 = behavior(a1,bumpy) X=? X = e2 r3 = role({e2,e3},children) f3 = feature(r3, next-generation) f4 = feature(r3,sitting) f5 = feature(r3,listening) b2 = behavior(r3, face-to-face)

action - the designer chose to add rocks (e2,e3) as “children” (r3) to “listen to his stories”, and finally determined they should be “face to face” (b2), thus removing an incompatible element, which was not “facing” the old storyteller (e4) (Table 4). Detachment deals with the intuitive transference of entities (attributes, features, roles etc.) from one element to others, during the design process. As an example, we present Fig. 11 in which the initially selected rock (e1) was removed, only to be replaced by a different rock positioned in the same location (e2). The original rock helped the designer to establish the position for the first element in the space, but could not stand due to its weight. Its position attribute was transferred to the next element, which served as the first element in the actual final composition which represented a flower.

Towards Modelling Interpretation of Structure …

485

Fig. 10 RI elaboration based on initial roles assigned to an element

Fig. 11 “Flower and fallen petal”; simple example of attribute detachment

8 Discussion We have attempted to systematically approach RI activity by identifying entities which lie at its core. These entities were then used to model design sessions, in a manner which enables to trace some of the contributing factors for the development

486

Y. Kahlon and H. Fujii

of different RIs. We now focus on the relevance of the three phenomena, presented in the previous section, to understanding interpretation in-action. First, when discussing the issue of concept formation, Gero asks “where do concepts come from?” [21]. Analogically, we can raise the following important question: where do RIs come from? Viewing interpretation as an act of attributing meaning to design elements, and considering the importance of metaphoric roles for RI, it may be useful to first attempt to explain how metaphoric roles are attributed. As shown in Fig. 8, while roles often have an analogical basis, their emergence in the design process cannot be accounted for merely on the basis of analogical relations, but only as an interaction between different design worlds. In this specific case, the role emerged at least as a product of 1) the focused domains of discourse, i.e. scale, metropolitans etc.; 2) the decision to start with a large element, “to set the framework”; and 3) the form of the rock. This points to the potential of expanding the situated FBS framework to include discursive factors contributing to the design. For example, the current domain of discourse, which helps in shaping the course of action, and thus may affect or trigger different processes and transformations. Second, Clancey [6] maintains that one core phenomenon in situatedness is the reassignment of meanings, as a result of perceiving changes in the environment. Constructive elaboration thus enables the extension of interpretation over a period of time, during which reconsideration and revision can take place. This is consistent with the emphasis on the importance of time extension in perception processes, as propagated by Neisser [22]. For example, in Fig. 10 the physical properties of the rock first served to construct a spontaneous RI (“old man”) which was further sophisticated by elaborating suitable roles for the other elements (“children listening to his stories”). This was not done in a strict serial manner, but rather as an act of coordination, characterized by a simultaneous adjustment of the RI and the spatial relations. Hence, both the structure and the RI were conjointly conceived and reconceived. Third, the phenomenon of detachment may reflect designers’ tendency to try and maintain positive aspects of a structure, while essential changes are made to it. This is rendered possible by projecting properties of certain elements onto others, as an act of establishing an “element lineage” linking generations by inheritance. The subject’s attachment to the initial placement in Fig. 11 resulted in its “survival”, despite other inevitable changes (i.e. replacing an unstable element). All of the above point to the strong potential of analyzing RI activity from a situated perspective, which enables to understand complex events in design activity as a result of several, lower-level factors. We believe that further inquiry into recurrent patterns in interpretation, based on the proposed modelling approach, can lead to identification of important processes which are critical for RI activity. This may aid in formalizing such processes to be used in intelligent computational design systems.

Towards Modelling Interpretation of Structure …

8.1

487

Implications

The main implications are now briefly discussed. In recent years, research is conducted to develop computational design support systems which can effectively propose automatic interpretations to visual content. Such systems can benefit from the identification of key-entities and phenomena which characterize interpretation as a situated activity. The former can be used for creating explicit representations in a knowledge base, to enable further reasoning; the latter for considering new interpretation strategies, based on empirical data from designers. The system developed by Karimi et al. [4] proposes alternative visual interpretations for sketches made by a designer. In a related manner, the system by Jowers et al. [3] enables recognition and manipulation of sub-shapes, as a basis for re-interpretation. These systems, for instance, may be further developed to consider potential roles of design elements, as well as deeper sets of relations between them, as a basis for interpretation in conceptual design. Such efforts may be assisted by existing research on the role of metaphor and analogy in design [8, 15], and specifically on formally relating metaphoric descriptions with spatial configurations [23]. Furthermore, in other design fields such as engineering design, analogy is often used for concept generation, by locating “functionally relevant” [24] sources, to inform or inspire the designer regarding the task at hand. Given that the designer is reasoning with respect an existing structure, the ability to locate relevant sources by analogy is restricted by the manner in which the structure is encoded [25]. Including high level descriptions (when encoding), such as metaphoric roles of elements, can enhance our ability to relate sources with targets; by enriching the description of the structure with possible interpretations, as to identify similarities on a conceptual level.

8.2

Limitations

We mention several important aspects of the work which demand attention in the next steps of development, the first being the complexity of the structures under consideration. In contrast with the previously mentioned framework by Qian and Gero [20], which discusses complex elements (referred to as “substructures”), we currently account only for interpretation of single elements (“primitives”) and thus only deal with relatively simple structures. However, while JRG design activity does not enable actual physical attachment between elements to create compound objects, it is possible to consider compositions of several rocks as substructures, once they are visually grouped in a certain manner [26]. Complexity may be further considered at the element-level as well, via enabling representation at different degrees of abstraction, encouraging different RI (by increasing the variety in our rock collection, or by repeating the task with different design elements).

488

Y. Kahlon and H. Fujii

Additionally, it is necessary for the proposed modelling approach to explicitly consider the aspect of function in a deeper manner. In the special case of JRGs (designed mainly for viewing), function coincides with visual interpretation, to a large extent. However, to generalize our approach for RI to other design activities, we need to consider varying functionalities of the design elements. This may be done with minimal alteration of our current method, in the context of architectural design, by revising our design task theme to Chinese traditional gardens - these allow interaction with users (going in, climbing the rocks etc.), and therefore consider a wider range of functionalities. Finally, it is important to acknowledge a knowledge gap which should be filled, in order to implement our insights in artificial agents. Although computational systems which can assign simple interpretations exist, further work is required in order to develop the ability to select relevant ones in-context. We believe that the complex mechanisms at play here can also be studied from a similar perspective, which harnesses insights from situated cognition in design.

9 Conclusion A formal approach for studying RIs in design is proposed, enabling to understand interpretation as an interaction between the designer and the physical entities which compose the structure, as a situated activity. Accordingly, it can serve as a stepping stone towards enhancing current agents for supporting conceptual tasks in design, as well as contribute to current models for situated design dealing with interpretation.

References 1. Groenewolt A, Schwinn T, Nguyen L, Menges A (2018) An interactive agent-based framework for materialization-informed architectural design. Swarm Intell 12(2):155–186 2. Cai C, Tang P, Li B (2019) Inheriting Spatial Features of Chinese Garden Based on Prototype and Multi-Agent System. In: CAADRIA 2019 24th Int Conf Assoc Computational Arch Des Res Asia, pp 291–300 3. Jowers I, Prats M, Lim S, McKay A, Garner S, Chase S (2008) Supporting reinterpretation in computer-aided conceptual design. In: EUROGRAPHICS workshop on sketch-based interfaces and modeling, pp 151–158 4. Karimi P, Grace K, David N, Lou Maher M (2018) Creative Sketching Apprentice: Supporting Conceptual Shifts in Sketch Ideation. In: Design computing and cognition 2018, pp 721–738 5. Gero JS, Kannengiesser U (2004) The situated function-behaviour-structure framework. Des Stud 25(4):373–391 6. Clancey WJ (1997) Situated cognition: on human knowledge and computer representations. Cambridge University Press, New York

Towards Modelling Interpretation of Structure …

489

7. Hey JHG, Agogino AM (2007) Metaphors in conceptual design. In ASME international des engineering technical conference and computers and information in engineering conference, pp 1–10 8. Casakin HP (2006) Assessing the use of metaphors in the design process. Environ Plann Plann Des 33(2):253–268 9. Goldschmidt G (1988) Interpretation: its role in architectural designing. Des Stud 9(4):235– 245 10. Wittgenstein L (1958) Philosophical investigations. Basil Blackwell, Oxford 11. Goldschmidt G (1991) The dialectics of sketching. Creat Res J 4(2):123–143 12. Schön DA, Wiggins G (1992) Kinds of seeing and their functions in designing. Des Stud 13 (2):135–156 13. Holyoak K, Thagard P (1996) Mental leaps. MIT press, Massachusetts 14. Gentner D (1983) Structure-mapping: a theoretical framework for analogy. Cogn Sci 7:155– 170 15. Hey J, Linsey J, Agogino AM, Wood KL (2008) Analogies and metaphors in creative design. Int J Eng Educ 24(2):283–294 16. Kahlon Y, Fujii H (2019) Framework for metaphor-based spatial configuration design: a case study of Japanese rock gardens. Artif Intell Eng Des Anal Manuf 34:1–10 17. Searle JR (2004) Mind: a brief introduction. Oxford University Press, New York 18. Brereton M (2004) Distributed cognition in engineering design: negotiating between abstract and material representations. In: Goldschmidt G, Porter WL (eds) Design representation. Springer Verlag, London, pp 83–103 19. Smith G, Gero JS (2001) Situated design interpretation using a configuration of actor capabilities. In: Gero JS, Chase S, Rosenman MA (eds) CAADRIA 2001. University of Sydney, Key Centre of Design Computing and Cognition, pp 15–24 20. Qian L, Gero JS (1996) Function–behavior–structure paths and their role in analogy-based design. Artif Intell Eng Des Anal Manuf 10(4):289–312 21. Gero JS (1998) Concept formation in design: towards a loosely-wired brain model. Knowl Based Sys 11(7–8):429–435 22. Neisser U (1976) Cognition and reality: principles and implications of cognitive psychology. W.H.Freeman & Co Ltd., San Francisco 23. Kahlon Y, Fujii H (2020) A framework for concept formation in cad systems: case study of Japanese rock garden design. Comput Aided Des Appl 17(2):419–428 24. Fu K, Murphy J, Yang M, Otto K, Jensen D, Wood K (2015) Design-by-analogy: experimental evaluation of a functional analogy search methodology for concept generation improvement. Res Eng Des 26(1):77–95 25. Linsey JS, Laux JP, Clauss E, Wood KL, Markman AB (2007) Increasing Innovation: A Trilogy of Experiments Towards a Design-by-Analogy Method. In: ASME Internat Des Eng Tech Conf and Comp and Info in Eng Conf, pp 145–159 26. Van Tonder GJ, Lyons MJ (2005) Visual perception in Japanese rock garden design. Axiomathes 15(3):353–371

Interactive Visualization for Design Dialog Arefin Mohiuddin and Robert Woodbury

Abstract In this paper, we argue that computer aided design media should support interactivity so that designers can rapidly create, evaluate, and modify plentiful design alternatives. This will enable an engaging dialog with the design situation. To enable this in parametric modeling tools, we use a research through design approach and propose an interactive visualization, and report its evaluation. From the results, a rich design space of possible interactions and visualizations emerge.

1 Introduction Schön [1] argues that the design process is a “reflective conversation” between designer and design situation, and depends on the external medium. Goldschmidt [2] studies sketching within design, starting with the observation that “…architects engage in intensive, fast, frequent sketching when their first tackle a design task.” She argues that designers, specifically architects, use sketching to enable an oscillating argument between two modes: seeing-as, in which designers see sketches as figural representations and seeing that, in which designers probe sketches for meaning. For such a dialectics of sketching to occur, sketches must be produced abundantly and be available to dialog [3]. This “abundance of sketches” implies that a design medium should, in the very least, support many such figural representations. Among the attributes of what constitute sketches [4], inexpensiveness, abundance, and disposability should reduce barriers to the exploration of several alternatives, especially during the early design phases. Ambiguity prevents premature commitment and aids in emergence of the final design. Interactions with design media should enable “moves” and “arguments” to allows the designer to rapidly A. Mohiuddin (&)
R. Woodbury Simon Fraser University, British Columbia, Canada e-mail: [email protected] R. Woodbury e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. S. Gero (ed.), Design Computing and Cognition'20, https://doi.org/10.1007/978-3-030-90625-2_29

491

492

A. Mohiuddin and R. Woodbury

create, judge, and modify the representation, till it becomes a visual search for a desired outcome. Such interactions with the medium are congruous with near-term experimentation [5], the latter resembling reflective conversation [1]. We term such interactions with design media as “design dialog” [3]. Design media should offer sketch-like interactions for rapid analysis-synthesisevaluation cycles [6] on multiple design alternatives. This is our goal. We employ a research through design [7] approach to iteratively design and evaluate prototypes, informed by the Human–Computer Interaction (HCI) and Information Visualization (InfoViz) literatures. The expected contribution is a “articulation of a preferred state” [7], manifesting as prototypes and documentations of the ‘preferred state’. Through the design of interactive visualizations, we attempt to connect the cognitive science behind the design process with architects using pCAD tools. We aim to map the idea of design dialog to rapid analysis-synthesisevaluation cycles, and makes this process computable through novel interaction design. Also, according to this research method, by the design of an exemplary prototype knowledge is transferred from HCI and InfoViz research towards designing better pCAD tools. This paper increments our body of work on design alternatives, and situates our work with respect to existing tools and related research artefacts.

2 Background The design process involves synthesizing alternatives and searching for alternatives [8] in a problem space. Akin [9] observed that expert architects explore several alternatives before further developing a solution. Extant CAD tools are inadequate because they do not afford interactions supporting design dialog that let designers rapidly explore multiple alternatives. Terry and Mynatt’s [5] note this deficiency as the Single-state document model, that is, an interaction model that recognizes and “requires a document to be in one, and only one, state at any particular time, thereby imposing a serial, linear progression through a task that is at odds with the “messy”, highly iterative creative process.” Overcoming single-stateness, therefore, is a prime motivation behind the design of interfaces to support design dialog with multiple alternatives. Terry and Mynatt provide the following interaction design guidelines to combat single-stateness. 1. Near-term experimentation. Provide a space of previews of the effect of the application of several commands. This will allow the users to compare the effect of application of diverse sets of commands and support “what-if” explorations. The rapid ability to perform what-if explorations lowers effort to “try something out” and thus supports near-term experimentation. 2. Support for Variations—interfaces should allow the existence of several variations of the same document to be simultaneously juxtaposed. Rich history

Interactive Visualization for Design Dialog

493

keeping of user interactions is recommended. Such history should be open to navigation and branching off at points to create more variations. 3. Evaluation—the inability to compare items side-by-side was found to be a chief problem, thereby necessitating juxtaposition. Alternative representations are also recommended. “Interfaces should thus allow the user to create multiple views and representations of their data, in its current, past, and potential future states.” By extrapolation, changes should be visible across both views and representations. We refer to the above points as multi-state interaction guidelines collectively and argue that they support our concept of design dialog by enabling analysis-synthesisevaluation cycles in the form of rapid generation, comparison, and subsequent modification of multiple alternatives. Inter alia, these ideas guide our designs.

3 Parametric Computer Aided Design (pCAD) We choose pCAD as the medium of choice to apply multi-state interactions in support of design dialog. pCAD intends to support rapid exploration of alternatives and is now a principal medium used by architects and building engineers. Parametric models (PM) intrinsically enable exploring rapid and iterative exploration of alternatives by varying parameter values that define the model state [10]. The relationships (size, angle, connection logic, etc.) among the component geometries are defined by parameters that form an “abstraction” of the design problem and solution that is formulated by the designer [11]. Most popular pCAD tools (GenerativeComponents, Grasshopper, and Dynamo) present graph-based visual programming interfaces for PM. Graph independent nodes form the input variables to the script. Sliders and circular knobs (for numerical and ordinal variables) and drop-down menus or lists (for ordinal or categorical variables) are the most typical user-interface elements used for inputs. Graph dependent nodes contain the outputs, which may be numerical or categorical data, and a visual representation like an image or 3D model. Thus, a parametric model can be abstracted as a design “vector” of inputs and outputs. The following shortcomings of pCAD tools motivate us to design interfaces that support multi-state design dialog. 1. Single-state Problems On changing inputs values, outputs update interactively. However, the previous state is overridden with every change in input. Therefore, though theoretically allowing the exploration of almost an infinite set of alternatives, cognitive limitations allow only the understanding of a few alternatives by varying one input at a time and does not allow comparison or simultaneous viewing of states. Workarounds such as saving separate files, and the export of design vectors to

494

A. Mohiuddin and R. Woodbury

spreadsheets are employed. This hampers design dialog and effective comparison among alternatives. 2. Lack of Multiple Representations Schön [1] and Akin [4] observed the use of diverse representations by architects during the design process. 3D models are usually the only available representation in pCAD. In terms of interface and visualization design, pCAD tools should support multiple representations, e.g., the image, 3D model, and multiple visual encodings of inputs and outputs, tightly coordinated [12] with each other. 3. Lack of Data Visualization Capabilities Interfaces of parametric tools lack integrated visualization tools to compare alternatives, posing challenges to understanding statistical correlations between the inputs and outputs, known as sensitivity analysis. Popular multi-variate data visualization techniques, such as scatter-plots and parallel coordinates are widely used. The design of adequate data visualization capabilities is a current research problem [11]. These shortcomings are the ‘problem setting’ for our research through design methodology.

4 Interactive Visualization as a Solution A basic requirement of multi-state interaction is the ability to compare alternatives side-by-side [5, 13] or juxtaposition. We propose novel interactivity to enhance visualizations to enable rapid generation, modification, and comparison of multiple alternatives.

4.1

Interactive Design Gallery

In our first iteration, we proposed Design Gallery (DG) [14, 15] for saving states recorded from a parametric model (Fig. 1). Each state, which we term as an alternative, was represented in DG as a thumbnail image and 3D model accompanied by a list of uniquely named input and output parameters. Alternatives could be restored to the pCAD program to retrieve the model state. An important included feature was the ability to generate alternatives from the cartesian product (CP) [16] of manually selected input parameters from alternatives in the gallery. Our studies [14, 15] revealed that designers preferred using DG and frequently used it to iterate over and generate alternatives. Among the shortcomings, the key finding was that better information visualization methods for parameters were necessary. Also, we did not include the ability to modify alternatives in the gallery itself, relying on existing single-state modeler functions only.

Interactive Visualization for Design Dialog

495

Fig. 1 Saving alternatives from a Grasshopper™ definition. a The plugin sends the data captured to the gallery when a user clicks the Save to Gallery button. This data is also stored in a database. b A selection of input sliders with unique names. Notice the stacked arrangement resembles horizontally aligned parallel coordinate axes c A selection of output values with unique names. For both inputs and outputs, items may be added or removed as the user progresses with the model and saves more alternatives. The gallery will reflect this. d A collection name to receive the alternative (this may be changed once the alternative is already in the gallery). e A username for identification as several users can save to the same gallery. f Text tags that get attached to this alternative. These too may be modified once the alternative is in the gallery. Alternatives can be restored from the gallery to the modeler as the user is working on a model. The input values are applied to the current model. If the user follows a similar naming convention for input parameters, then values from one model can not only be applied to different models but different modelers as well, as we have demonstrated in our previous work. Plug-ins for Dynamo™ and GenerativeComponents™ are underway

5 Dynamic Exploration Using Parallel Coordinate Plots We propose the use of interactive parallel coordinate plots (PCPs), to address the shortcomings of the first iteration of DG. In a vast parameter space, infinite combinations exist and the relationship between inputs and outputs is not well understood [17]. In such cases, Spence [18] recommends interfaces that support “what-if” exploration, such as sliders, to allow users to “explore incrementally and dynamically” as both a filtering mechanism (e.g. in housing search with a finite data space) and solution generation or information synthesis (e.g. mortgage calculator with an infinite data space). The trial and error navigation of parameter spaces [19] in pCAD tools is another such manifestation of dynamic exploration for information synthesis, also achieved using sliders. Our interactions extend dynamic exploration of parameter spaces to be multi-state, i.e. involving multiple alternatives simultaneously, and enable the qualities of design dialog.

496

A. Mohiuddin and R. Woodbury

The following elaborates how PCPs enable dynamic exploration. Consider each axis of a PCP to have the affordance of a slider. Sliders can have two handles to signify a range. Several values of each parameter can also be represented by multiple handles [13]. Sliders are also typically placed next to each other in parallel, either horizontally or vertically, in most user interface designs. Connecting handles with a polyline (per alternative) on such “parallel sliders” will imbue them with all the information visualization benefits of PCPs as well as interaction qualities of sliders rolled into one. Dragging these handles along the axes modifies input parameter and edits values. This will not only allow exploring alternatives like using sliders in regular parametric modelers, but also enable changing multiple inputs (across both alternatives and parameters) and observing results simultaneously (see Fig. 7). To the best of our knowledge, there exists no implementation of PCPs in which variables can be directly modified. In addition to this capability, the following are more reasons to extend the interactivity of PCPs. 1. Pattern detection is important especially for interactive trade space exploration. PCPs “transform the search for relations among the variables into a 2-D pattern recognition problem.” [21]. 2. PCPs can be learned with minimal training [22] 3. PCPs are effective as multi-dimensional visualization tool, they have wide usage and there exists a large body of literature on improving readability, scaling, and interactivity for information exploration [23, 24]. 4. They have been widely adopted in practice [11] and are found in an increasing number of commercial tools and research prototypes. 5. From the perspective of the design domain, PCPs match the appearance of aligned, stacked sliders for input parameters—a universal arrangement convention in graph-based parametric modelers.

6 Design and Features In this section we describe the implemented and proposed novel interactions of the parallel coordinate plot, henceforth referred to as the parallel coordinate controller (PCC). The PCC is another view of the data in a collection of alternatives. Alternatives in the DG can be arbitrarily grouped in collections by users. Collections can be viewed as the default thumbnail view (Fig. 2), the PCC, or as a scatter-plot matrix (Fig. 4), or all three views simultaneously. Hovering or selecting alternatives brushes them across views. Hovering over thumbnails brings up a card view (Fig. 3) with an enlarged thumbnail that can be toggled to a 3D model, and details of parameters in a tabular form. The card also contains a field for entering text tags, that are applied to all selected alternatives. Alternatives can be selected using these tags. A work-in-progress query language (Fig. 2) allows more granular selection of alternatives by parameter values.

Interactive Visualization for Design Dialog

497

Fig. 2 Default thumbnail view of alternatives in three collections in the Design Gallery. The highlighted (red) thumbnails represent selected alternatives, and some occur in multiple collections. The work-in-progress query language as a selection mechanism (Upper right, blue). The alternatives with the least ten values of output Residential_Area have been selected as the result of this query

In the PCC (Fig. 5), each alternative vector is represented in typical parallel coordinates style as polylines intersecting each attribute axis at the corresponding values, marked by a circular handle. Axis display can be toggled on or off from a drop-down menu. Sliders can be reordered. The following are implemented key features of the PCC design, including some proposed extensions to these features.

6.1

Selection Techniques: Brushing and Filtering–

1) Direct selection (brushing)—clicking or rectangle selection of polylines or handles selects alternatives. Brushed states are colour-coded—blue for hovered, red for selected, and darker red for selected and hovered (Fig. 5). Un-brushed items are grey. As mentioned, brushing is linked across visualizations in the gallery. Hovering brings items to the foreground and aids selecting overlapped polylines. We propose a pop-up, numbered selection list of alternatives for selecting in regions of heavy overlap. 2) Filtering with range bars created by double-clicking below axes can perform dynamic queries. Multiple, adjustable and re-orderable range bars can be created per axis thus allowing visual construction of a sub-set of Boolean queries (Fig. 6).

498

A. Mohiuddin and R. Woodbury

Fig. 3 On hovering over a thumbnail, a card with details of the alternative pops up. The input and output data is in tabular form. Scrolling further down reveals more data and the tagging interface. Tags may be added or deleted, and clicking a tag selects all alternatives with that tag

Interactive Visualization for Design Dialog

499

Fig. 4 Scatterplot matrix view (cropped to fit) of a collection. Dimensions to plot are chosen from a drop-down menu (in this case BaseRotation, Residential_Area, Commercial_Area, and Retail_Area). Three alternatives have been selected, and they are simultaneously highlighted in all plots, and also the default collection view named “SG2018” (left) in which they occur. In this particular model, outputs are highly positively correlated, that is, they tend to vary together

Fig. 5 Parallel coordinate view of a collection with five alternatives. a Attributes to be visualized can be chosen from a dropdown menu. b Input axes c output axes d selected and hovered alternative. Hovering brings the item to the top. Hovering nearing a handle shows value in tooltip. e selected alternative. d has one output value (Retail_Area) less than the unselected (grey) alternatives e has one output value more (Total_Area) than the rest (as described in Fig. 1). Axes can be reordered by dragging handles f. Note that the output areas show strong positive correlation with each other in this model, hence the nearly straight output plots. This relationship is also revealed in the scatterplots (Fig. 4)

500

A. Mohiuddin and R. Woodbury

Fig. 6 Selection using range bars. Range bars can be moved and extended, and each axis can have multiple range bars that may be separate or overlapped. Currently multiple ranges on each axis equal a Boolean OR operation and Boolean AND across axes. We plan to modify this behavior to include other Boolean operators to allow construction of complex visual queries

6.2

Multi-state Dynamic Exploration Techniques

1. Manual methods and direct manipulation—dragging a handle of an input parameter horizontally to a new value edits the alternative by recomputing the PM and updating the thumbnail. A remote computer executes this computation. Dragging polyline segments move connected handles in tandem. Multiple segments and handles across alternatives can be selected and dragged in tandem for parallel, simultaneous edits (Fig. 7). Pressing a modifier key creates a copy of selected alternatives and lets users edit non-destructively. These techniques let users extend the manual trial and error operations to multiple alternatives. For future work, we propose previews [25] of edit operations results as a look-ahead. Letting users choose from these previews supports near-term experimentation.

Interactive Visualization for Design Dialog

501

Fig. 7 Parallel editing of two selected alternatives a and b. In (1) handle of a at BaseRotation is dragged from value 0.9 to 1.2. BaseRotation value of b moves in tandem by equal amount. Values may be entered directly into the appearing tooltip, and the delta will be applied across selected alternatives. Thumbnails indicate processing has begun and on completion of computation, the images and the output plots will update. In (2) Dragging multiple sets of handles across multiple axes enables parallel editing across multiple loci. We also plan to generate thumbnail previews of edit operations

2. Goal-based generation—“sketching” a polyline across input axes with the mouse cursor creates a new polyline (Fig. 8). The values at its intersections with the axes are used to compute a new alternative. This technique allows users compose alternatives with a specific sets of values rapidly. Handles of existing alternatives, i.e. input parameters already present on different alternatives may be “sketched” across as well with a polyline, thus allowing the generation of alternatives from users’ previous parametric explorations, an operation very desirable for exploring alternatives using history [26]. 3. Automated generation—We chose to implement the Cartesian product (CP) generator (Fig. 9) as the first method of automated generation, as included in the first DG iteration. In the PCC, users select polylines, segments or input handles. From the input values contained within these selected items, a CP is calculated. Each result of the CP is a new alternative vector that populates the gallery. This is similar to alternative synthesis through merge and recombination of previously explored input values in parametric models [27]. CPs can be very large, and one obvious reduction is to generate only their boundary, that is, the set of alternatives generated by taking the CPs of only the extreme values of each parameter.

502

A. Mohiuddin and R. Woodbury

Fig. 8 Creating a new alternative by “sketching” a polyline across axes. It may be achieved by double-clicking on desired points on the axes (cursor) or by a smooth drawing action (dotted line). The handles of the resulting green line can then be adjusted by dragging before committing to the values. Existing handles may also be clicked. A new alternative is computed by applying these values to a default model

Fig. 9 a Two alternatives selected for cartesian product. b The result of the cartesian product. Thumbnails show new alternatives being created

Interactive Visualization for Design Dialog

503

7 Evaluation We conducted informal, open-ended interviews with thirty (n = 30) domain experts by demonstrating the prototype or allowing casual play to receive general comments and feedback without any formal procedure. This method of evaluation, though not comprehensive, is valid and common in visualization design research [28] and used to assess “intuitiveness and functionality,” “probe for utility and usability” and “evaluate and improve implementation of the ideas.” The interviewees included tool builders with CAD companies; researchers within practice and academia; and expert users in computational design—the latter group in a multi-day workshop. Several of these experts had used gallery display systems such as the Thornton Tomasetti Design Explorer 2 or had developed their own informal tools and workflows for alternatives. Two researchers reviewed notes and recordings of the interviews taken by one researcher, and the following themes occurred. Phrases in quotes are actual statements. 1) “Live connection with modeler”—a strong liking for the ability to restore inputs values to the pCAD program was noted. Dislike for the approach in extant tools like Design Explorer 2 was expressed where pre-generation of a data set is required, which cannot be further modified. This results in extreme premature commitment and high viscosity according to Cognitive Dimensions [29]. 2) Use as a “presentation tool”—interviewees agreed that the DG+PCC was a good analytic tool to present design alternatives to different stakeholders. It contains better features for organization, such as collections and tags, and better query capabilities compared to popular tools. 3) “Standalone” model editing tool—the capability to edit models in the “cloud” without requiring an installation of the pCAD tool elicited several positive responses. Interviewees claimed that this could allow clients or other stakeholders to experiment with the model without worrying about losing original content, like a “sand-box.” One interviewee claimed that they would use it to explore their designs remotely from a lightweight portable device like a tablet. One interviewee remarked they would use the cloud computation capability to batch process large numbers of alternatives. 4) Novel Design Space Exploration Tool—One interviewee stated that they would presume that any perceived “gaps” in the PCC represented unexplored design space, and that they would expect sketching a line through such gaps, to include both inputs and outputs, should return the feasibility of such an alternative or its nearest neighbor. Another asked, “Can you draw over outputs?” Here, we cannot deny the suggestion of rich interactions that can be designed around ‘sketching’ or modifying of outputs, along the lines of inverse design [30], where alternatives are computed from specified output values. We leave this for future work.

504

A. Mohiuddin and R. Woodbury

Experts made many suggestions for using the PCC to interactively generate new alternatives. The PCC can provide devices not only for composing alternatives directly but also for quickly creating large sets of related alternatives. The ideas we heard include selecting two or more alternatives (paths through the PCC) as arguments to a Cartesian product operator (or a reduced operator that, for instance, uses only minimum and maximum values for each parameter); using a single alternative as the argument for a sensitivity analysis that shows how parameters individually and collectively affect the design; selecting multiple alternatives and then simultaneously modifying some of their parameter values (parallel editing); selecting an output value and an alternative and using the output value as a goal for a search algorithm modifying the alternative; and using selections parameters to update other alternatives simultaneously. These diverse proposals for alternative generation and modification, if implemented, suggest how users would likely engage with the tool, in a manner strongly suggesting our notion of design dialog. None of these operations are practical in a parametric modeler’s single state interface, supporting the logic of a separate and novel interface based on interacting with collections of alternatives. Further, several of the experts described work in the PCC as supporting different goals and tasks. One reviewer used the phrase “workshop feel” to describe using the gallery for tasks involving multiple what-if scenarios. Another described the PCC as combining engineering inquiry with visual play over designs. All experts gave ideas for extending the PCC and its coordinated views. This reveals the design space of such interfaces. 5) Coordinated and Multiple Views—Parallel coordinates are good for analyzing overall trends in multi-variate data, but poor at explaining bi-variate relationships. Comparatively, scatterplots are better at visualizing bi-variate relationships, and therefore the combination of the two work best [24]. This is also the basic interface for the Design Explorer 2 and Autodesk’s Refinery. Each coordinated and multiple view [12] provides specific kinds of interactions and supports different tasks. All experts agreed that such view complexes are needed. 6) Collections—Interviewees said they would use collections. This is consistent with Shireen et al.’s [31] observation on how designers focus only a few subsets of alternatives at a time. More research is needed on collection management. 7) Inputs Propose: Outputs Dispose—Designs have purpose and how well they fit that purpose plays a central role in design work. Some purposes can be made explicit; and the term design performance is often used to describe such and are typically quantitative. Other purposes remain implicit and are matters for discourse and professional judgement, for example, issues of architectural form and composition. The latter are typically implicit, that is, recognized and not computed. Parametric models typically capture some of the explicit purposes as outputs, for example, energy use, daylighting and cost may each be calculated from a design. Almost every expert asked us, in some form, to show how outputs are shown in the interface and what actions can be based on them. For instance, we repeatedly were asked how selecting output ranges could make new

Interactive Visualization for Design Dialog

505

collections; how output values could be used to specify goals for automated generation and search; and to demonstrate new ways of comparing designs based on their outputs. Inputs propose solutions whereas outputs help organize them for further design work. 8) Make-Filter-Repeat—Two groups of experts (we interviewed some expert teams within firms as groups) described a repetitive workflow of (1) taking a sample of the design space (by varying parameters); (2) querying or filtering the sample for “interesting” designs; and (3) repeating the process based on the set of interesting designs. We note that this pattern resembles the idea of design dialog too, where we see rapid generation of alternatives, which are then evaluated (filtered), and refined either by modification or further narrowed sampling, and the whole process repeated. This pattern was observed in the first version of DG as well, but the inclusion of the PCC affords a better analysis of alternatives and improved exploration tools.

8 Conclusion and Future Work The feedback received during the evaluation has been positive and has not only confirmed the potential usefulness of the system in design space exploration with the properties of design dialog, but also that our design directions are mostly valid. The make-filter-repeat strategy closely resembles the analysis-synthesis-evaluation cycles that we were aiming to enable to support design dialog. It has also opened up a range of possibilities for new interface designs where direct interaction with visualizations leads to generation and modification of alternatives. Our resulting design of a tool, the potential usefulness of this tool in addressing a problem setting, and the opening up of a space of further interaction design, meets the expectations of research outcomes as stated in the research through design methodology [7]. In interaction design, the ‘usefulness’ of the resulting artefact (as a prototype) is more an indicator of research success rather than rigor in choosing methods [32]. Three issues need our immediate attention. (1) Cognitive overload due to over-plotting [23] in the PCC and too many thumbnails [33] in the gallery, and is a typical problem suffered by parallel coordinate plots in general. Hierarchical data clustering [34] is a potential solution to this problem. (2) We have assumed an invariant parametric model, however, in practice architects constantly modify the parametric model itself as the design problem formulation changes, which may add or remove input and output parameters. We have partially addressed this issue (see Fig. 5) and continue to do so for the future. (3) We have designed the PCC only to support numerical values for now. We are aware that parameters are often categorical variables. Both (2) and (3) can be solved with designs like Parallel Sets [35] for categorical data. For (3) specifically, each parametric model can be represented as a categorical variable.

506

A. Mohiuddin and R. Woodbury

Fig. 10 Converging to a target configuration using DG+PCC. The curves are traced out by a strut on configurations of a 4-bar linkage controlled by five input parameters. Four outputs give a measure of the how close the drawn curve is to the target, such as inter-curve distance, centroid distance, difference in length, and ratio between lengths. The curve traced by the current bar configuration (blue) is in green, while the target curve is in cyan. The left-most image shows a close match. The model is unpredictable, in that the relationships between inputs, outputs, and resultant curves are hard to discern, and discontinuous, because some configurations produce broken curves. This is a type of design problem (goal-based), among others, with which we plan to test the too

For the next iteration, we plan to conduct empirical studies on the actual use of the tool and investigate whether there is an improvement in performance of the designs produced, the user experience of the tool, and observe strategies and patterns in design exploration (designer behavior) [28]. A pilot study with two (n = 2) participant, who were asked to match a target parametric 4-bar linkage configuration (Fig. 10), showed promise. Participants leveraged DG+PCC’s rapid generation capabilities to “converge” to the solution. We will report the results of the full study in the future to convey whether interactions with the tool resembled design dialog, and their impact on the design process. Acknowledgements This work was supported by the Natural Sciences and Engineering Research Council of Canada, Bentley Systems Inc., and Smartgeometry.

References 1. Schön DA (1983) The reflective practitioner: how professionals think in action. Basic Books, New York 2. Goldschmidt G (1991) The dialectics of sketching. Creat Res J 4(2):123–143 3. Woodbury R, Mohiuddin A (2019) Dialogues of exploration: new tasks, new tools. In: Computational and cognitive models of creative design 4. Buxton (2007) Sketching user experiences getting the design right and the right design, 1st edn. Elsevier/Morgan Kaufmann 5. Terry M, Mynatt ED (2002) Recognizing creative needs in user interface design. In: Proceedings of the 4th conference on creativity & cognition, New York, NY, USA, pp 38–44. https://doi.org/10.1145/581710.581718. 6. Cross N (2001) Chapter 5–design cognition: results from protocol and other empirical studies of design activity. In: Eastman CM, McCracken WM, Newstetter WC (eds) Design Knowing and Learning: Cognition in Design Education. Elsevier Science, Oxford, pp 79–103 7. Zimmerman J, Forlizzi J, Evenson S (2007) Research through design as a method for interaction design research in HCI. In: Proceedings of the SIGCHI conference on Human

Interactive Visualization for Design Dialog

8. 9.

10.

11.

12.

13.

14.

15.

16. 17.

18.

19.

20. 21.

22.

23. 24.

25.

507

factors in computing systems–CHI 2007, San Jose, California, USA, p 493. https://doi.org/10. 1145/1240624.1240704. Simon HA (1996) The science of design: creating the artificial. In: The sciences of the artificial, MITP Akin Ö (2001) Chapter 6–variants in design cognition. In: Eastman CM, McCracken WM, Newstetter WC (eds) Design knowing and learning: cognition in design education. Elsevier Science, Oxford, pp 105–124 Robert W (Robert Francis) (2010) Elements of parametric design/Robert Woodbury ; with contributions by Onur Yüce Gün, Brady Peters and Mehdi (Roham) Sheikholeslami. Routledge, London; New York Bradner E, Iorio F, Davis M (2018) Parameters tell the design story: ideation and abstraction in design optimization. In: Proceedings of the symposium on simulation for architecture & urban design, San Diego, CA, USA, 2014, pp 26:1–26:8. http://dl.acm.org/citation.cfm?id= 2664323.2664349. Accessed 11 Oct 2018 Roberts JC (2007) State of the art: coordinated & multiple views in exploratory visualization. In: Coordinated and multiple views in exploratory visualization, 2007. CMV 2007. Fifth international conference on, pp 61–71. https://doi.org/10.1109/CMV.2007.20 Lunzer A, Hornbæk K (2008) Subjunctive interfaces: extending applications to support parallel setup, viewing and control of alternative scenarios. ACM Trans Comput Human Inter 14(4):1–44. https://doi.org/10.1145/1314683.1314685 Mohiuddin A, Woodbury R, Ashtari N, Cichy M, Mueller V (2017) A Design gallery system: prototype and evaluation. In: ACADIA 2017: disciplines & disruption proceedings of the 37th annual conference of the association for computer aided design in architecture (ACADIA) Cambridge, MA 2–4 November, 2017, pp 414–425. http://papers.cumincad.org/cgi-bin/ works/Show?acadia17_414. Accessed 11 Oct 2018. 978-0-692-96506-1 Woodbury R, Mohiuddin A, Cichy M, Mueller V (2017) Interactive design galleries: a general approach to interacting with design alternatives. Des Stud 52:40–72. https://doi.org/ 10.1016/j.destud.2017.05.001 Woodbury RF, Burrow AL (2006) Whither design space? AI EDAM 20(2):63–82. https://doi. org/10.1017/S0890060406060057 Tweedie L, Spence R, Dawkes H, Su H (1996) Externalizing abstract mathematical models. In: Proceedings of the SIGCHI conference on human factors in computing systems, New York, NY, USA, p 406. https://doi.org/10.1145/238386.238587 Spence R (2014) Interaction. In: Spence R (ed) Information visualization: an introduction. Springer International Publishing, Cham, pp 173–223. https://doi.org/10.1007/978-3-31907341-5 Sedlmair M, Heinzl C, Bruckner S, Piringer H, Möller T (2014) Visual parameter space analysis: a conceptual framework. IEEE Trans Visual Comput Graph 20(12):2161–2170. https://doi.org/10.1109/TVCG.2014.2346321 Thornton Tomasetti CS (2015) Design explorer. http://tt-acm.github.io/DesignExplorer/ Inselberg A (2018) Visual data mining with parallel coordinates. In: Social science research network, Rochester, NY, SSRN Scholarly Paper ID 85868, https://papers.ssrn.com/abstract= 85868. Accessed 01 Aug 2018 Siirtola H (2000) Direct manipulation of parallel coordinates. In: 2000 IEEE conference on information visualization. An international conference on computer visualization and graphics, London, UK, pp 373–378. https://doi.org/10.1109/IV.2000.859784 Heinrich J, Weiskopf D (2013) State of the art of parallel coordinates. In: Eurographics (STARs), pp 95–116. https://doi.org/10.2312/conf/EG2013/stars/095-116 Johansson J, Forsell C (2016) Evaluation of parallel coordinates: overview, categorization and guidelines for future research. IEEE Trans Visual Comput Graph 22(1):579–588. https://doi. org/10.1109/TVCG.2015.2466992 Terry M, Mynatt ED (2002) Supporting experimentation with side-views. Commun ACM 45 (10):106–108. https://doi.org/10.1145/570907.570942

508

A. Mohiuddin and R. Woodbury

26. Terry M, Mynatt ED, Nakakoji K, Yamamoto Y (2004) Variation in element and action: supporting simultaneous development of alternative solutions. In: Proceedings of the SIGCHI conference on human factors in computing systems, New York, NY, USA, pp 711–718. https://doi.org/10.1145/985692.985782 27. Zaman L, et al. (2015) GEM-NI: a system for creating and managing alternatives in generative design. In: Proceedings of the 33rd annual ACM conference on human factors in computing systems, New York, NY, USA, pp 1201–1210. https://doi.org/10.1145/2702123. 2702398. 28. Lam H, Bertini E, Isenberg P, Plaisant C, Carpendale S (2012) Empirical studies in information visualization: seven scenarios. IEEE Trans Visual Comput Graph 18(9):1520– 1536. https://doi.org/10.1109/TVCG.2011.279 29. Green TRG, Petre M (1996) Usability analysis of visual programming environments: a `cognitive dimensions’ framework. J Vis Lang Comput 7:131–174 30. Coffey D, Lin C, Erdman AG, Keefe DF (2013) Design by dragging: an interface for creative forward and inverse design with simulation ensembles. IEEE Trans Visual Comput Grap 19 (12):2783–2791. https://doi.org/10.1109/TVCG.2013.147 31. Shireen N, Erhan H, Woodbury R, Wang I (2017) Making sense of design space. In: Çağdaş G, Özkar M, Gül LF, Gürer E (eds) Computer-aided architectural design. Future trajectories. Springer Singapore, Singapore, pp 191–211. https://doi.org/10.1007/978-981-10-5197-5_11 32. Fallman D, Stolterman E (2010) Establishing criteria of rigour and relevance in interaction design research. Dig Creat 21(4):265–272 33. Gleicher M, Albers D, Walker R, Jusufi I, Hansen CD, Roberts JC (2011) Visual comparison for information visualization. Inf Vis 10(4):289–309. https://doi.org/10.1177/ 1473871611416549 34. Beham M, Herzner W, Gröller ME, Kehrer J (2014) Cupid: cluster-based exploration of geometry generators with parallel coordinates and radial trees. IEEE Trans Visual Comput Graph 20(12):1693–1702 35. Bendix F, Kosara R, Hauser H (2005) Parallel sets: visual analysis of categorical data. In: IEEE Symposium on information visualization, 2005. INFOVIS 2005, pp 133–140. https:// doi.org/10.1109/INFVIS.2005.1532139.

Composing Diverse Design Teams: A Simulation-Based Investigation on the Role of Personality Traits and Risk-Taking Attitudes on Team Empathy Mohammad Alsager Alzayed, Scarlett R. Miller, and Christopher McComb

Abstract Empathy is known to help engineering designers develop a deeper understanding of the users’ needs. However, prior research on individuals has identified that individual differences, such as personality traits and risk-taking attributes, could significantly impact designer’s empathy. While this prior research provides a context for why individual differences may impact a designer’s empathy, engineering design activities are typically deployed in teams, and thus a team-centered view of empathy is also needed. As such, the goal of the current study was to investigate the role of the diversity in personality traits and risk-taking attitudes in impacting team empathy in engineering design. This was accomplished through a computational simulation of 13,482 teams generated by a statistical bootstrapping technique drawing upon a data set from 103 first-year engineering students. When composing highly empathic teams, the results suggest that we should compose teams that are diverse in the extraversion, openness, and neuroticism personality traits in addition to being diverse in ethical, health/safety and social risk-taking. This is important since empathy could allow design teams to deeply understand the needs of diverse users and subsequently solve those users’ problems.

1 Introduction The ability to understand the feelings and circumstances of others [1], also known as empathy, has proven to be an effective driver of creative design outcomes [2] and thus is seen as a core constituent of engineering design [3]. While this prior work S. R. Miller The Pennsylvania State University, University Park, PA, USA M. A. Alzayed Kuwait University, Kuwait City, Kuwait C. McComb (&) Carnegie Mellon University, Pittsburgh, PA, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. S. Gero (ed.), Design Computing and Cognition'20, https://doi.org/10.1007/978-3-030-90625-2_30

509

510

M. A. Alzayed et al.

highlighted the importance of empathy in engineering design, research has also identified that individual differences may impact an individual’s empathy [4–8]. Two such individual differences that are critical in shaping the design outcomes of engineering designers and that have been found to influence empathy are personality traits [5, 8, 9] and risk-taking attitudes [6, 7, 9]. For example, Eysenck et al. [8] found trait empathy to be positively correlated to the personality attribute neuroticism while Shu et al. [6] reported that individuals that take an empathic perspective were also found to be risk averse. While this line of research provides some evidence on the role of personality and risk taking on an individual level, we do not know how they impact team empathy. This is problematic because teams are an essential component of engineering design [10], due to their ability to support problem solving [11] and improve the exploration of the solution space [12]. Studying empathy in a team setting is of particular importance in this study due to the belief that a team of designers could have the ability to come up with more creative solutions than individual designers alone [13]. As such, investigating the impact of the composition of teams on team empathy is warranted. Specifically, the diversity of team attributes in terms of risk taking attitudes and personality attributes is of interest in the current study due to prior research that has identified the impact of diversity (e.g. racial [14], cognitive [15], cultural [16]) on empathic behavior. This also resonates with Stephen Covey’s popular philosophy that states that “strength lies in differences, not similarities” [17]. Formalizing the role of team diversity in personality traits and risk-taking on team empathy will provide the design community with a better understanding of how to formulate empathic design teams. As such, the objective of this paper was to identify the role of team personality and risk-taking attributes on team trait empathy. We turn our attention to studying trait empathy, a dispositional quality that allows for the understanding of the emotions, circumstances, and needs of others [1], as opposed to designers’ perceived empathy, due to prior research that reported that one’s empathic tendencies can be most accurately identified with their trait empathy [4]. Specifically, this paper sought to study the average (commonly referred to as elevation [18]) and standard deviation (commonly referred to as diversity [18]) of team’s trait empathy, due to recent research that found that both empathy elevation and diversity impacted creative design outcomes in the design process [19]. This research is one of the first to study trait empathy on a team level and provide some of the first insights on the empathic composition of teams in engineering design.

2 Related Work In order to lay the foundation for the current investigation, it is critical to review the literature on the impact of personality traits and risk-taking attitudes on an individual’s empathy, and how diversity could impact design teams’ empathy. Personality traits and risk-taking attitudes are the focus in the current study due to prior work that found that those two attributes are critical in shaping the design

Composing Diverse Design Teams: A Simulation-Based Investigation …

511

outcomes of engineering designers [9] and that have been found to influence an individual’s empathy [5–8]. On an individual level, personality traits have been found to impact an individual’s empathy. For example, Eysenck et al. [8] found trait empathy to be positively correlated to the personality attribute neuroticism while exploitativeness in the narcissism scale was negatively related to three different empathy scales [20]. Additionally, risk-taking attitudes, another facet of personality that is related to design outcomes in engineering design [9], were also found to be related to trait empathy. Specifically, individuals that take an empathic perspective were found to be risk averse [6]. This phenomenon has also been confirmed in a behavioral study by Ogawa et al. [7] that found that empathic concern as measured by Davis’s Interpersonal Reactivity Index (IRI) [4] was related to risk-averseness. While prior work highlights the role of those individual characteristics (e.g., personality, risk-taking, etc.) as predictors of empathy, that previous line of research is not based on an engineering sample, and has not been studied on a team level. On a team level, diversity in personality attributes been used as a factor to form teams in engineering design for the purposes of driving innovation [21], productivity [22, 23], and leadership [24, 25], but not empathy. By measuring team personality elevation and team personality diversity [18], Neuman, Wagner, and Christiansen concluded that teams with diverse personalities performed better on tasks on an unmanned aerial vehicle control system [21]. On the same line of research, Sook Kim et al. [26] found that diversity, in terms of team members’ creative modes, positively impacted team cohesiveness. In the context of empathy, prior research has reported that diversity could be a mediator to empathic behavior in social settings [14]. In engineering design, Wong, Sorris, and Siddique [27] claim that empathy is a precursor to an inclusive and diverse environment in engineering design. While that body of research suggests a linkage between diversity and empathy, it has not empirically studied the relationship between team empathy and personality diversity, or diversity in risk-taking attitudes.

3 Research Design and Methodology Based on this prior work, the objective of this paper was to identify the role of team personality and risk-taking attributes on team trait empathy. This paper sought to study the average (commonly referred to as elevation [18]) and standard deviation (commonly referred to as diversity [18]) of team trait empathy. The factors studied in this investigation are summarized in Fig. 1. Specifically, the following hypotheses were devised: 1. Diversity in personality traits could be used to predict higher levels of trait empathy elevation since prior research has reported that diversity could be a mediator to empathic behavior [14–16].

512

M. A. Alzayed et al.

Fig. 1 Summary of the factors studied in the current investigation

2. Diversity in risk-taking attitudes could be used to predict higher levels of trait empathy elevation since prior research has reported that diversity could be a mediator to empathic behavior [14–16]. The remainder of this section highlights the experimental procedure aimed at addressing these research questions.

3.1

Participants and Procedure

The data set for this study was derived from a study conducted with 103 first-year engineering students (73 men and 30 women) from four sections of a cornerstone engineering design course that participated in an 8-week design project [28]. The design project focused on the following engineering challenges: (1) lack of safe water, sanitation, and hygiene services, (2) access to vaccinations, (3) indoor and ambient air pollution, and (4) road traffic injuries [28]. While design teams were tasked with the same design problems, two of the four course sections focused on designing for the developing world, while the remaining sections focused on designing for the developed world. Prior to the start of the study, participants completed a 28-item survey that measured their trait empathy. They were also asked to complete a 30-item survey assessing their risk-taking attitudes and a 120-item personality test as homework assignments in week 5. The data set from the previous study [28] was used to run a computational simulation of 13,482 nominal groups [29]. Nominal groups are defined as individuals that work independently and pool their solutions together

Composing Diverse Design Teams: A Simulation-Based Investigation …

513

near task completion [29] and have been used in simulation-based research since this grouping technique has been found to foster input from all team members [29], as well as boost team productivity [30]. Specifically, the aim of the simulation was to study the role of personality traits and risk-taking attributes in impacting team empathy.

3.2

Data Collection Instruments

This section summarizes the metrics used to explore the factors critical to achieving the research objectives.

3.2.1

Trait Empathy

Participants’ trait empathy was measured using the interpersonal reactivity index (IRI) [4], a 28-item survey answered on a 5-point Likert scale ranging from “does not describe me well” to “describes me very well”. The IRI was utilized in prior research in assessing the empathic tendencies of engineering students [31], and includes the following 4 subscales: perspective taking, fantasy, empathic concern, and personal distress. First, to obtain a trait empathy score for each of the 103 participants, the IRI scores on empathic concern, fantasy, and perspective-taking were averaged for each participant [4]. Personal distress was not included in the analysis since it has been found to be negatively related to the other three IRI subscales [4], and hence adding personal distress with the other three IRI subscales is not advised [32]. Second, to obtain a trait empathy score for each of the 13,482 simulated teams, team empathy elevation and team empathy diversity were considered. Team empathy elevation takes the average across all team members’ trait empathy scores while team empathy diversity takes the standard deviation of team members’ trait empathy scores [18].

3.2.2

Personality Traits

Personality traits were measured using the short Five Factor Model online questionnaire [33], a short form of the International Personality Item Pool Representation of the Revised NEO Personality Inventory [33]. Specifically, the results of the 120-item survey assign participants a score (ranging from 0 to 100) on the five personality traits: extraversion, agreeableness, conscientiousness, neuroticism, and openness. To obtain personality scores for each simulated team, team personality diversity [18] was considered i.e. the standard deviation for each of the personality traits.

514

3.2.3

M. A. Alzayed et al.

Risk-taking Attitudes

Risk taking attitudes were measured using a 30-item shortened version of the psychometric domain-specific risk-taking scale [34] due to the subscales’ relevance in design-related tasks and its prior utilization in engineering research [9]. The instrument includes the following subscales: health/safety, ethical, social, recreational, and financial risk-taking attitudes. To obtain risk-taking scores for each simulated team, team risk-taking diversity was considered for each of the five risk-taking subscales.

3.3

Simulation Procedure

Due to the potential cost and time associated with user studies [35], a computational simulation was used in the current study to model a large sample of design teams. The use of simulation modelling is justified with prior research that found that computational simulated teams accurately resembled characteristics of human teams [36]. In this study, four-person teams were simulated from the 103 participants. Specifically, the simulation setup controlled for instructor, design context, and design problem; participants from different design problems, contexts, or instructors were not mixed in the same team, due to prior work that found that these factors might impact designer’s empathy [19]. While every possible team member type was considered, each type can be satisfied by a number of different participants in the dataset. In the simulation, the selection of those participants to form the teams is thus randomized. Each of the 9 team types included different combinations of all participants to get every combination with replacement. This technique closely relates to the statistical bootstrapping technique, a technique that involves “re-sampling the data with replacement many times to get an empirical estimate of the entire sampling distribution” ([37], p. 1). This method has been employed by Wright [38] to create nominal groups in a prior study and has been implemented to model nominal teams in engineering design research [35, 36].

4 Results and Discussion The main goal of the current study was to investigate the role of personality traits and risk-taking attitudes in impacting team empathy in engineering design. Based on prior work [14, 15], we hypothesized that diversity in personality traits could be used to predict higher levels of trait empathy elevation. Similarly, it was hypothesized that the diversity in risk-taking attitudes could be used to predict higher levels of trait empathy elevation [14, 15]. In order to answer our research questions, statistical analyses were computed using SPSS 25.0, and a significance level of 0.05

Composing Diverse Design Teams: A Simulation-Based Investigation …

515

was used in all analyses. In addition, effect sizes were classified according to Cohen [39]. In order to understand the role of personality traits and risk-taking attitudes on team trait empathy, two linear regression models were computed to predict simulated teams’ (1) empathy elevation and (2) empathy diversity from the diversity of the personality traits (openness, extraversion, neuroticism, agreeableness and conscientiousness), and the diversity of the risk-taking attitudes (social, ethical, health/ safety, recreational, and financial). Prior to the analysis, statistical assumptions were checked. The results showed the linearity of the independent variables as assessed by partial regression plots and a plot of studentized residuals against the predicted values. By visual inspection of a plot of studentized residuals, the assumption of homoscedasticity was met. There was no multicollinearity in the independent variables, as assessed by tolerance values greater than 0.1. As assessed by the studentized deleted residuals greater than ±3 standard deviations, there were 25 and 14 outliers for the first and second regression models, respectively. The outliers were found to have no significant impact on the significance of the results and therefore, the full analysis is presented here. Additionally, there were no leverage values greater than 0.2, and no values for Cook’s distance above 1. Finally, normality was confirmed by visually inspecting the histograms and Q-Q plots. Based on these results, the analysis proceeded as planned. The results of the first regression model showed that the diversity in personality attributes and risk-taking attitudes significantly predicted team empathy elevation, F (10,13,465) = 580.563, p = 0.008, R2 = 0.301, a medium effect size, see Fig. 2 for a summary of the contributing predictors. Specifically, the diversity in team extraversion, neuroticism, and openness as well diversity in ethical, health/safety, and social risk-taking promoted team empathy elevation while the remaining personality traits and risk-taking attitudes negatively predicted team empathy elevation. While the first regression model investigated the role of personality traits and risk-taking attributes on team empathy elevation, the second regression model investigated the impact of those factors on team empathy diversity. The results from the second regression model showed that team empathy diversity was significantly predicted from the diversity in personality traits and risk-taking attributes, F (10,13,465) = 374.869, p < 0.001, R2R2 = 0.218, a medium effect size. Specifically, the diversity in team openness and agreeableness in addition to teams’ ethical, financial, health/safety and recreational risk-taking promoted team empathy diversity while the remaining personality traits and risk-taking attitudes negatively predicted team empathy diversity, see Fig. 2 for a summary of the contributing predictors. These findings partially support our first hypothesis that the diversity in personality traits were related to team empathy [14, 15]. However, it pointed out that the diversity in some personality attributes negatively impact the empathy elevation and diversity in teams. Specifically, if highly empathic teams are desired, our results

516

M. A. Alzayed et al.

Fig. 2 Standardized beta coefficients of the statistically significant predictors from the two linear regression models displaying the relationship between the diversity of simulated teams' personality traits and risk-taking attributes and (1) team trait empathy elevation and (2) team trait empathy diversity

call for the importance of focusing diversity in team extraversion, neuroticism, and openness. Meanwhile, if empathy diverse teams are desired, our results call for the importance of focusing diversity in the openness and agreeableness personality traits. Similarly, these findings partially support our second hypothesis that the diversity in risk-taking attributes was related to team empathy [14, 15]. Specifically, if highly empathic teams are desired, our results call for the importance of focusing diversity in team ethical, health/safety, and social risk-taking. Meanwhile, if empathy diverse teams are desired, our results call for the importance of focusing diversity on ethical, financial, health/safety, and recreational risk-taking attitudes, but not social risk-taking. When composing highly empathic teams, our results point to the composition of teams that are diverse in extraversion, openness, and neuroticism in addition to being diverse in ethical, health/safety and social risk-taking. Meanwhile, if empathic diverse teams are sought, our results point to the composition of teams that are diverse in the openness and agreeableness personality traits as well as ethical, financial, health/safety, and recreational risk-taking attributes. However, these results call for future research that could validate these findings with a large sample of design teams in a human-subjects study. Taken as a whole, the results from this research can be used to guide empathic team formation. However, it warrants future research that would assess the role of other individual differences (e.g., gender [4]) with relation to team empathy.

Composing Diverse Design Teams: A Simulation-Based Investigation …

517

5 Conclusions, Limitations, and Future Work The main goal of the current study was to investigate the role of personality traits and risk-taking attitudes in impacting team empathy in engineering design. This was accomplished through a computational simulation of 13,482 teams of noninteracting individuals generated by a statistical bootstrapping technique drawing upon a data set from 103 first-year engineering design students. Taken as a whole, the results from this research corroborate previous research on individuals that found that personality traits and risk-taking attitudes predicted an individual’s empathy [5–8]. Specifically, the main findings from this exploratory study call for the importance of focusing diversity in team extraversion, openness, and neuroticism in addition to the diversity in teams’ ethical, health/safety and social risk-taking when composing highly empathic teams. Meanwhile, if empathy diverse teams are desired, the results call for the importance of focusing diversity in the openness and agreeableness personality traits as well as ethical, financial, health/ safety, and recreational risk-taking. However, there are several limitations that lead to exciting avenues for future research. First, while this study assessed the role of personality traits and risk-taking attitudes on team empathy, future research should investigate other individual differences, such as the gender diversity prevalent in a team. Second, simulated team members were not aware of their team membership due to the nature of the simulation study. Thus, this study does not account for social effects (e.g., social loafing [30]) and the roles of team members [40–43]. Thus, future research is warranted to situate these results in a human-subjects study. Additionally, this study presented an aggregated measure of trait empathy that takes the average of three (perspective-taking, fantasy, empathic concern) of the four IRI empathic tendencies. Thus, future research is needed to assess the role of individual differences with relation to each of the four IRI empathic tendencies. Finally, while this study explored the role of personality traits and risk-taking attitudes of first-year engineering students, future research is warranted to explore these relationships across other populations. Taken as a whole, this research is one of the first to study empathy at a team level.

References 1. McLaren K (2013) The art of empathy: a complete guide to life‘s most essential skill. Sounds True 2. Johnson DG, Genco N, Saunders MN, Williams P, Seepersad CC, Hölttä-Otto K (2014) An experimental investigation of the effectiveness of empathic experience design for innovative concept generation. J Mech Des 136(5):051009 3. Tang X (2018) From ‘empathic design’ to ‘empathic engineering’: toward a genealogy of empathy in engineering education 4. Davis MH (1980) A multidimensional approach to individual differences in empathy

518

M. A. Alzayed et al.

5. Costa P, Alves R, Neto I, Marvao P, Portela M, Costa MJ (2014) Associations between medical student empathy and personality: a multi-institutional study. PLoS ONE 9(3):e89254 6. Shu J, Hassell S, Weber J, Ochsner KN, Mobbs D (2017) The role of empathy in experiencing vicarious anxiety. J Exp Psychol Gen 146(8):1164 7. Ogawa A, Ueshima A, Inukai K, Kameda T (2018) Deciding for others as a neutral party recruits risk-neutral perspective-taking: model-based behavioral and fMRI experiments. Sci Rep 8(1):12857 8. Eysenck SB, Pearson PR, Easting G, Allsopp JF (1985) Age norms for impulsiveness, venturesomeness and empathy in adults. Personal Individ Differ 6(5):613–619 9. Toh CA, Miller SR (2016) Creativity in design teams: the influence of personality traits and risk attitudes on creative concept selection. Res Eng Des 27(1):73–89 10. Knight DW, Carlson LE, Sullivan JF Improving engineering student retention through hands-on, team based, first-year design projects. In: Proceedings of the international conference on research in engineering education, Honolulu, HI 11. Felder RM, Silverman LK (1988) Learning and teaching styles in engineering education. Eng Educ 78(7):674–681 12. Stempfle J, Badke-Schaub P (2002) Thinking in design teams-an analysis of team communication. Des Stud 23(5):473–496 13. Paulus PB, Larey TS, Dzindolet MT (2001) Creativity in groups and teams. In: Groups at work: theory and research, pp 319–338 14. Duncan GJ, Boisjoly J, Levy DM, Kremer M, Eccles J (2003) Empathy or antipathy? The consequences of racially and socially diverse peers on attitudes and behaviors 15. Roberge M-E (2013) A multi-level conceptualization of empathy to explain how diversity increases group performance. Int J Bus Manag 8(3):122 16. Carrell LJ (1997) Diversity in the communication curriculum: impact on student empathy. Commun Educ 46(4):234–244 17. Covey S (2013) Stephen R. Covey quotes. GoodReads.com 18. Neuman GA, Wagner SH, Christiansen ND (1999) The relationship between work-team personality composition and the job performance of teams. Group Org Manag 24(1):28–45 19. Alsager Alzayed M, Miller S, Menold J, Huff J, McComb C (2020) Can design teams be empathically creative? A simulation-based investigation on the role of team empathy on concept generation and selection. In: ASME 2020 international design engineering technical conferences and computers and information in engineering conference, St. Louis, Missouri 20. Watson P, Grisham SO, Trotter MV, Biderman MD (1984) Narcissism and empathy: validity evidence for the narcissistic personality inventory. J Pers Assess 48(3):301–305 21. Devlin SP, Flynn JR, Riggs SL (2018) Connecting the big five taxonomies: understanding how individual traits contribute to team adaptability under workload transitions. In: Proceedings of the human factors and ergonomics society annual meeting, Los Angeles, CA. SAGE Publications, Los Angeles, pp 119–123 22. Jensen D, Feland J, Bowe M, Self B. A 6 hats based team formation strategy: development and comparison with an MBTI based approach. In: Proceedings of the 2000 Annual Conference, pp 5.9.1–5.9.16 23. Wilde DJ (2008) Teamology: the construction and organization of effective teams. Springer Science & Business Media 24. Jensen D, Wood J, Wood K (2003) Hands-on activities, interactive multimedia and improved team dynamics for enhancing mechanical engineering curricula. Int J Eng Educ 19(6):874– 884 25. Jensen D, Wood K (2000) Incorporating learning styles to enhance mechanical engineering curricula by restructuring courses, increasing hands-on activities, & improving team dynamics. Proceedings of the ASME publication and presentation for the award for the most innovative curriculum for the year 2000, presented at the ASME annual conference

Composing Diverse Design Teams: A Simulation-Based Investigation …

519

26. Kim MS, Kim YS, Kim TH (2007) Analysis of team interaction and team creativity of student design teams based on personal creativity modes. In: Proceedings of the ASME 2007 international design engineering technical conferences and computers and information in engineering conference. American Society of Mechanical Engineers Digital Collection, pp 55–67 27. Norris RL, Siddique Z, Altan MC, Baldwin J, Merchan-Merchan W (2016) Cognitive empathy in design course for a more inclusive mechanical engineering. Proceedongs of the ASME 2016 international design engineering technical conferences and computers and information in engineering conference. American Society of Mechanical Engineers Digital Collection 28. Alsager Alzayed M, Miller S, McComb C Does empathy beget creativity? Investigating the role of trait empathy in idea generation and selection. Proceedings of the design computing and cognition 29. Van De Van A, Delbecq AL (1974) The effectiveness of nominal, Delphi, and interacting group decision making processes. Acad Manag J 17(4):605–621 30. Mullen B, Johnson C, Salas E (1991) Productivity loss in brainstorming groups: a meta-analytic integration. Basic Appl Soc Psychol 12(1):3–23 31. Surma-aho AO, Bjorklund Katja TA, Holtta-Otto K (2018) Assessing the development of empathy and innovation attitudes in a project-based engineering design course. In: 2018 ASEE annual conference & exposition 32. Hatcher SL, Nadeau MS, Walsh LK, Reynolds M (1994) The teaching of empathy for high school and college students: testing. Adolescence 29(116):116 33. Johnson J (2014) Measuring thirty facets of the five factor model with a 120-item public domain inventory: development of the IPIP-NEO-120. J Res Pers 51(1):78–89 34. Weber EU, Blais A-R, Betz NE (2002) A domain-specific risk-attitude scale: measuring risk perceptions and risk behaviors. J Behav Decis Mak 15(4):263–290 35. Alzayed MA, McComb C, Hunter ST, Miller SR (2019) Expanding the solution space in engineering design education: a simulation-based investigation of product dissection. J Mech Des 141(3):032001 36. McComb C, Cagan J, Kotovsky K (2017) Optimizing design teams based on problem properties: computational team simulations and an applied empirical test. J Mech Des 139 (4):041101 37. Mooney CZ, Duval RD, Duvall R (1993) Bootstrapping: a nonparametric approach to statistical inference. Sage, London 38. Wright DB (2007) Calculating nominal group statistics in collaboration studies. Behav Res Methods 39(3):460–470 39. Cohen J (1988) Statistical power analysis for the behavioral sciences. Psychology Press, Hove 40. Belbin RM (2012) Team roles at work. Routledge, London 41. Collopy AX, Li C, Liu T, Adar E, Papalambros PY (2019) Coordination in design teams as a lens to identifying team roles. Proceedings of the design society: international conference on engineering design. Cambridge University Press, Cambridge, pp 79–88 42. Fisher S, Hunter T, Macrosson W (1998) The structure of Belbin’s team roles. J Occup Organ Psychol 71(3):283–288 43. Wilde DJ (1999) Design team roles. In: Proceedings of the 1999 ASME design engineering technical conferences

Emergence of Engineering Design Self-Efficacy in Engineering Identity Development John Jongho Park, Elizabeth Starkey, Nathan Hyungsok Choe, and Christopher McComb

Abstract In engineering education, the goal is not only to teach the technical engineering skills but to positively influence the formation of students’ engineering identity to enhance future success. Often, first-year engineering classes that have a large formative role in students’ academic careers are taught using a project-based learning approach. However, the effect of such courses on the growth and development of students’ engineering identity, and in turn their design cognition, has not been studied in detail. This work studied the growth of engineering identity of 15 weeks in one such course. Engineering identity was assessed through several engineering identity instruments as well as the use of natural language processing.

1 Introduction In engineering education, the goal is not only to teach the technical engineering skills but to positively influence students’ identity development to enhance future success. Project based learning has been adopted as a new way to better prepare students for the workforce [1] by giving them practical experiences and encouraging systems thinking, two areas where industry has indicated weaknesses in entry level graduates [2]. Several researchers have claimed that learning is associated with development of one’s self-efficacy, or confidence in their ability to accomplish a task [3]. In engineering design, researchers believe that self-efficacy may be more important due to the project-based nature of most design courses [4]. In engineering design courses, project-based learning often starts with an open-ended design prompt and students work through a series of design steps to solve the problem J. J. Park
E. Starkey The Pennsylvania State University, University Park, PA, USA N. H. Choe The George Washington University, Washington, DC, USA C. McComb (&) Carnegie Mellon University, Pittsburgh, PA, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. S. Gero (ed.), Design Computing and Cognition'20, https://doi.org/10.1007/978-3-030-90625-2_31

521

522

J. J. Park et al.

using an engineering design process [5]. While there are many different options for which design process to follow, these steps typically include empathizing with the user; defining the problem and developing customer needs; ideation and concept selection; prototyping; and testing. These steps are often followed by iteration; as students are encouraged to “fail early, fail often” and to learn from their failures [6]. While some studies have investigated the impact of project-based learning on metrics of self-efficacy and identity, they have not investigated engineering design self-efficacy and engineering identity over the course of a first-year project-based engineering design course. Identity is in many ways inextricable from design cognition, and therefore a deeper understanding of the former will contribute to the latter. With that in mind, our goal in this work is to investigate the evolution of engineering design self-efficacy and engineering identity over the course of a semester long engineering design course. In addition to these measures, we investigate the responses to several questions related to engineering identity, asked to participants at the beginning and end of the semester.

2 Related Work In order to understand the connection between engineering design self-efficacy and engineering identity in engineering design, relevant literature was reviewed and is discussed in the following sections.

2.1

Engineering Design Self-Efficacy

In order to be successful, one does not only need self-confidence, but also a goal. The idea of self-confidence with a goal is embodied by measures of perceived self-efficacy, or one’s belief in their ability to complete a task or achieve a goal [3, 7]. The idea of self-efficacy has been studied in many domains, and 4 main categories that build into the construct of self-efficacy have been identified as: performance/personal mastery, observation of other’s success, verbal persuasion, and physiological arousal (judgements of anxiety/vulnerability to stress) [8]. While all of these categories play a role in the construct, performance/personal mastery may have the greatest impact on increasing self-efficacy [9]. Self-efficacy has been studied in many domains for its connection with future success [10]. High levels of self-efficacy have been linked to high persistence and willingness to expend more effort for activities. Lastly, Healey and Hays reported that self-efficacy development was highly associated with individuals’ professional identity development [11]. The Engineering Design Self-Efficacy (EDSE) instrument developed by Carberry et al. [4] has been investigated and validated and will be used as the theoretical framework for this paper.

Emergence of Engineering Design Self-Efficacy in Engineering …

2.2

523

Undergraduate Students’ Engineering Identity Development

The broad definition of the term identity includes individuals’ concept about the self, and has been adopted by various disciplines [12, 13]. The construct of identity has come to represent both personal perceptions of the self and internalized messages about the self that originate in one’s environment [14]. Particularly in regard to this project, identity not only includes beliefs and perceptions of the self in terms of proficiencies and skills but is also informed by technical skills development and environment. This occurs at intra-individual level of identity (e.g., gender, ethnicity) but also at more inter-individual levels associated with professional discipline (engineering design) and training. Eliot and Turns [15] defined engineering identity as a professional identity that associated with engineering practices and it develops by acquiring engineering knowledge, skills, and experiences that are amalgamated around an engineering activities or profession. Engineering identity has been developed by incorporating several traditional identity theories [16–18]. First, those traditional identity theories influenced the development of a STEM identity in undergraduates, which then progressed into the engineering identity. Traditional identity theories and STEM identity frameworks provide the solid groundwork to develop a framework of engineer-identity formation of students that highlights their competence, research experience, and professional socialization. In addition, STEM identity literature highlights the importance of performance of and competence in discipline-related activities. Serving as a theoretical framework for this study, we used an undergraduate engineering identity measures that Choe and Borrego [19] developed. They identified factors that contribute to engineering identity development are engineering interest, engineering competence, engineering recognition from others, and interpersonal skills competence. These factors were modified from Carlone and Johnson’s [20] science identity framework that emphasize the importance of social recognition and collaboration skills identity [21].

2.3

Research Rationale

Based on an analysis of the existing literature, it becomes clear that engineering identity is a professional identity influenced by technical self-efficacies. Moreover, identity contributes to individual enactment of design cognition. In consideration of the significance of acquiring self-efficacy, which impacts one’s professional identity, investigating the process of acquisition of engineering design skills would help to elucidate the mechanisms of students’ engineering identity development.

524

J. J. Park et al.

3 Method This study was conducted with 88 first year engineering students. The methodological approach for this study is addressed below.

3.1

Educational Context

The introductory engineering design course used in this study is a 16-week long project-based course that meets for *6 h a week. The course has a mixture of lecture and lab time, with short lectures followed by activities that support larger design projects. Throughout the 16-week course, two large design projects (*8 weeks each) are completed. The first design project is selected by the instructor and varies in topic, while the second project is client driven and completed by many sections of the course across multiple instructors. The course focuses on introducing students to an engineering design process and also practicing that design process. Learning a design process is supported by developing communication skills, hands on “making” experiences, and encouraging systems thinking [5].

3.2

Participants

In the first week of the class, 88 students participated the survey as a pre-test. In the 15th week of the class, 78 students in the same class participated the survey as a post-test. Of the participants who participated pre-test, and therefore completed the demographics survey, 23 identified as women, 64 as men, and one preferred to not answer, with 99% in the 18–21 age range. Of the sample, 82 students were engineering students, and six others were non-engineering majors. Racial/ethnic groups represented included 65% European-American/White, 21% Asian-American/ Pacific Islander, 4% Latino/Chicano/Hispanic, 4% African-American/Black, 2% Middle Eastern, with 3% representing other categories or non-responders.

3.3

Procedure and Measures

Quantitative data was collected via a survey administered during the engineering design class using Qualtrics, an online survey site. Before procedures began, an overview of the study was provided, and consent was obtained. At the end of the first week of the semester, students were given a survey to complete containing demographic questions, an engineering design self-efficacy (EDSE) instrument, an

Emergence of Engineering Design Self-Efficacy in Engineering …

525

affinity toward elements of engineering practice instrument and an engineering identity instruments. The 9-item EDSE instrument developed by Carberry et al. [4] was used and measured on a 10-point likert scale. The 30-item instrument of affinity toward elements of engineering professional practice were used and measured on a 5-point likert scale. These affinity toward elements of engineering professional practice items were considered important measures because undergraduate students’ engineering identities were higher for those who had higher affinity elements in 6 different categories: framing and solving problems; design; project management; analysis; collaboration; and tinkering [21]. To measure engineering identity, a single item instrument with an 8-option Venn diagram response was used [22]. The students were also asked to answer 5 open ended questions with a 100-word minimum, as a means of assessed their engineering identity in a more organic way than the direct instruments. These questions were: (1) What is your definition of engineering? (2) Describe yourself as an engineer. (3) What is it about engineering that makes you want to become an engineer? (4) Can you pinpoint one event that made you want to become an engineer? and (5) Is there anyone in your life that you think of as a mentor in engineering/STEM? The survey was administered in-class electronically using Qualtrics. The same survey questions were asked both the first week (pre-test) and 15th week (post-test) of a semester long-class.

3.4

Data Cleaning and Analysis

To measure reliability, Cronbach alpha tests were conducted for all affinity elements. All Cronbach alpha values ranged from 0.73 to 0.86, which is acceptable range [21]. We conducted t-tests and bivariate correlation to answer our research questions. For the t-tests, Levene’s tests were conducted to confirm the variances in the pre-test and the post-test are similar. Based on the result of the Levene’s test, we ran t-tests with either equal or unequal variance. Due to large number of t-tests are performed, we set the p-value below 0.01 to use a more conservative threshold. For the correlation, we also set the p-value to below 0.01.

3.5

Latent Semantic Analysis

The open-ended responses to the five questions were assessed using Latent Semantic Analysis (LSA) [23]. This method embeds each response in a Euclidean latent semantic space which makes it possible to measure distances between these documents. First, a common dictionary of the words present across the set of responses was constructed, and common words that were likely to have little meaning were removed. Next, each response was represented as a term-frequency vector, in which elements of the vector contained the frequency with which each word occurred in the document. Together, the term-frequency vectors were used to

526

J. J. Park et al.

construct a term-document matrix, and Singular Value Decomposition [24] was applied to the matrix to decrease the dimensionality of the vectors. The reduced-dimensional vector representations exist in a semantic latent space, which facilitates easy comparisons between the vectors using Euclidean distance. This technique has been used extensively in the engineering design community, including the assessment of team communications [25], story-telling [26], and divergent-convergent patterns of solution search [27].

4 Results Before presenting results pertaining to each research question, the latent semantic space visualization is provided in order to present context for subsequent discussion.

4.1

Latent Semantic Space Visualization

Figure 1 provides representations of the latent semantic space discovered through the application of LSA. These plots were specifically generated using the t-distributed Stochastic Neighbor Embedding (t-SNE) algorithm [28]. This is similar to methodology used in other work to study student responses [29]. The left image shows the distribution of question responses at the beginning of the semester, and the right image provides the distribution at the end of the semester. There is not any significant change in the structure of the latent space between these two spaces, indicating that students tended to provide answer that drew upon the same general concepts. However, questions 1 and 5 are particularly clearly differentiated from the other data in both plots, indicating that their answers

Fig. 1 The distribution of responses in the latent semantic space, both for semester-beginning (left) and semester-end (right)

Emergence of Engineering Design Self-Efficacy in Engineering …

527

tended to be semantically unique. Meanwhile, questions 2, 3, and 4 tended to have a greater degree of overlap, indicating semantic similarity. These results show that LSA may be useful for distinguishing between different components of engineering identity (i.e., those corresponding to each question) but may not be useful for determining growth or change in identity over time.

4.2

RQ1: How do the Pre-test Scores of Design Self-Efficacy Differ from Post-test Scores?

Our first research question aims at to understand if and how engineering design self-efficacy differs from pre to post engineering design class In answering our first research question, Table 1 shows that all nine design self-efficacy items were significantly different between pre-test and post-test, with all increasing from pre to post.

4.3

RQ2: How do the Pre-test Scores of Engineering Identity and Professional Practice Factors of Engineering Identity Differ from Post-test Scores?

Our second research question aims to understand if and how engineering identity and professional factors of engineering identity differ pre- to post-engineering design class. Results for aggregate categories of the 30-item engineering identity instrument are shown in Table 2. In comparing the impact that taking engineering design class, we found that among affinity elements, students’ perceived competencies of framing and solving problems and design were increased at the post-test compared to pre-test. On the other hand, scores of the other four factors, project

Table 1 Comparison of mean responses of design self-efficacy from pre-test to post test Construct/variables Conduct engineering design Identify a design need Research a design need Develop design solutions Select the best possible design Construct a prototype Evaluate and test a design Communicate a design Redesign Note * p < 0.01; All items are 10-point

Pre

Post

p-value

7.24 (2.24) 7.53 (2.26) 7.85 (2.09) 7.82 (2.02) 8.02 (1.92) 7.64 (2.32) 7.98 (1.77) 8.13 (2.13) 7.93 (2.27) scale; n = 55

9.35 (1.46) 9.67 (1.25) 9.24 (1.82) 9.33 (1.41) 9.58 (1.44) 8.82 (2.03) 9.29 (1.92) 9.8 (1.31) 9.16 (1.78)

0.0000* 0.0000* 0.0000* 0.0000* 0.0000* 0.0001* 0.0000* 0.0000* 0.0002*

528

J. J. Park et al.

Table 2 Comparison of mean responses of engineering identity and its professional factors from pre-test and post-test

Construct/variables

Pre

Post

4.13 4.30 (0.64) (0.67) 3.71 3.95 (0.61) (0.70) Project managementa 4.11 4.20 (0.72) (0.77) 4.12 4.28 Analysisa (0.70) (0.74) 3.96 4.07 Collaborationa (0.70) (0.67) a 4.02 4.07 Tinkering (0.73) (0.71) 5.95 4.78 Engineering identityb (1.33) (1.24) Notes *p < 0.0; n = 66; pre/post are Mean (SD) a Items on a 10-point scale b Items on an 8-point scale Framing/solving problemsa Designa

p-value 0.0002* 0.0004* 0.2317 0.0245 0.0586 0.3971 0.0000*

management, analysis, collaboration, and tinkering, were not significantly different from post-test to pre-test. In addition, student engineering identity was significantly decreased from pre-test to post-test.

4.4

RQ3: What are the Association Between Engineering Identity and Affinity Toward to Elements of Engineering Practices at the Post-test?

Table 3 shows results relevant to our third research question. Specifically, affinity toward elements of engineering practices factors was positively and significantly correlated to the other factors. For the engineering identity, three influences on elements of engineering practices factors were significant. Design, Collaboration, and Tinkering were positively correlated with engineering identity.

Emergence of Engineering Design Self-Efficacy in Engineering …

529

Table 3 Correlation between engineering identity and affect toward to elements of engineering practices (Post-test)

5 Discussion This study was conducted to understand the impact of an engineering design course on engineering self-efficacy and engineering identity and their interaction. Our main findings were: (1) engineering design self-efficacy increased over the course of the semester; (2) the framing and solving problems and design factors of engineering identity increased over the course of the semester; and (3) the direct measure of engineering identity decreased over the course of the semester. These results and their implications for engineering design are discussed in the following subsections. Our first research question investigated if engineering design self-efficacy changed from pre-to post-engineering design course and our results showed that there were significant increases in all factors of EDSE. These results indicate that the class served its purposes, since design self-efficacy in each category was above a 9 (out of 10) on average in the post-test assessment (Table 2). While all factors were high, some were exceptionally high, with the “Communicate a design” item scoring 9.8 out of 10 points on average. Since one of the main objectives of this course is design communication, increases in this area indicate that the objectives of the course are being met. Throughout this course, students practice their communication skills through written, oral, and graphical communication where they are given feedback from their peers and their instructors. These opportunities to communicate and gain feedback can provide students with opportunities for personal mastery and verbal persuasion as well as watching the success of their classmates to help them grow their self-efficacy. On the other hand, the “construct a prototype” item has the lowest mean among the design self-efficacy items in the scale. Since making and “developing a maker mindset” are also objectives of this course, these results are unexpected. Perhaps, some of the design components are more challenge than others for engineering students to master. While prototyping and making are emphasized, there are many types of prototyping and these skills might not be as easy to master over the course of the semester.

530

J. J. Park et al.

Although EDSE increased from the beginning to end of the semester, we do not know how this changed at other times during the semester. Future work should record EDSE at more points during the semester in order to understand how individual components of the course are impacting these measures and pinpoint areas to increase prototyping self-efficacy. Next, research question investigated if engineering identity or factors of engineering identity changed from pre-to post-engineering design course and our results showed that there were significant increases in the framing and solving problems and design factors while the direct measure of engineering identity decreased. These results indicate that students’ understanding of engineering identity may have changed after the first semester compared to the beginning of the semester. While the direct measure of engineering identity decreased, they are seeing increases in two areas that influence engineering identity: framing and solving problems and design. Our last, research question was what the associations between engineering identity are and affinity toward elements of engineering practices at the post-test. The results indicated that students’ affinity elements factors for, design, collaboration, and tinkering were positively correlated with engineering identity. It shows that positive experience of designing, collaboration, and tinkering may contribute to developing students’ engineering identity.

6 Conclusions and Future Work The model shown in Fig. 2 is presented as a summary of results on engineering identity development that are associated with and various engineering design competencies (collaboration, design, and tinkering).

Fig. 2 Engineering identity development in design classes

Emergence of Engineering Design Self-Efficacy in Engineering …

531

In future work, other potential influencing factors of engineering identity development should be investigated as arrows of the figure show (engineering skills, professional skills, other identities, engineering interest). These investigations may show more detailed progression of students’ engineering identity development by identifying positive influences and hindrances. This follow-up study can provide verification of certain factors of engineering identity. In addition, it should be noted that the participants of this study are mostly first year engineering students. Therefore, if senior students are investigated the above model will potentially change. There is both intellectual and theoretical merit to comparing and contrasting engineering identity between first year students and senior students within the context of engineering design.

References 1. Duderstadt JJ (2010) Engineering for a changing world. In: Holistic engineering education. Springer, New York, pp 17–35 2. Giges NS (2014) Changes afoot in engineering education. ASME.org. https://www.asme.org/ topics-resources/content/changes-afoot-in-engineering-education 3. Bandura A (1982) Self-efficacy mechanism in human agency. Am Psychol 37(2):122–147. https://doi.org/10.1037/0003-066X.37.2.122 4. Carberry AR, Lee H-S, Ohland MW (2010) Measuring engineering design self-efficacy. J Eng Educ 99(1):71–79 5. Ritter S, Bilen S (2019) EDSGN 100: a first-year cornerstone engineering design course. In: 2019 FYEE conference, pp 1–7 6. Thomke S, Reinertsen D (2012) Six myths of product development. Harv Bus Rev 90:84–94 7. Bandura A (1977) Self-efficacy: toward a unifying theory of behavioral change. Psychol Rev 84:191–215 8. Bandura A, Adams NE (1977) Analysis of self-efficacy theory of behavioral change. Cogn Ther Res 1(4):287–310 9. Bandura A, Adams NE, Beyer J (1977) Cognitive processes mediating behavioral change. J Pers Soc Psychol 35:125–139 10. Pajares F (1996) Self-efficacy beliefs in academic settings. Rev Educ Res 66(4):543–578 11. Healey A, Hays D (2012) A discriminant analysis of gender and counselor professional identity development. J Couns Dev 90:55–62 12. Tonso KL (2006) Student engineers and engineer identity: campus engineer identities as figured world. Cult Stud Sci Educ 1:273–307 13. Beauchamp C, Thomas L (2009) Understanding teacher identity: an overview of issues in the literature and implications for teacher education. Camb J Educ 39:175–189 14. Park JJ, Schallert DL (2019) Talking, reading, and writing like an educational psychologist: the role of discourse practices in graduate students’ professional identity development. Learn Cult Soc Interact 22:1–11 15. Eliot M, Turns J (2011) Constructing professional portfolios: sense-making and professional identity development for engineering undergraduates. J Eng Educ 100:630–654 16. Gee JP (2000) Identity as an analytic lens for research in education. Rev Res Educ 25:99–125 17. Erik H (1994) Identity and the life cycle. WW Norton & Company 18. Wenger E (1999) Communities of practice: Learning, meaning, and identity. Cambridge University Press

532

J. J. Park et al.

19. Choe NH, Borrego M (2019) Prediction of engineering identity in engineering graduate students. IEEE Trans Educ 62:182–187 20. Carlone HB, Johnson A (2007) Understanding the science experiences of successful women of color: science identity as an analytic lens. J Res Sci Teach 44:1187–1218 21. Choe NH., Martins LL, Borrego M, Kendall MR (2019) Professional aspects of engineering: improving prediction of undergraduates’ engineering identity. J Prof Issues Eng Educ Pract 145:04019006-1–04019006-8 22. Borrego M, Patrick A, Martins L, Kendall M (2018) A new scale for measuring engineering identity in undergraduates. In: ASEE annual conference and exposition, conference proceedings, pp 1–3 23. Deerwester S, Dumais ST, Furnas GW, Landauer TK, Harshman R (1990) Indexing by latent semantic analysis. J Am Soc Inf Sci 41:391–407 24. Golub GH, Reinsch C (1970) Singular value decomposition and least squares solutions. Numer Math 14:403–420 25. Dong A, Hill AW, Agogino AM (2004) A document analysis method for characterizing design team performance. J Mech Des 126(3):378–385 26. Song S, Dong A, Agogino AM (2003) Time variation of design ‘story telling’ in engineering design teams. In: Proceedings of the international conference on engineering design, ICED, pp 1–10 27. Fu K, Cagan J, Kotovsky K (2010) Design team convergence: the influence of example solution quality. J Mech Des 132(11):111005 28. van der Maaten L, Hinton G (2012) Visualizing non-metric similarities in multiple maps. Mach Learn 87(1):33–55 29. McComb C, Tehrani FM (2014) Research and practice group methodology: a case study in student success. In: 2014 ASEE pacific-southwest conference, pp 1–15

Design Neurocognition and Physiology

Designing-Related Neural Processes: Higher Alpha, Theta and Beta Bands’ Key Roles in Distinguishing Designing from Problem-Solving S. Vieira, J. S. Gero, V. Gattol, J. Delmoral, S. Li, G. Cascini, and A. Fernandes Abstract This paper presents results from an experiment studying differences between designing and problem-solving in professional industrial designers, using EEG to measure neurophysiological activations. We compare neurophysiological activation and frequency bands power between three prototypical tasks, a problem-solving layout design task, an open layout design task and an open design sketching task. The study draws on the neurophysiological results from 18 experiment sessions with professional designers. Results indicate significant differences in activations between the problem-solving task and the open design tasks, in terms of aggregate, temporal and frequency bands power across participants. Higher alpha, theta and beta frequency band values play a key role in the open design sketching task when compared to the layout tasks.

1 Introduction Creativity related neural processes have been widely investigated [1], however, designing related neural processes are still in the early stages of study. Distinguishing designing from creativity, specifically through measuring the brain activation during designing and when being creative can have an impact in the unfolding of the neuroscience of designing and creativity research. With this ultimate aim in mind, we report research in this paper that aims to distinguish neurophysiological activation of designers while designing from problem-solving. S. Vieira (&)
S. Li
G. Cascini Politecnico di Milano, Milan, Italy e-mail: [email protected] J. S. Gero UNCC Charlotte, Charlotte, NC, USA V. Gattol AIT, Vienna, Austria J. Delmoral
A. Fernandes INEGI-FEUP, Porto, Portugal © The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 J. S. Gero (ed.), Design Computing and Cognition'20, https://doi.org/10.1007/978-3-030-90625-2_32

535

536

S. Vieira et al.

Problem-solving is distinguished from designing by the fixedness of the problem and solution spaces. Problem-solving can also occur as an activity within designing. The commensurability of measurements of neurophysiological studies in design research makes for a robust approach for studying brain activity during designing [2]. Design studies based on functional magnetic resonance imaging (fMRI) started a decade ago [3] with a controlled experiment reporting preliminary results on the distinction between designing and problem-solving. The present experiment extends this study. However, through the use of the electroencephalography technique (EEG) we aim at results based on frequency bands of brain waves to distinguish designing from problem-solving tasks. Studies using EEG commenced more than 40 years ago [4] investigating cortical activation during multiple tasks. Some 20 years later a study on categorization tasks of experts and novices [5] made use of EEG in design research. In the last 10 years, single domain-related EEG design studies [6–9], functional near-infrared spectroscopy (fNIRS) design studies [10] and fMRI studies focused on sustainability judgments [11], design ideation and inspirational stimuli [12] of mechanical engineers, graphic designers [13] and architects [14] have been used to understand the acts of designing from a neurophysiological perspective. As designing is a temporal activity EEG has started to play a role in design research because of its high temporal resolution, readily available software, reduction in the cost of portable equipment and relatively little need for specialized training.

1.1

EEG Studies

Electroencephalography records electrical brain activity with electrodes placed along the scalp. Neurons transmit signals down the axon and the dendrites via an electrical impulse. EEG activity reflects the summation of the synchronous activity of thousands or millions of neurons: by having similar spatial orientation, their ions line up and create waves to be detected. Pyramidal neurons of the cortex are thought to produce the most EEG signal because they are well-aligned and fire together [15, 16]. EEG measures electromagnetic fields generated by this neural activity. Activity from deep sources of the brain is more difficult to detect than currents near the skull, thus EEG is more sensitive to cortical activity [16, 17]. Despite EEG’s limited spatial resolution, it offers high temporal resolution in the order of milliseconds in a portable device which makes it a highly suitable tool to investigate designing as a temporal activity. The present study uses a low-cost EEG device, which has some limitations when compared to medical grade systems. These limitations of stability and robustness, low number of channels, lower signal to noise ratio, lower sampling rate and multiplexed measurement of electrodes are well within what we are measuring: the unfolding neurophysiological activations during tasks of the experiment session. A design session of 45 min generates between 600 and 1,200 segments using

Designing-Related Neural Processes …

537

protocol analysis, where each segment codes a cognitive action. This results in an average between 2.25 and 4.5 s per action. The sampling rate of the low-cost EEG device used in this study is higher than the cognitive activation rate found in previous cognitive studies (Gero and McNeil, 1998, Goldschmidt 2014). This is well within the capacity of a low-cost EEG device. Although the low-cost EEG equipment have lower signal to noise ratio potentially resulting in a higher variability of the results and higher standard deviation, the statistical approach we describe, compensates for these effects. Low-cost, noninvasive portable EEG equipment becomes a viable tool for the level of resolution we are interested in and for achieving preliminary results and tentative evidence that can support more advanced research [18, 19]. In design research, frequency bands results derived from EEG signals have been widely used as a measurement tool. They have been used to compare visual thinking spent during solution generation with solution evaluation of a layout task [9]. EEG frequency bands have been associated with design activities. In particular, frequency bands beta 2, gamma 1 and gamma 2 [8], higher alpha power have been found to be associated with open ended tasks and divergent thinking, while theta and beta power have been found to be associated to convergent thinking in decision-making and constraints tasks [7]. Higher beta power has been associated also with visual attention and higher alpha power with visual association in expert designers.

2 Aim This paper describes a study that forms part of a larger research project whose goal is to investigate neurophysiological activation of designers across multiple design domains [20]. The study reported in this paper is focused on the frequency bands analysis of industrial designers’ neurophysiological activations using an EEG headset in the context of performing problem-solving and design tasks in a laboratory setting. The aim of the study is to: • investigate the neurophysiological activation differences and frequency bands of industrial designers when performing designing and problem-solving tasks. We explore the differences between a constrained layout design task based on problem-solving and two open design tasks, one based on open layout design and the other an open design sketching. The analysis focuses on the neurophysiological activation differences observed and their frequency bands, along the execution of the tasks, distinguishing activations between brain regions. We investigate the following research questions: • What are the differences in the neurophysiological activations of industrial designers’ frequency bands when problem-solving and designing?

538

S. Vieira et al.

• What brain regions show differences in the neurophysiological activations of the frequency bands of industrial designers when problem-solving and designing? • What are the differences in the neurophysiological activations of industrial designers’ frequency bands and brain regions when addressing an open design task?

3 Methods The research questions are examined using the problem-solving task as the reference and statistically comparing the open design tasks with the reference task. We compare absolute values known as transformed power (POW), for total signal and frequency bands, and task-related power (TRP), where the problem-solving neurophysiological activation of the highly constrained layout design task is the base for comparison. The tasks and experimental procedure were piloted prior to the full study [20].

3.1

Participants

The participants comprised 29 industrial designers, results from eleven of them were not used due to a variety of measurement problems. This study was approved by the local ethics committee of the University of Porto. Results are based on 18 right-handed participants, aged 25–43 (M = 31.7, SD = 7.3), 10 men (age M = 35.1, SD = 7.2) and 8 women (age M = 27.5, SD = 5.1). The participants are all professionals (experience M = 7.8, SD = 5.6). The analysis of covariance (ANCOVA) indicates no significant effect of the EEG scores after controlling for experience, F(1, 16) = 2.3, p = 0.15.

3.2

Experiment Tasks Design

The experiment consisting of a sequence of 4 tasks was previously described in [20]. For this paper the focus is on Task 1, Task 3 and Task 4, and their neurophysiological activations of total and bands frequency, Table 1 and Fig. 1. The study is part of a larger experiment and Task 2 is not used in this paper. Task 1 is considered a problem-solving task as the problem itself is well-defined and highly constrained, and the set of equivalent solutions is unique [3]. By adding an open layout design task to the previous experiment, we produced a block experiment in order to determine whether the open layout design task produces different results. The open layout design Task 3 has no predetermined final state and the task is open-ended.

Designing-Related Neural Processes …

539

Table 1 Description of the problem-solving, basic design and open design tasks Task 1 Problem-solving

Task 3 Open layout design

Task 4 Open sketching design

In Task 1 the design of a set of furniture is available and three conditions are given as requirements. The task consists of placing the magnetic pieces inside a given area of a room with a door, a window and a balcony

In Task 3 the same design available is complemented with a second board of movable pieces that comprise all the fixed elements of the previous tasks, namely, the walls, the door, the window and the balcony. The participant is told to arrange a space

In the free-hand sketching Task 4, the participants are asked to: propose and represent an outline design for a future personal entertainment system

Fig. 1 Depiction of the problem-solving Task 1, open layout design Task 3 and open free hand sketching design Task 4

We added a fourth open design task that uses free-hand sketching after Task 3. Task 4 is an ill-defined and fully unconstrained task unrelated to formal problem-solving. Each participant was given two sheets of paper (A3 size) and three drawing tools, a pencil, graphite and a pen. The open design tasks (Tasks 3 and 4) although of a different nature both require defining the problem and the solution spaces. We expect that a neurophysiological subtraction between the tasks will reveal brain electrical fields more strongly involved in designing. The complete tasks sequence is described in Vieira et al. [20].

3.3

Setup and Procedure

A tangible interface for individual task performance was built based on magnetic material for easy handling, Fig. 3. Differently from the original tasks [3], the magnetic pieces were placed at the top of the vertical magnetic board to prevent extraneous signal noise due to eye and head horizontal movements. A pre-task was

540

S. Vieira et al.

designed so that participants could familiarize themselves with the use of the EEG headset and with maneuvering the magnetic pieces that make up the physical interface [20]. One researcher was present in each experiment session to instruct the participant and to check for recording issues. A period of 10 min for setting up and a few minutes for a short introduction are necessary for informing each participant, reading and signing of the consent agreement and set the room temperature. A complete description of the procedure is reported in [20]. The researcher followed a script to conduct the experiment so that each participant was presented with the same information and stimuli. The participants were asked to stay silent during the tasks and use the breaks for talking and clarifying doubts. If needed, extra time was given to the participants, in particular in Tasks 3 and 4, so they could find a satisfactory solution.

3.4

Data Collection Methods

EEG activity was recorded using a portable 14-channel system Emotiv Epoc+. Electrodes are arranged according to the 10–10 I.S, Fig. 2. Electromagnetic interference of the room was checked for frequencies below 60 Hz. Each of the Emotiv Epoc+ channel collects continuous signals of electrical activity at their location. The participants performed the tasks on the physical magnetic board, with two video cameras capturing the participant’s face and activity and an audio recorder. All the data captures were streamed using Panopto software (https://www.panopto. com/), Fig. 3. The design sessions took place at the University of Porto, between March and July of 2017, and June and September of 2018, and in the Mouraria Creative Hub,

Fig. 2 Emotiv Epoc+ electrodes arrangement (10–10 I.S.) and experiment setup using the headset

Designing-Related Neural Processes …

541

Fig. 3 Screen capture depicting audio, video and screen captures streaming in Panopto

Lisbon, during August 2018 in rooms with the necessary conditions for the experiment, such as natural lighting sufficient for performing experiments between 9:00 and 15:00 and no electromagnetic interference.

3.5

Data Processing Methods

For completeness we repeat the data processing methods presented in [20]. The fourteen electrodes were disposed with 256 Hz sampling rate, low cutoff 0.1 Hz, high cutoff 50 Hz. We adopted the blind source separation (BSS) technique based on canonical correlation analysis (CCA) for the removal of muscle artifacts from EEG recordings [22, 23] adapted to remove the short EMG bursts due to articulation of spoken language, attenuating the muscle artifacts contamination on the EEG recordings [24]. The BSS-CCA algorithm, by using correlation as a criterion to measure independence of signals, takes into account temporal correlation. By establishing an ordering system of the separated singular valued components of the signal, the outputted components are sorted so that the highest correlated sources represent EEG sources and the lowest correlated sources represent noise. By systematically eliminating a subset of the bottom sources, the EEG signals from all subjects used in this study were cleaned. Thus, data processing includes the removal of Emotiv specific DC offset with the Infinite Impulse Response (IIR) filter and BSS-CCA. Data analysis included total and band power values on individual and aggregate levels using MatLab and

542

S. Vieira et al.

EEGLab open source software. All the EEG segments of the recorded data were used for averaging throughout the entire tasks. Results testing the BSS-CCA procedure’s efficacy on EEG signals from participants performing sketching tasks in three conditions including pen and paper confirm its efficacy, as exemplified in Fig. 4. The statistical approach we describe, compensates for the potential effects on the measurements due to the limitations of the equipment. A more detailed description is reported in [25]. The motor actions involved in the tasks using the tangible interface and the free hand sketching and their corresponding EEG signals are of the same source, thus we claim that the BSS-CCA procedure filters the signal of both artifacts. This leaves us signals associated with the design activity. Each of the Emotiv Epoc+ channel’s continuous electrical signal was processed to produce multiple measures. Here we report on three measures: task-related power (TRP), total signal transformed power (Pow) and frequency bands transformed power (Pow). The TRP is the task-related power, typically calculated taking the resting state as the reference period per individual [16, 24, 26]. This method, normally time-locked to fractions of seconds, is used across sessions that take minutes to allow for the design activity to unfold. We consider our experiment locked for the complete unfolding of the cognitive activities involved in each task. We take the problem-solving Task 1 as the reference for the TRP. The Pow is the transformed power, more specifically the mean of the squared values of microvolts per second (µV/s) for each electrode processed signal per task. This measure tells us about the amplitude of the signal per channel and per participant magnified to absolute values. We present Pow values on aggregates of participants’ individual results, per total task and for each task deciles for the temporal analysis of each task.

Fig. 4 Exemplary results showing the removal of the artifacts (raw signal dotted lines) through the BSS-CCA algorithm during sketching Task 4

Designing-Related Neural Processes …

543

Of the 29 sessions recorded, 5 had recording problems ranging from failing to switch on all the equipment, equipment failures, failure to complete the experiment and participants’ handedness. We processed the Pow and the TRP measurements for each participant per total task and temporal deciles. After a z-score was conducted in the analysis of Pow across tasks per participant data to determine outliers, the criteria for excluding participants were based on the evidence of 6 or more threshold z-score values above 1.96 or below −1.96 and individual measurements above 2.81 or below −2.81. After determining the outliers, we calculated the mean and standard deviation of each measure for each cohort. After the division of the Pow into time deciles (which provides the basis for the analysis of temporal stages) amplitudes leading to two and a half standard deviations from the mean as threshold values were excluded per channel. In this process a further 11 experiments were excluded leaving 18. The same procedure was applied to the frequency bands. This approach involves the decomposition of the EEG signal into component frequency bands. We used the typical frequency bands and their approximate spectral boundaries, delta (0.1–4 Hz), theta (4–7 Hz), alpha 1 (7–10 Hz), alpha 2 (10– 13 Hz), beta 1 (13–16 Hz), beta 2 (16–20 Hz) and beta 3 (20–28 Hz). By the adoption of lower and upper alpha boundaries, and the beta waves, we expect to find results that can be related to the literature in other domains.

3.6

Data Analysis Methods

We take the problem-solving Task 1 as the control/reference period for the task-related power (TRP) calculations. Thus, for each electrode, the following formula was applied taking the log of the Pow of the corresponding electrode i, in Task 1 as the reference period. By subtracting the log-transformed power of the reference period (Powi, reference) from the activation period (Powi, activation) for each trial j (each one of the 3 tasks per participant), according to the formula: TRPi ¼ logðPowi ; activationÞj  logðPowi ; referenceÞj By doing this, negative values indicate a decrease of task-related power from the reference (problem-solving Task 1) for the activation period, while positive values indicate a power increase [28] (power and activation refer to brain wave amplitude). Statistical Analyses We performed standard analyses based on the design of the experiment: always a repeated-measures design with pairwise comparisons to follow up on specific differences with task, hemisphere, electrode and decile as within-subject factors. These analyses were performed for the dependent variable of Pow and for all the within-subject variables. The threshold for significance in all the analyses is p  0.05. To compare the TRP scores, we performed an analysis of the full data set by running a 4  2  7 repeated-measures ANOVA, with the within-subject

544

S. Vieira et al.

factors 4 tasks, 2 hemispheres and 7 electrodes per hemisphere and extracted the data for the three tasks of interest. To compare the Pows we performed the same analysis for them. To compare the three tasks, we used the results from the analysis of the full data set. We performed pairwise comparisons of the seven frequency bands’ Pow by running for each of them a 2  2  7 ANOVA, with the within-subject factors of task, hemisphere and electrode. We then compared the frequency bands’ Pow across temporal deciles for the two open design tasks. The division of each design session’s data into temporal deciles allows a more detailed analysis of the temporal dimension. We performed an analysis by running a 2  2  7  10 ANOVA, with the within-subject factors of task, hemisphere, electrode and decile.

4 Analysis of Results Results of total task-related power (TRP) across the 18 participants, indicate that the tasks can be distinguished from each other, Fig. 5. The radar diagram plot simulates the two hemispheres by distributing the electrodes (10–10 IS) symmetrically around a vertical axis. Total TRP scores per electrode can be considered by comparing the vertical scale values and across the three tasks. Once the problem-solving Task 1 (reference task) is subtracted from itself to produce the reference, it shows up as an orange circle with a value of zero for all electrode measurements. The difference is shown as higher or lower activation of the electrodes/regions per task within or beyond the Fig. 5 Task-related power (TRP) for the 14 electrodes by taking problem-solving Task 1 as the reference period for the open design tasks (open layout design Task 3 and open sketching design Task 4)

Total TRP Industrial Designers (18) Task1

Task3 2

AF3

Task4

AF4

1.5 1

F3

F4

0.5 0 F7

F8

-0.5 -1 -1.5

FC5

FC6

T7

T8

P7

P8 O1

O2

Designing-Related Neural Processes … Fig. 6 Transformed power (Pow) and channels of statistically significant differences between Task 1 and Task 3, and between Task 1 and Task 4 highlighted with a solid circle

545

Total POW Industrial Designers Task3

Task1 AF3 F3

F7

100 90 80 70 60 50 40 30 20 10 0

Task4

AF4 F4

F8

FC5

FC6

T7

T8

P7

P8 O1

O2

orange circle border. We found a significant main effect of: task, p = 0.02, and hemisphere, p = 0.02. The open design tasks (Tasks 3 and 4) show increased amplitudes from the problem-solving Task 1, Fig. 5.

4.1

Analysis of Transformed Power across Tasks

Total transformed power (Pow) for each task across the 14 channels are depicted in Fig. 6. The plot simulates the two hemispheres by distributing the electrodes (10– 10 IS) symmetrically around a vertical axis. Pow scores per electrode (average of entire task) can be considered by comparing the vertical scale values and across the three tasks. The open design sketching task, Task 4, shows higher neurophysiological activations than Task 1 and is followed in amplitude by the open layout design task, Task 3. Below we report on statistically significant (p  0.05) differences between tasks, shown as solid circles channel by channel. The pairwise comparisons reveals that the problem-solving Task 1 differs significantly from: • open layout design Task 3 (p < 0.01), and • open design sketching Task 4 (p < 0.001). The significant main effects are presented in Table 2.

546

S. Vieira et al.

Table 2 Significant main effects from the repeated-measurement ANOVA

task (p