Software Architecture: Research Roadmaps from the Community 3031368460, 9783031368462

Citation preview

Patrizio Pelliccione Rick Kazman Ingo Weber Anna Liu   Editors

Software Architecture Research Roadmaps from the Community

Software Architecture

Patrizio Pelliccione • Rick Kazman • Ingo Weber • Anna Liu Editors

Software Architecture Research Roadmaps from the Community

Editors Patrizio Pelliccione Computer Science Area Gran Sasso Science Institute L’Aquila, Italy Ingo Weber Technical University of Munich School of CIT, and Fraunhofer Gesellschaft München, Germany

Rick Kazman University of Hawaii Honolulu, HI, USA Anna Liu Amazon Web Services Sydney, Australia

ISBN 978-3-031-36846-2 ISBN 978-3-031-36847-9 https://doi.org/10.1007/978-3-031-36847-9

(eBook)

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland Paper in this product is recyclable.

Preface

This edited book aims to provide a collection of the latest research roadmaps in the field of software architecture. Software architecture is a fundamental aspect of software engineering that has a significant impact on the success of software systems. Software architecture research involves the study of the design and analysis of software systems, focusing on the high-level structure and organization of software components, as well as the interactions and relationships between them. It also focuses on the non-technical aspects of software design: how teams are organized and how they communicate and work together. Over the years, the field of software architecture research has evolved to address the challenges and opportunities that have risen with the increasing complexity and longevity of software systems. The International Conference on Software Architecture (ICSA) is a series of conferences that has existed, in various forms and with shifting names, for over 20 years. In these conferences, leading researchers and practitioners have gathered annually to discuss the latest results in software architecture research and to share the many challenges and advances of software architecture in practice. In 2022, ICSA was held in two formats: a virtual conference in March with the usual program and a working conference in August that took place in Honolulu, Hawaii. The latter comprised various focused workshops to discuss a wide range of cutting-edge research roadmaps that were attempting to determine and perhaps even shape the future of software architecture research. This book contains a distillation of the outputs from those ICSA 2022 working sessions, as well as the subsequent work from many of the authors who were also participants of ICSA 2022. The first three chapters investigate software architecture for emerging classes of software systems with widespread interest, including quantum computing, artificial intelligence-centric systems, and systems within value-based ecosystems. Subsequent chapters investigate the role of architecture in relation to modern development processes, sharing of data as an enabler for furthering research in software architecture, and teaching software architecture. And the final chapter focuses on how we may improve the teaching of software architecture. In the following, we provide an overview of the chapters. Chapter 1 “Challenges and Opportunities in Quantum Software Architecture,” provides an overview of the v

vi

Preface

issues related to designing software architectures for quantum computing systems. With the advent of quantum computing, software architects face new challenges in designing software architectures that can leverage the power of quantum computing. This chapter provides an overview of the current state of the art in quantum software architecture research and discusses future research opportunities in this emerging field. Chapter 2 “Data-Intensive Software Systems: To AI and Beyond,” presents a roadmap for data-intensive and AI-based software system architectures. The authors identify, classify, and reflect on the main challenges for the development of these systems, focused on data collection, analysis, and post-processing. The roadmap summarizes the main research challenges identified, including metadata and model management, model deployment architectures and scalability, and observability and adaptability. Chapter 3 “Architecting and Engineering Value-Based Ecosystems,” presents a future where people, systems, machines, and other entities form a dynamic and highly unpredictable ecosystem. Such ecosystems are organized around emerging goals, which can be collaborative as well as competitive or even malicious. Enhanced by artificial intelligence, humans may find themselves unprotected in their interaction with the digital world, not only in terms of their safety but also in terms of other higher values such as fairness and ethics. The authors in this chapter invite the research community to direct its focus toward methods and techniques for architecting digital ecosystems, where values such as fairness, ethics, and privacy are built in, and outline aspects and steps toward a better path. Chapter 4 “Continuous Alignment Between Software Architecture Design and Development in CI/CD Pipelines,” presents the importance of maintaining continuous alignment between software architecture design and the development process, for the purpose of clarity and ease of maintenance of the system, as well as the ability to detect architectural decay. The authors propose to extend CI/CD pipelines with methods to achieve architecture alignment, e.g., architecture recovery, to ensure updates in the codebase are reflected in updated architectural documentation where necessary. Chapter 5 “An Empirical Basis for Architecture Research,” discusses the motivation for having a repository of curated, standardized, clean, well-maintained, documented, easily accessible, reusable, and shared datasets to catalyze broader and more ambitious programs of empirical studies in software architecture research. This chapter provides an overview of the problems, motivations, and opportunities currently related to mining and sharing datasets for researchers in software architecture. The authors explore and describe which artifacts should be included in such a dataset, including code, documentation, and requirements, but also other architecturally relevant artifacts, such as architectural decision records, models, and other types of documentation. This information can also be complemented with revision history logs, social metadata, and email or chat discussion archives. This chapter calls for the community to work toward this goal. Chapter 6 “A Better Way to Teach Software Architecture,” first describes how software architecture education is not regularly taught in university curricula, and when it is taught, it is typically only through the heroic efforts of individual educators. Because there is a lack of good teaching material, software architecture is seldom taught, and because it is seldom taught, there is little push for better materials. This chapter

Preface

vii

aims to establish a base guideline for what software architecture topics need to be taught, both technical and non-technical topics, and suggests appropriate teaching methods to best prepare students to be software architects in practice. In summary, this book provides an overview of the latest research and developments in software architecture, covering a wide array of current and emerging topics. While the initial goal of this book was to bring together the body of knowledge across the ICSA community and to articulate research roadmaps for key topics, we hope this book also serves as a generally interesting resource for all those interested in the design and management of effective software architectures. Specifically, to researchers and students, we hope this book will become a valuable resource to aid you in identifying fruitful paths for future research, and we look forward to hearing about your latest software architecture research results and challenges in future ICSA events. L’Aquila, Italy Honolulu, HI, USA München, Germany Sydney, Australia

Patrizio Pelliccione Rick Kazman Ingo Weber Anna Liu

Contents

Challenges and Opportunities in Quantum Software Architecture . . . . . . . . Tao Yue, Wolfgang Mauerer, Shaukat Ali, and Davide Taibi Software Architectures for AI Systems: State of Practice and Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ian Gorton, Foutse Khomh, Valentina Lenarduzzi, Claudio Menghi, and Dumitru Roman Architecting and Engineering Value-Based Ecosystems . . . . . . . . . . . . . . . . . . . . . Patrizio Pelliccione, Barbora Buhnova, Sebastian Gottschalk, Ingo Weber, and Gregor Engels Continuous Alignment Between Software Architecture Design and Development in CI/CD Pipelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Davide Taibi, Yuanfang Cai, Ingo Weber, Mehdi Mirakhorli, Michael W. Godfrey, John T. Stough, and Patrizio Pelliccione An Empirical Basis for Software Architecture Research . . . . . . . . . . . . . . . . . . . . Rick Kazman, Roberto Tonelli, and Cesare Pautasso

1

25

41

69

87

A Better Way to Teach Software Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 Rick Kazman, Yuanfang Cai, Michael W. Godfrey, Cesare Pautasso, and Anna Liu

ix

Challenges and Opportunities in Quantum Software Architecture Tao Yue

, Wolfgang Mauerer

, Shaukat Ali

, and Davide Taibi

Abstract Quantum computing is a relatively new paradigm that has raised considerable interest in physics and computer science in general but has so far received little attention in software engineering and architecture. Hybrid applications that consist of both quantum and classical components require the development of appropriate quantum software architectures. However, given that quantum software engineering (QSE) in general is a new research area, quantum software architecture—a sub-research area in QSE—is also understudied. The goal of this chapter is to provide a list of research challenges and opportunities for such architectures. In addition, to make the content understandable to a broader computer science audience, we provide a brief overview of quantum computing and explain the essential technical foundations.

Wolfgang Mauerer’s work is funded by the German Federal Ministry of Education and Research within the funding program quantum technologies—from basic research to market—contract numbers 13N15645 and 13N16092. Tao Yue and Shaukat Ali are supported by Qu-Test (Project#299827) funded by the Research Council of Norway and Simula’s internal strategic project on Quantum Software Engineering. T. Yue () · S. Ali Simula Research Laboratory, Oslo, Norway e-mail: [email protected]; [email protected]; [email protected] W. Mauerer Technical University of Applied Sciences Regensburg, Regensburg, Germany Siemens AG, Corporate Technology, München, Germany e-mail: [email protected] D. Taibi University of Oulu, Oulu, Finland Tampere University, Tampere, Finland e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 P. Pelliccione et al. (eds.), Software Architecture, https://doi.org/10.1007/978-3-031-36847-9_1

1

2

T. Yue et al.

Introduction There is, in general, the consensus among the quantum computing (QC) community that QC applications will primarily be hybrid; that is, some features will be executed efficiently on classical computers, whereas some specialized features will be executed efficiently on quantum computers [1, 2]. This pattern is by pure choice; most known quantum algorithms, from Shor’s seminal factoring algorithm to recent variational approaches, rely on classical sub-routines at their very core that cannot be beneficially replaced by pure quantum approaches. As a result, there will be close interaction between classical and quantum software, demanding careful consideration of both worlds when designing quantum software architectures (QSAs). QSAs of quantum software define the high-level structure of the software system, including its components, their interactions, and data flow. QSA is a relatively new research area within the broader area of quantum software engineering (QSE). Therefore, we believe it is crucial to identify highlevel research questions (RQs) related to QSAs for hybrid applications, followed by devising potential research directions that deserve attention from the community. Thus, our goal is to present high-level research challenges for QSAs. To achieve the above goal, we identify four high-level RQs. The first RQ identifies research directions on whether we need to consider architectural designs for quantum software or whether classical design considerations are sufficient. The second RQ focuses more specifically on architecture-level concerns for quantum software, such as issues related to optimizing architectures, integration of various components, and interactions with hardware. The third RQ gets into further details of whether the knowledge gained from the classical world can be transferred to the quantum world, such as with respect to relevant standards, architecture specification languages, and architectural patterns. Finally, the fourth RQ discusses research directions related to processes of executing hybrid applications on a diverse set of computing resources from an architectural perspective. We organize the rest of the chapter as follows: section “Background” provides some relevant background. Section “Overview” discusses the overall organization of the RQs, whereas sections “RQ1: Do We Need to Consider QSA Designs in a Specialized Way?”–“RQ4: How to Run Hybrid Applications from an Architectural Perspective?” discuss each RQ in detail. Finally, we conclude the chapter in section “Conclusion”.

Background Since QC is a new paradigm for many members of the software architecture community, let us briefly summarize the characteristics that differ from the classical computation. How hardware is implemented varies widely among commercial vendors (even more so between research prototypes). For instance, superconducting

Challenges and Opportunities in Quantum Software Architecture

3

circuits, particle traps (cold ions, neutral atoms), ion implantation into crystal lattices, or optical technologies find use. Computations can be performed by applying quantum logic gates on quantum states (which is, conceptually, similar to applying classical logical gates on classical bits) or by using optimization-like approaches (quantum annealing) that are not equivalent in their computational capabilities to the quantum gate model, yet have some advantages in terms of physical implementation [3] and practical utility [4]. Details from both approach and implementation do not just have a substantial impact on the algorithmic level [5, 6], but may also influence architecturally relevant qualities [7], and can impact architecture decisions. In particular, it holds for any currently implemented quantum computer that (a) results are usually approximate, (b) execution times vary stochastically (even for identical programs on identical machines), and (c) algorithmic scalability depends on low-level details like the circuit generation from source code. Likewise, quantum computers will eventually be used as (a) an embedded component of a classical system that acts as a computational accelerator (like graphics processing units (GPUs), tensor processing unit (TPUs), or floating-point units (FPUs)) or (b) a cloud resource that is accessed via APIs. While the same distinction holds classically, hardware constraints (for instance, the need to cool chips to nearzero temperature for some approaches) will amplify differences between these two variants in a much larger way than what architects are currently accustomed to. It is not yet well understood where the exact boundaries between software architecture, software engineering, and algorithmic details reside for quantum computers. A final distinction that is not known from classical computing is between the following types of systems: – NISQ (noisy intermediate-scale quantum computer)—these systems are commercially available, usually in a prototypical form, at the time of writing. – PISQ (perfect (error-corrected) intermediate-scale quantum computer)—such machines are fully error-corrected, large-scale QC—these may be available within years, decades, or perhaps even never, as many technological and some foundational physical problems will need to be solved. – Special-purpose accelerators (e.g., annealers, coherent Ising machines, (Gaussian) boson sampling) whose computational capabilities may not necessarily be polynomially equivalent to gate-based quantum computers.

Overview QSA is still in its early stage and is primarily a less-touched research area. Therefore, in this chapter, we aim to answer four basic RQs, which we believe are relevant to the QSA community at the current stage. In the first step, we aim to check whether it is sufficient to rely on knowledge gained from the classical world to the quantum world regarding architectural designs

4

T. Yue et al.

(RQ1, section “RQ1: Do We Need to Consider QSA Designs in a Specialized Way?”). To answer this RQ, we particularly focus on checking the current practice of accessing quantum computers since access modes have direct impacts on architectural designs. We further check whether existing modeling notations for architecture descriptions can be reused because an architectural design needs to be specified and used as a medium for facilitating communications among stakeholders. Furthermore, to enable architectural-level analyses of trade-offs of design choices, a taxonomy of quantum-specific constraints and quality attributes is expected to be proposed, which we think is worth discussing in RQ1 as well. We further elaborate on the importance of striking a balance involving lowlevel, quantum-specific design choices at the architectural design level. One of the challenges for software engineers to develop quantum software is a lack of knowledge in quantum mechanics, physics, etc. There is, obviously, a practical need to raise the level of abstraction while keeping necessary details in the architectural design. Motivated by the positive answer to RQ1, when answering RQ2 (section “RQ2: What Are Relevant Architecture-Level Concerns for QC?”), we first define what we mean by an optimal architecture. Based on the definition, we then discuss relevant architectural-level concerns. One paramount concern is about how to integrate quantum components into existing architectures. We further highlight properties of quantum hardware that have an ineligible impact on result quality, speed of calculation, degree of variability in runtime, etc. We believe these properties are essential to consider during architectural designs. Furthermore, we list a set of other considerations (e.g., the impact of the evolution of QC hardware on architectural designs) and call for contributions. By knowing the architecture-level concerns identified in RQ2, in RQ3 (section “RQ3: Is Knowledge Gained from the Classical World Still Applicable to the Quantum World, and to Which Extent?”), we examine whether knowledge gained from the classical world is still applicable to the quantum world to address these concerns and present the literature of existing proposals particularly targeting QSAs. Specifically, we check whether existing architecture standards are still applicable and whether new standards are required for QSAs. We further check whether existing and classical world architecture description languages (ADLs) can be used for designing QSAs and discuss the state-of-the-art ADLs proposed specifically for quantum software. We also present existing proposals on quantum software architectural patterns. Considering that quantum computers are typically accessed via the cloud, we particularly discuss Quantum DevOps and quantum services. In RQ4, we present our thoughts on a possible process of running hybrid applications on diverse computing resources. Particularly, we elaborate on key roles/players in the process and their tasks.

Challenges and Opportunities in Quantum Software Architecture

5

RQ1: Do We Need to Consider QSA Designs in a Specialized Way? This RQ looks into whether we need to consider QSA in a specialized way as compared to the classical way or whether the classical software architecture also applies to quantum software. We discuss this from multiple dimensions below.

Accessing Quantum Computers Through Cloud In the current state of practice, quantum computers are located in cloud computing infrastructures. A user gets access to them through cloud services. Commercial players such as Amazon with its Braket platform1 provide access to various types of quantum computers such as gate-based, ion trap-based, and neutral-atom-based quantum computers. The same trend exists for other setups, too, e.g., NordIQuEst2 in a Nordic and Estonian infrastructure that provides access to Nordic quantum computers from Chalmers University, Sweden, and Technical Research Centre of Finland (VTT), Finland3 through European high-performance computing (HPC) platforms, e.g., LUMI4 and eX3.5 Irrespective of quantum computer types, services provided via the cloud do not differ from a standard way of accessing other services from a technical point of view. To this end, benefiting from existing architectural design notations to capture a user’s connections to various quantum computers via cloud services might be the right direction to go. In section “Quantum DevOps and Quantum Services”, we will discuss this more. In terms of research, we foresee the need for methods with software tools to help users automatically and optimally decide which quantum computers to use from the cloud infrastructure for their specific QC applications, which will provide the best computational capability with minimum noise. One work in the literature [8] describes various characteristics of QC platforms, followed by how different quality attributes, such as availability, performance, and portability, impact these attributes (also see some details in RQ2 and RQ3). Interested readers may consult the reference [8] for more details. To build such methods, one needs to develop a comprehensive quantum software architectural framework, which captures characteristics of various types of quantum computers, such as computational power measured as quantum volume, type of technologies, and physical arrangements of

1 https://aws.amazon.com/braket/. 2 https://nordiquest.net/. 3 https://www.vttresearch.com/. 4 https://www.lumi-supercomputer.eu/. 5 https://www.ex3.simula.no/.

6

T. Yue et al.

qubits. Such an architectural framework can be considered the foundation of helping in optimally deploying QC applications. In general, we believe that accessing quantum computers through highperformance computing (HPC) platforms will remain one of the future dominant ways for QC. Therefore, QSAs shall consider various aspects of cloud computing.

Use of Existing Notations via Extensions One aspect to investigate is whether a QSA could be captured with existing modeling notations, e.g., UML component diagrams, by providing extensions. Examples of QC concepts include gates, qubits, superposition, entanglement, etc. Keeping our example of a UML component diagram, one could define a Unified Modeling Language (UML) profile consisting of a set of stereotypes and their associated attributes, data libraries, and constraints, which could be used to model QSAs. We feel that existing architectural design methods/notations/tools/methodologies can largely be reused. One point, however, to consider is that quantum software is basically about making quantum circuits transpiled for a given gate model-based quantum computer for its specific gate set and other characteristics. Unfortunately, all this is too low level, and high-level abstractions are missing. Thus, once such high-level abstractions are built, the architecture of quantum software might look quite similar to a classical one, especially if we manage to develop successful abstractions away from low-level concepts of qubits and gates. We believe that building useful abstractions is the right way for QC to be more successful, as highlighted in existing work [9]. One immediate area of research that one could explore is empirically evaluating existing languages used for capturing the architecture of classical software for QSA of diverse QC applications. Examples of these languages include UML, Systems Modeling Language (SysML), and other architecture description languages (e.g., Architecture Analysis and Design Language (AADL) and Wright [10]). With such empirical evaluation, one could assess the strengths and weaknesses of each language in terms of capturing the architecture of quantum software. Once the weaknesses have been identified, we can assess whether we can extend these languages or need entirely new ones for QSA. In section “Architecture Description Languages”, we discuss more about applying existing modeling notations for architectural descriptions by examining the literature. When designing QSAs, one may need to introduce pictorial representations (e.g., similar to Feynman diagrams) of quantum computation processes into architecture documents. This may be more important than in the classical world, as quantum software development is more complex, so high-level abstraction is important, which can also provide the abstraction for quantum computation in the architecture. Toward this direction, we foresee the need to develop novel graphical notations that could extend the existing architectural design notations. Such notations could be

Challenges and Opportunities in Quantum Software Architecture

7

at different levels of abstraction depending on a user’s background. For instance, if a user has no background in quantum mechanics, then the details that are irrelevant to the user may be omitted.

Need for Quantum-Specific Constraints In a classical architectural design, it is common to identify and capture constraints that are used to make trade-off decisions about design choices and perform optimization accordingly. When making the architecture of quantum software, do we also need to capture some constraints that are specific to the quantum domain? There are indeed constraints that are specific to the quantum domain. Taking an example of superconducting quantum computers, physical qubits can be implemented in different ways, e.g., star structure. How such qubits are implemented has a significant impact on the architectural design of the quantum software, which shall be executed on a quantum computer with a particular implementation structure. Given that the current quantum computers are immature with an abundance of hardware errors (e.g., in qubits, gates, crosstalk), specifications of such constraints are also essential for optimizing the architectural design of quantum software for a specific quantum computer. In addition, other factors for optimization, such as limited computation resources, continuously advancing quantum technologies, and optimal ways of splitting quantum and classical tasks, need to be considered. In terms of future research, our recommendations are as follows. First, we must design a novel taxonomy of constraints specific to quantum software architectural designs. Such a taxonomy shall be generic and applicable to various QC applications and quantum computers. Second, the taxonomy shall also provide the provision for specific domain-specific architectural constraints. Third, there is a need to develop novel methods to specify various constraints. Fourth, we need an optimization method that can use these constraints and additional information (e.g., information about the underlying architecture of a quantum computer) and automatically generate an optimal QSA. Toward this end, one would like to explore the novel field of quantum meta-heuristics [11]. Such meta-heuristics aim to solve optimization problems in the Hilbert space, which will be our context. However, currently, the field is immature and needs extensive development. One might consider that, when the chapter is written, it might be premature to develop the taxonomy because a wide range of QC hardware exists without having any of them dominant and more are emerging. In our opinion, developing the taxonomy might help converge to a set of generic understandings (in the form of constraints) that would help perform analyses and optimizations. Moreover, with the rapid advancements in QC, new constraints are emerging, and therefore, the taxonomy needs to be constantly evolved.

8

T. Yue et al.

Classifying Quantum-Specific Quality Attributes While optimizing QSAs, one needs to consider quantum-specific quality attributes. Such quality attributes could be functional or extra-functional. An example of the functional quality attribute includes circuit depth.6 For example, if an algorithm has more than one implementation as quantum circuits, then the implementation with a lower circuit depth is preferred. In terms of extra-functional quality attributes, the generic classification, such as performance, portability, and robustness, still holds. However, these categories need to be specialized for QC. Regarding research recommendations, we believe there is a need to build a detailed taxonomy of functional and extra-functional quality attributes. Such attributes shall cover both quantum-specific attributes and attributes from a classical domain that apply to QC. Such a taxonomy can be continuously refined with more concrete metrics as those are developed. In addition, users can use such taxonomy for their own purposes.

Prevalence of Low-Level Design Choices in Architectural Designs For architectural design optimization, we must consider available resources regarding acceptable noise level, number of qubits, etc. These “low-level” details might be needed at the very early stage of the decision-making process in contrast to the classical counterpart. As we will detail in section “RQ2: What Are Relevant Architecture-Level Concerns for QC?”, it is vital to strike the proper balance regarding the level of abstraction. Thus, dedicated research methodologies and software tools are needed to elicit such requirements at the earlier stage of QSA design.

RQ2: What Are Relevant Architecture-Level Concerns for QC? Before addressing the question of what architecture-level concerns are relevant for software systems that employ quantum processing units (QPUs) and their associated custom or generic software components, we need to set our notion of an optimal architecture, which can be the goal of optimally tending to all the concerns we will discuss in this section:

6 In the context of quantum circuits, the circuit depth represents the longest path in a given quantum circuit.

Challenges and Opportunities in Quantum Software Architecture

9

Optimal Architecture: a collection of classical and quantum software, and their interrelations, used to solve a given computational task within the required degree of correctness, using an acceptable amount of time, and respecting necessary and unavoidable trade-offs in the use of resources. Note that while this software-centric definition is independent of quantum hardware, an optimal quantum architecture will strongly depend on the specific technical details of the underlying QPUs. This is certainly true in the NISQ area [3], but may likely also apply beyond if variational quantum algorithms remain a technique of choice. The latter, however, will and can only be decided once sufficiently mature hardware has become available, or variational mechanisms (and related variants like quantum approximate optimization algorithm (QAOA) or variational quantum eigensolver (VQE)) will be placed on more solid grounds with improved theoretical underpinning, or both. While it might be tempting to, from a software architecture point of view, eliminate the signature of specific properties of quantum hardware by layers of abstraction [12], the degree of influence imposed by the varying physical foundations of different QPU approaches can endanger the potential quantum advantages when a too strong degree of abstraction is tried. While we will detail this consideration below, it is important to note that quantum software architectures cannot be decoupled from quantum system architectures and vice versa. Consequently, many architectural considerations will differ from traditional software architectural designs and require (a) the awareness of software architects in the first place and (b) a better understanding of low-level details that need to be subject to the relevant academic curricula or post-graduate industrial education efforts. We below discuss relevant criteria that allow us to achieve optimal quantum software architectures by focusing on two major aspects: influence and relevance of hardware details on architectural properties and quantum-classical integration. We believe that such considerations will differ between essentially four domains: general-purpose computing (where cost considerations prevail), HPC (focusing on latency and throughput), embedded industrial computing (focusing on latency and hardware integration), as well as solving singular problems of outstanding importance beyond pure demonstrations of quantum advantage.

Integrating Quantum Components into Existing Architectures In contrast, how to integrate quantum components into existing architectures is less of a concern, as some of the authors, for instance, argue in Ref. [13]. Essentially, we need to distinguish between two access modes: (a) cloud based (or using some other queuing systems) with asynchronous means of interacting with the QPU— for instance, vendors IBMQ, D-Wave, or Rigetti currently operate using this access mode; and (b) queue local based with synchronous access that can be established by either direct attachment of the QPU to a classical system via a local network (with

10

T. Yue et al.

Table 1 Possible combinations of algorithmic types and QPU access modes (left) and a general interaction pattern between QPU and CPU (right)

...

...

...

|0> |0>

Measure

|0> |0>

Iterate State Prep.

|0>

...

Algo/Access Cloud/Queue Local 1 2 Stochastic Variational 3 4

controlled maximal latency) or even via direct on-chip integration of CPU and QPU, assuming future extended physical manufacturing and engineering capabilities.7 This mode of operation is targeted by many HPC use cases [14] and is currently pursued by vendors like IQM or IonQ. On the other hand, we can broadly classify known quantum algorithms alongside two characteristics: (a) There are stochastic/approximate algorithms that usually require interaction with a classical system, but deliver a result that, on perfect error-corrected hardware, is either sufficiently close to a desired optimal solution or a perfect solution with a high probability—for instance, Shor’s seminal factoring algorithm or Grover search [15], as do quantum annealing-based approaches [3]. Alternatively, there are (b) variational algorithms that run a parameterized quantum circuit multiple times and optimize employed quantum gates in each run toward producing an optimal desired solution based on classical numerical optimization techniques—for instance, QAOA [16], VQE [17, 18], or many machine learning techniques like QRL [19]. Here, the number of iterations and therefore the runtime are unknown upfront, as the convergence behavior of the underlying parameter optimization is not yet fully understood (yet, see, e.g., Refs. [20, 21] for the current state of hypotheses and understanding). The situation is summarized in Table 1. Depending on the particular class an approach falls into, there will be different implications on architectural properties. However, once the functional and nonfunctional properties of a quantum component are given, the integration into any existing architecture can in every case be described by a single call to an API that follows the “data in, result out” pattern. This is nearly trivial from a programmer’s point of view, but of course, the aspect of interest for architecture is how the functional and non-functional properties reflect on the overall architecture.

7 Note that for iteration-based algorithms, such details may make a difference [7], albeit we can safely ignore these in this architecture-centric overview.

Challenges and Opportunities in Quantum Software Architecture

11

Concerning Hardware-Specific Properties in Architectural Designs Quantum hardware design is very much in flux, with even individual vendors experimenting with different physical designs (e.g., IBM with qubits based on superconducting Josephson junctions and silicon quantum dots). Any quantum problem, at least in the era of NISQ machines, requires a specific quantum encoding,8 which then needs to be translated into specific physical inputs for particular machines. While this seems identical to translating high-level languages into machine code at a superficial level, the properties of the translation may vary widely with the concrete physical design of the machine (see, e.g., Ref. [7, 22]).

Hardware and Software Coupling Properties of underlying hardware (as well as the translation process itself [7, 22]) influence result quality, the speed of calculation, the ability to find a result at all, the range of input data that can be processed, and the degree of variability in runtime, as these properties directly arise as a consequence of: – The degree and type of noise/loss on the capability of individual qubits to store data and the influence of decoherence that leads to increasing loss of quantum information with deeper circuits/longer programs – The connectivity between qubits (e.g., which qubits can be subjected to joint operations and which are not) and imperfections arising from the coupling structure – The ability to prepare initial states for computation and intermittently “clear” qubits for reuse – Imperfections of the measurement process that turns quantum information into classical results such that they can be interpreted and processed by classical IT These aspects must be considered in designs that strive for optimal architecture for a given use case. Especially for variational algorithms, there is a direct tradeoff between longer circuits/larger number of iterations (that can be shown to lead to monotonically increasing result quality for some approaches like QAOA) and the consequences like the increasing influence of noise, which, in turn, deteriorates result quality.

8 It is doubtful if it will be possible to produce perfect error-corrected machines for each imaginable use case; we expect noisy machines to remain relevant also in the long run if it will be possible to achieve quantum advantage with them, albeit this remains an open scientific question [3].

12

T. Yue et al.

Hardware-Specific Optimization Logical circuits (the primary output of a translation/compilation from a high-level description in any quantum framework into an executable sequence of operations on quantum states) are, in principle, independent of the executing QPU. But they are already implicitly connected with the target hardware by the choice of gate set, which is used to manipulate individual or multiple qubits. When a logical circuit is transpiled to a concrete physical implementation, logical gates must be replaced by (combinations of) available native hardware gates, and this influences architectural properties like scalability, runtime, result quality, and so forth. There is optimization potential by designing hardware-optimized algorithms [3] that center around the operations available in specific implementations, thus eliminating some of the aforementioned overhead. However, this does not solve (and is independent of) the connectivity problem: Since not every qubit can interact with each other qubit, any necessary operations between “remote” qubits must be enabled by first bringing the qubits into close enough proximity and then execute the gate. (Logically) moving qubits around is possible with so-called swap gates that exchange the quantum state of two qubits (without requiring a measurement that would destroy the quantum state). As, in turn, native swap gates are not available on most architectures, it needs to be replaced by a sequence of three controlled-not gates, increasing circuit depths even further. Enhancing the connectivity of the underlying systems, possibly in a problem-specific way, can provide improvements, albeit at the cost of co-designing hardware and software. Likewise, technologies like cold neutral atoms provide universal connectivity between qubits—albeit under the negative influence by prolonged gate execution times [23] as compared to other implementation platforms, which especially deteriorates the temporal performance of iterative schemes 3 , 4 . Some parts of applications could therefore be optimized for different types of quantum technologies. However, it remains a question whether doing so is commercially viable and whether the arising disadvantages (for instance, having to synchronize two queues in the case of cloud access 1 , 3 ) defeat the advantages gained by a multi-QPU scheme.

Consequences As we have discussed, low-level properties of QPUs have a direct (and often unaccustomed to the classical world) impact on architecture-level properties. We, therefore, believe that (a) the awareness of this relation is available to software architects and (b) mapping between the architectural quality and low-level properties needs to be defined and maintained in any architectural design document or model, of course satisfying established requirements on traceability [24].

Challenges and Opportunities in Quantum Software Architecture

13

Additionally, we expect that not the scalability of algorithms is the most relevant criterion, but the absolute performance for a given set of inputs; many industrial use cases (e.g., factory control, production planning, long-term resource distribution [4]) work on input data sets of fixed (or merely varying) sizes, whereas absolute execution times are the most relevant concern. Similar considerations apply to HPC workloads.

Other Considerations The properties of QC hardware are subject to considerable flux, and even many of the elementary theoretical characteristics (like runtime!) of algorithms themselves are not yet well understood. This induces uncertainties in the design of QSAs. We find that more research is desirable and necessary to understand such variability’s short- and long-term implications. – Evolution of QC Hardware: At this stage of QC development, we might also need to consider the fast evolution of QC technologies and the architectural design needs to particularly take care of changes of underlying QC technologies. To this end, architecture decay issues that quantum applications might need to face need to be addressed. – Empirical Studies: To perform architectural design optimization, empirical studies are needed to collect evidence. Currently, any possibility to compare structured approaches to architectural design with the classical world is limited, which is, however, also caused by the fact that no large-scale applications of QC are known. – Verification of Quantum Results: It may be a challenge to verify the results of quantum computations depending on the class of algorithms [25, 26] or to ascertain the correct functioning of QPUs [27]. This concern should also be considered at the architectural level, as it may influence correctness, performance, and quality properties. – Deployment of QC in Real-Time/Safety-Critical Contexts: Especially in scenario 1 (and to a lesser degree in 2 ), it is impossible to specify upper bounds on the execution time of a computation. To integrate such applications into industrial control scenarios, likewise, it is known that tail latencies, even if they are rare, can cause substantial issues in data processing schemes [28]. Latency is also known to be among the most important performance characteristic in HPC scheduling [29]. Appropriate countermeasures (e.g., fallback to safe states, classical refinement of approximate results, latency tailing) must be considered directly at the architectural level.

14

T. Yue et al.

RQ3: Is Knowledge Gained from the Classical World Still Applicable to the Quantum World, and to Which Extent? This question is raised to examine whether we can reuse methodologies, tools, and standards, among others, created for developing classical software for developing quantum software and to which extent they can be reused. This question is crucial as we aim to save as much effort as possible.

Standards The ISO/IEC/IEEE 42010:2022 (Software, systems and enterprise—Architecture description) standard9 is a well-known and commonly practiced standard for devising architecture descriptions. In this standard, terminologies such as architecture, architecture description (AD), architecture description framework (ADF), architecture description language (ADL), architecture view and viewpoints, stakeholders, and concerns are defined. The standard also defines their relationships. For instance, concern is linked to a stakeholder describing her/his concern on the relevance or importance of a matter. Since these concepts are defined at a very high level, which is intended by most standards, we believe the conceptual model of the ISO/IEC/IEEE 42010:2022 is still applicable to the quantum world. However, we like to mention that bringing concepts from the quantum world will be useful when instantiating the conceptual model. For instance, in addition to typical viewpoints such as operational, logical, technical, and deployment viewpoints, it might be important to bring integration viewpoints where interactions of the classical world and quantum world meet for developing hybrid software applications, as we discussed in section “Integrating Quantum Components into Existing Architectures”. Also, a deployment viewpoint might be tailored to consider different computing resources, as some might be quantum computers of the same or different types. Another example is that we might need quantum-specific modeling solutions to describe architecture views. We will discuss this in detail in section “Architecture Description Languages”. In summary, existing standards are still generally applicable to guide the development of QSAs. Methodologies and tools conforming to these standards might need to be customized for developing quantum and hybrid applications to enhance their applicability and usability.

9 ISO/IEC/IEEE

42010:2022: https://www.iso.org/standard/74393.html.

Challenges and Opportunities in Quantum Software Architecture

15

Architecture Description Languages Different modeling notations (e.g., UML, SysML) can be used to describe an architecture. For instance, shortly after UML was standardized by the Object Management Group (OMG), Rich Hilliard [30] investigated the application of UML in the context of IEEE P1471, which was superseded by ISO/IEC/IEEE 42010 later on and concluded that UML provides a set of notations that can be used to model various architecture aspects of systems. In QSA, attention has been naturally drawn to UML. For instance, the authors of the short paper [31] suggested distinguishing UML model elements in terms of whether they carry quantum information. Such a suggestion is necessary; however, it is far from being sufficient for specifying architectures of quantum software. In [32], the authors presented novel ideas for developing quantum software modeling languages along with a conceptual model of quantum programs. They illustrated how to model the state-based behavior of quantum programs. In [33], the authors presented a position paper arguing the potential benefits of applying model-driven engineering (MDE) practices for developing software. Specifically, the authors highlighted that QC resources are mostly accessed via cloud services (e.g., Amazon Braket) and therefore, current modeling languages for cloud computing might be worth being investigated. Representing processes by graphical abstractions is not uncommon in quantum physics; for instance, Feynman diagrams [34] are widely used representations of certain mathematical aspects of quantum electrodynamics and related theories. However, they are in one-to-one correspondence with calculations and do not provide an abstraction in the software engineering sense, but instead make complex formulas easier accessible to intuition. Quantum circuit diagrams, which are also commonly employed to visualize quantum algorithms, reside at a higher level of abstraction from a physical point of view, yet correspond to the layer of assembly language from a computer science point of view. This is also well below the usual levels of abstractions as they appear in software architecture and engineering. Approaches like quantum pictorialism, as proposed and detailed by Coecke and Kissinger [35], might pave a road that bridges between physics and computer science via visual abstractions. Yet, none of the existing solutions seems sufficient to model the architecture of any non-trivial software application with quantum components. Moreover, we observe that to develop a practical-useful ADF and ADL, we first need access to real-world quantum applications, which are hardly available to researchers at the moment. Existing modeling notations (e.g., UML) and paradigms (e.g., MDA) can be a natural foundation for developing quantum-specific ADFs and ADLs. We do not foresee the need for radically new modeling notations for now, which is partially due to our limited knowledge of real-world quantum applications.

16

T. Yue et al.

Architectural Patterns Architectural patterns (e.g., client-server patterns) represent ideas, practices, and solutions that are reusable for addressing reoccurring problems. By definition, this is still applicable to QSA. Various patterns and pattern languages have been proposed in the literature for supporting quantum software development and testing. For instance, Leymann [36] proposed a template for structuring pattern documents, including fields such as name, intent, problem statement, solution, and known use. With this template, Leymann proposed an initial set of basic patterns for developing quantum algorithms, such as Initialization (aka State Preparation) to answer the question of how the input of a quantum register can be initialized, and Creating Entanglement, and Phase Shift. However, as stated clearly by the author, these patterns are design (not architectural) patterns for developing quantum algorithms. Weigold et al. [37] also proposed six encoding patterns to facilitate efficient data loading of quantum algorithms; three of the six patterns are for data encoding and state preparation (e.g., preparing an arbitrary state), and two of them are for unitary transformation (e.g., matrix encoding), and one is for measurement pattern (i.e., post-selective measurement). In [38], Zhao et al. presented a set of bug patterns (“erroneous code idioms or bad coding practices” in [38]) such as insufficient initial qubits, along with a classification of them, in the context of developing quantum programs with Qiskit. However, it is worth noting that these existing works focus on identifying patterns at the design and implementation levels, not on the architectural level. Consequently, the reported design and bug patterns are all quantum-specific. As discussed in sections “Integrating Quantum Components into Existing Architectures” and “Concerning Hardware-Specific Properties in Architectural Designs”, we see the need to define patterns for, for instance, various modes of integrating quantum components.

Quantum DevOps and Quantum Services Inspired by the agile software development principle, DevOps was proposed to shorten the software development lifecycle and enable continuous delivery. In principle, being a set of recommended practices, DevOps does not put restrictions on using any specific architectural styles. However, the microservices architecture is often applied together with DevOps for developing continuously deployed software systems, which is composed of small services communicating via APIs and the cloud. From the architectural aspect, the microservices architecture is distributed and loosely coupled to facilitate fault isolation, continuous and fast integration of

Challenges and Opportunities in Quantum Software Architecture

17

new features, etc. For instance, Azure DevOps10 is a platform that implements DevOps with a set of cloud-hosted services for building, testing, and deploying programs in various programming languages on multiple operating platforms. To get access to QC resources, the cloud is mainly used for two purposes: (1) getting access to classical computing resources for the purpose of simulating quantum computers, such as the early stage of validation of quantum software that can be achieved, and (2) getting access to limited and shared quantum computers that are often placed in very specialized room conditions. Therefore, a number of cloud-based QC platforms (e.g., Amazon Braket,11 D-Wave Leap12 , and Xanadu’s Quantum Cloud13 ) have been made available for use via the cloud. Therefore, DevOps for QC emerged. For instance, Azure Quantum proposes two DevOps loops:14 the outer loop (which is a complete DevOps cycle) and the inner loop, which is quantum-specific and involves activities (e.g., programming quantum software, executing quantum software on simulators or quantum computers, estimating required computing resources) that need to be performed by quantum architects and engineers. In the literature, Gheorghe-Pop et al. [39] promoted the concept of Quantum DevOps in the context of dealing with uncertainties of NISQ computers and envisioned building Quantum DevOps by extending the traditional DevOps concept. To enable the realization of this concept, a set of quantum-specific software engineering methodologies (including tool support), across several key phases of a software development lifecycle, such as requirements engineering, design, and testing, are needed but are largely unavailable up to date, as also highlighted in [9]. As one of the key elements of DevOps, Gheorghe-Pop et al. [39] mentioned that, for testing quantum software to be run on NISQ computers, it is needed to go through three testing phases: testing in a simulator without noise, testing in a simulator with injected noise mimicking real qubit environment, and testing on real quantum computers. In the literature, a set of approaches (e.g., [40–43]) on testing quantum software have been proposed. However, to realize Quantum DevOps in the NISQ era or beyond, scalable and efficient testing approaches across the three quantum software testing phases are expected. To conclude, currently, the concept of Quantum DevOps seems promising; however, supporting its realization, methodologies, and tools across various software development lifecycles needs to be proposed or matured. Moreover, investigations on whether the microservices architecture is still applicable for Quantum DevOps are needed as well.

10 Azure

DevOps: https://azure.microsoft.com/en-us/products/devops/. Braket, https://aws.amazon.com/braket/. 12 D-Wave Leap: https://cloud.dwavesys.com/leap/login/?next=/leap/. 13 Xanadu Cloud: https://www.xanadu.ai/. 14 DevOps for QC at Azure Quantum: https://learn.microsoft.com/en-us/azure/architecture/guide/ quantum/devops-for-quantum-computing. 11 Amazon

18

T. Yue et al.

In addition to the idea of Quantum DevOps, back in 2004, the authors of [44] promoted to introduce a layered software architecture to support the development of quantum algorithms by following a four-phase design flow going from a high-level representation of a quantum algorithm all the way to a technologyspecific implementation, through a series of transformations. Recently, Moguel et al. [45] investigated opportunities and challenges of introducing the serviceoriented architecture (SOA) architectural style to the quantum world in the context of developing classical-quantum hybrid software systems, where quantum software systems are invoked as quantum services. The proposal comes naturally because most quantum computers can currently be accessed through the cloud (section “Integrating Quantum Components into Existing Architectures”). The authors experimented with Amazon Braket by wrapping quantum services as classical services and demonstrated the potential. But, the authors also concluded that, based on their investigation, there is an insufficient benefit for running quantum algorithms as classical services and new methodologies and tools are needed, which we also agree.

RQ4: How to Run Hybrid Applications from an Architectural Perspective? As discussed in section “RQ4: How to Run Hybrid Applications from an Architectural Perspective?”, there are various ways of getting access to quantum computers, and some properties of quantum hardware (e.g., scalability, runtime, result quality) are impacted by the degree and type of noise, connectivity of qubits, etc. of quantum hardware. All these aspects have an influence on achieving the optimal architecture we envision. However, before achieving such an optimal architecture for running quantum software on quantum hardware, a common practice is using quantum simulators such that quantum programs can be run and tested before being executed on quantum hardware. We envision that in the NISQ era, architecture optimization (section “RQ2: What Are Relevant Architecture-Level Concerns for QC?” might need to rely on quantum computer simulators of various types, including simulators simulating pure qubits and various hardware noises. There are two possible ways of performing a simulation: (1) using a quantum simulator (with the fundamental limits being about 45 qubits for noise gatebased simulation and 10,000 qubits for noisy quantum annealing) and carefully extrapolating results to larger computing resources and (2) using an effective model, based on the structure of a quantum circuit, imperfections of gates, speed of decoherence, etc., to predict most likely performance/result quality characteristics. Moreover, it is important to identify QSA that covers both classical and quantum components of software systems, including, for instance, user interfaces (or APIs), databases, and computation parts. Submitting a computation request to quantum hardware needs a series of steps starting from data pre-processing. When results are

Challenges and Opportunities in Quantum Software Architecture

19

Fig. 1 Running hybrid applications—an architectural perspective

ready, they will also need to be processed to be usable. This process is particularly commonly seen for systems running artificial intelligence (AI). Currently, AI is executed on traditional machines that send a specific part of the computation to GPUs. Developers only need to develop their AI algorithms, and a dedicated software or platform is responsible for orchestrating the computation between the CPU and GPU. It is not yet possible to have a hybrid and seamless architecture that covers both the quantum and classical worlds. Developers must manually decide what part of the computation needs quantum hardware and which needs classical ones. Commonly, developers manually send input data to quantum hardware, and when results are ready, they fetch the results and continue the elaboration or processing with other classical machines. We foresee the need for an integrated quantum and classical computing architecture. A process mainly has the following steps (as also illustrated in Fig. 1): – Developers develop quantum and classical code to implement their systems. We foresee the possibility of developing a framework, with which developers can specify which part of the computation must be sent to quantum computers and which part must be computed with classical hardware resources. – Resource Orchestrator automatically orchestrates and therefore sends computation tasks to quantum and/or classical hardware resources. – Resource Orchestrator sends the quantum program to the quantum program compiler, which compiles the program and executes it on quantum hardware, which could be identified by the orchestrator or manually. – Result Interpreter receives results once the execution of the quantum program is concluded. Hardware noise is an important aspect to consider in this step. The result interpreter should be able to understand if there is the need to run the program once more, based on a pre-defined confidence level, for instance.

20

T. Yue et al.

Let’s consider a practical example to illustrate the process. A weather forecast company might decide to use QC to provide a web service reporting the weather forecast in certain areas. The developers will develop the code for the weather forecast station composed of four main steps: (1) obtain sensor data; (2) process the data and send the pre-processed data to the QC algorithm that has been developed; (3) interpret the data returned by the quantum algorithm; and (4) save results in a database accessible via the web service. With this process, we envision that software development effort will be reduced as the splitting of tasks and allocating them to different resources can be, in an ideal situation, automatically performed by the resource orchestrator. We are aware that currently, there is no such an approach, and developers need to manually code each step, manually interpret results, and then again manually send them to an accessible database. In the far future, we foresee the option of adopting QC algorithm seamlessly with a process like the one we sketched.

Conclusion Quantum software architecture (QSA) of quantum computing (QC) applications that require both classical and QC resources is an understudied area of research. Nonetheless, within quantum software engineering, researchers and practitioners are getting more and more interested in it. Due to this reason, the objective of this chapter was to provide an initial set of research challenges and opportunities for QSA after examining a set of research questions and also put light on future research directions. These discussions are preliminary and need to be further enhanced with more thorough investigations, such as systematic literature reviews, empirical evaluations, and surveys with industry and researchers.

References 1. Weigold, M., Barzen, J., Leymann, F., Vietz, D.: Patterns for hybrid quantum algorithms. In: Barzen, J. (ed.) Service-Oriented Computing, pp. 34–51. Springer International Publishing, Cham (2021) 2. Callison, A., Chancellor, N.: Hybrid quantum-classical algorithms in the noisy intermediatescale quantum era and beyond. Phys. Rev. A 106, 010101 (2022). [Online]. https://link.aps. org/doi/10.1103/PhysRevA.106.010101 3. Bharti, K., Cervera-Lierta, A., Kyaw, T.H., Haug, T., Alperin-Lea, S., Anand, A., Degroote, M., Heimonen, H., Kottmann, J.S., Menke, T., Mok, W.-K., Sim, S., Kwek, L.-C., AspuruGuzik, A.: Noisy intermediate-scale quantum algorithms. Rev. Mod. Phys. 94, 015004 (2022). [Online]. https://link.aps.org/doi/10.1103/RevModPhys.94.015004 4. Bayerstadler, A., Becquin, G., Binder, J., Botter, T., Ehm, H., Ehmer, T., Erdmann, M., Gaus, N., Harbach, P., Hess, M., Klepsch, J., Leib, M., Luber, S., Luckow, A., Mansky, M., Mauerer, W., Neukart, F., Niedermeier, C., Palackal, L., Pfeiffer, R., Polenz, C., Sepulveda, J., Sievers, T., Standen, B., Streif, M., Strohm, T., Utschig-Utschig, C., Volz, D., Weiss,

Challenges and Opportunities in Quantum Software Architecture

21

H., Winter, F.: Industry quantum computing applications. EPJ Quantum Technology 8(1), 11 (2021). [Online]. https://epjquantumtechnology.springeropen.com/track/pdf/10.1140/epjqt/ s40507-021-00114-x.pdf 5. Gabor, T., Zielinski, S., Feld, S., Roch, C., Seidel, C., Neukart, F., Galter, I., Mauerer, W., C. Linnhoff-Popien, Assessing solution quality of 3sat on a quantum annealing platform. In: Proceedings of the International Workshop on Quantum Technology and Optimization Problems (QTOP) (2019). [Online]. https://arxiv.org/abs/1902.04703 6. Schönberger, M., Scherzinger, S., Mauerer, W.: Ready to leap (by co-design)? join order optimisation on quantum hardware. In: Proceedings of ACM SIGMOD/PODS International Conference on Management of Data (2023) 7. Wintersperger, K., Safi, H., Mauerer, W.: QPU-system co-design for quantum HPC accelerators. In: Proceedings of the 35th GI/ITG International Conference on the Architecture of Computing Systems. Gesellschaft für Informatik (2022) 8. Sodhi, B.: Quality attributes on quantum computing platforms. ArXiv, vol. abs/1803.07407 (2018) 9. Ali, S., Yue, T., Abreu, R.: When software engineering meets quantum computing. Commun. ACM 65(4), 84–88 (2022) 10. Yao, H., Ma, Y.: An exploration for the software architecture description language of wright. ICIC Express Letters 8, 3481–3487 (2014) 11. Dahi, Z.A., Alba, E.: Metaheuristics on quantum computers: inspiration, simulation and real execution. Futur. Gener. Comput. Syst. 130, 164–180 (2022). [Online]. https://www. sciencedirect.com/science/article/pii/S0167739X21004969 12. Schönberger, M., Franz, M., Scherzinger, S., Mauerer, W.: Peel | pile? cross-framework portability of quantum software. In: QSA@IEEE International Conference on Software Architecture (ICSA) (2022) 13. Krüger, T., Mauerer, W.: Quantum annealing-based software components: an experimental case study with sat solving. Q-SE@ICSE (2020). [Online]. https://arxiv.org/abs/2005.05465 14. Humble, T.S., McCaskey, A., Lyakh, D.I., Gowrishankar, M., Frisch, A., Monz, T.: Quantum computers for high-performance computing. IEEE Micro 41(05), 15–23 (2021) 15. Nielsen, M.A., Chuang, I.L.: Quantum Computation and Quantum Information: 10th Anniversary Edition. Cambridge University, Cambridge (2011) 16. Farhi, E., Goldstone, J., Gutmann, S.: A Quantum Approximate Optimization Algorithm (2014) 17. Peruzzo, A., McClean, J., Shadbolt, P., Yung, M.-H., Zhou, X.-Q., Love, P.J., Aspuru-Guzik, A., O’Brien, J.L.: A variational eigenvalue solver on a photonic quantum processor. Nat. Commun. 5(1), 4213 (2014). [Online]. https://doi.org/10.1038/ncomms5213 18. Tilly, J., Chen, H., Cao, S., Picozzi, D., Setia, K., Li, Y., Grant, E., Wossnig, L., Rungger, I., Booth, G.H., Tennyson, J.: The variational quantum eigensolver: a review of methods and best practices. Phys. Rep. 986, 1–128 (2022). The Variational Quantum Eigensolver: a review of methods and best practices. [Online]. https://www.sciencedirect.com/science/article/ pii/S0370157322003118 19. Franz, M., Wolf, L., Periyasamy, M., Ufrecht, C., Scherer, D., Plinge, A., Mutschler, C., Mauerer, W.: Uncovering instabilities in variational-quantum deep q-networks. J. Frankl. Inst. 8 (2022) 20. Zhou, L., Wang, S.-T., Choi, S., Pichler, H., Lukin, M.D.: Quantum approximate optimization algorithm: performance, mechanism, and implementation on near-term devices. Phys. Rev. X 10, 021067 (2020). [Online]. https://link.aps.org/doi/10.1103/PhysRevX.10.021067 21. Streif, M., Leib, M.: Training the quantum approximate optimization algorithm without access to a quantum processing unit. Quantum Science and Technology 5(3), 034008 (2020). [Online]. https://dx.doi.org/10.1088/2058-9565/ab8c2b 22. Sivarajah, S., Dilkes, S., Cowtan, A., Simmons, W., Edgington, A., Duncan, R.: t—ket: a retargetable compiler for nisq devices. Quantum Science and Technology 6(1), 014003 (2020). [Online]. https://dx.doi.org/10.1088/2058-9565/ab8e92

22

T. Yue et al.

23. Henriet, L., Beguin, L., Signoles, A., Lahaye, T., Browaeys, A., Reymond, G.-O., Jurczak, C.: Quantum computing with neutral atoms. Quantum 4, 327 (2020). [Online]. https://doi.org/10. 22331/q-2020-09-21-327 24. Bass, L., Clements, P., Kazman, R., Software Architecture in Practice. ser. SEI series in software engineering. Addison-Wesley, New York (2003). [Online]. http://books.google.fi/ books?id=mdiIu8Kk1WMC 25. Dasgupta, S., Humble, T.S.: Characterizing the reproducibility of noisy quantum circuits. Entropy 24(2), 244 (2022). [Online]. https://www.mdpi.com/1099-4300/24/2/244 26. Mauerer, W., Scherzinger, S.: 1-2-3 reproducibility for quantum software experiments. In: QSANER@IEEE International Conference on Software Analysis, Evolution and Reengineering (2022) 27. Pashayan, H., Wallman, J.J., Bartlett, S.D.: Estimating outcome probabilities of quantum circuits using quasiprobabilities. Phys. Rev. Lett. 115, 070501 (2015). [Online]. https://link. aps.org/doi/10.1103/PhysRevLett.115.070501 28. Dean, J., Barroso, L.A.: The tail at scale. Commun. ACM 56, 74–80 (2013). [Online]. http:// cacm.acm.org/magazines/2013/2/160173-the-tail-at-scale/fulltext 29. Reuther, A., Byun, C., Arcand, W., Bestor, D., Bergeron, B., Hubbell, M., Jones, M., Michaleas, P., Prout, A., Rosa, A., Kepner, J.: Scalable system scheduling for HPC and big data. J. Parallel Distrib. Comput. 111, 76–92 (2018). [Online]. https://www.sciencedirect.com/ science/article/pii/S0743731517301983 30. Hilliard, R.: Using the uml for architectural description. In: International Conference on the Unified Modeling Language, pp. 32–48. Springer, Berlin (1999) 31. Pérez-Delgado, C.A., Perez-Gonzalez, H.G.: Towards a quantum software modeling language. In: Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops, pp. 442–444 (2020) 32. Ali, S., Yue, T.: Modeling quantum programs: challenges, initial results, and research directions. In: Proceedings of the 1st ACM SIGSOFT International Workshop on Architectures and Paradigms for Engineering Quantum Software, ser. APEQS 2020, pp. 14–21. Association for Computing Machinery, New York (2020). [Online]. https://doi.org/10.1145/3412451.3428499 33. Gemeinhardt, F., Garmendia, A., Wimmer, M.: Towards model-driven quantum software engineering. In: 2021 IEEE/ACM 2nd International Workshop on Quantum Software Engineering (Q-SE), pp. 13–15. IEEE, New York (2021) 34. Feynman, R.P.: Space-time approach to quantum electrodynamics. Phys. Rev. 76, 769–789 (1949). [Online]. https://link.aps.org/doi/10.1103/PhysRev.76.769 35. Coecke, B., Kissinger, A.: Picturing Quantum Processes: A First Course in Quantum Theory and Diagrammatic Reasoning. Cambridge University, Cambridge (2017) 36. Leymann, F.: Towards a pattern language for quantum algorithms. In: International Workshop on Quantum Technology and Optimization Problems, pp. 218–230. Springer, Berlin (2019) 37. Weigold, M., Barzen, J., Leymann, F., Salm, M.: Encoding patterns for quantum algorithms. IET Quantum Commun. 2(4), 141–152 (2021) 38. Zhao, P., Zhao, J., Ma, L.: Identifying bug patterns in quantum programs. In: 2021 IEEE/ACM 2nd International Workshop on Quantum Software Engineering (Q-SE), pp. 16–21. IEEE, New York (2021) 39. Gheorghe-Pop, I.-D., Tcholtchev, N., Ritter, T., Hauswirth, M.: Quantum devops: towards reliable and applicable nisq quantum computing. In: 2020 IEEE Globecom Workshops (GC Wkshps, pp. 1–6. IEEE, New York (2020) 40. Wang, X., Arcaini, P., Yue, T., Ali, S.: Generating failing test suites for quantum programs with search. In: O’Reilly, U.-M., Devroey, X. (eds.) Search-Based Software Engineering, pp. 9–25. Springer International Publishing, Cham (2021)

Challenges and Opportunities in Quantum Software Architecture

23

41. Ali, S., Arcaini, P., Wang, X., Yue, T.: Assessing the effectiveness of input and output coverage criteria for testing quantum programs. In: 2021 14th IEEE Conference on Software Testing, Verification and Validation (ICST), pp. 13–23 (2021) 42. Wang, X., Arcaini, P., Yue, T., Ali, S.: Application of combinatorial testing to quantum programs. In: 2021 IEEE 21st International Conference on Software Quality, Reliability and Security (QRS), pp. 179–188. IEEE, New York (2021) 43. Mendiluze, E., Ali, S., Arcaini, P., Yue, T.: Muskit: a mutation analysis tool for quantum software testing. In: The 36th IEEE/ACM International Conference on Automated Software Engineering, Tool Demonstration. IEEE/ACM, New York (2021) 44. Svore, K., Cross, A., Aho, A., Chuang, I., Markov, I.: Toward a software architecture for quantum computing design tools. In: Proceedings of the 2nd International Workshop on Quantum Programming Languages (QPL), pp. 145–162 (2004) 45. Moguel, E., Rojo, J., Valencia, D., Berrocal, J., Garcia-Alonso, J., Murillo, J.M.: Quantum service-oriented computing: current landscape and challenges. Softw. Qual. J. 30(4), 983–1002 (2022)

Software Architectures for AI Systems: State of Practice and Challenges Ian Gorton, Foutse Khomh, Valentina Lenarduzzi, Claudio Menghi, and Dumitru Roman

Abstract Intelligent, AI software systems must process an ever-growing amount of data during operations. Engineering such systems is challenging as it requires integrating standard software components with data-intensive, distributed software platforms. Developing these systems requires close and ongoing collaborations between data scientists, who provide the domain knowledge, and software architects and engineers, who operationalize, deploy, and evolve the system. This chapter describes the state of practice and future research areas for AI and AI-based software system architectures. We identify and reflect on the fundamental engineering challenges for developing these systems, focusing on data collection, integration, inference, and continuous model updates and validation. From these challenges, we derive areas of future research for the software architecture community. These challenges include analyzing the observability of the AI system, identifying uncer-

I. Gorton () Khoury College of Computer Sciences, Northeastern University, Seattle, WA, USA e-mail: [email protected] F. Khomh SWAT Lab., DGIGL, Polytechnique Montreal, 2500 Chemin de Polytechnique, Montreal, QC, Canada e-mail: [email protected] V. Lenarduzzi M3S Group, University of Oulu, Oulu, Finland e-mail: [email protected] C. Menghi Dipartimento di Ingegneria gestionale, dell’informazione e della produzione, University of Bergamo, Bergamo, Italy McMaster University, Hamilton, ON, Canada e-mail: [email protected] D. Roman SINTEF Digital, SINTEF AS, Forskningsveien 1, Oslo, Norway e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 P. Pelliccione et al. (eds.), Software Architecture, https://doi.org/10.1007/978-3-031-36847-9_2

25

26

I. Gorton et al.

tainties and change management strategies, and developing new tools to support the architecture development process of the AI systems.

Introduction The last decade, fueled by advances in hardware, software, and algorithms, has seen an explosion in the deployment of AI systems [10, 16, 22]. Hardware advances such as graphics processing units (GPUs) and tensor processing units (TPUs) have made it feasible to train models on massive data collections and simplified machine learning model development. Open-source software frameworks including TensorFlow [26], PyTorch [20], and Apache Spark [3] have made it feasible to rapidly deploy and evolve intelligent systems at Internet scale. The speed of progress has been truly remarkable, and it is likely to continue in the next decade. These intelligent systems pervade all aspects of business and society. Internet search, recommendation engines, advertisement placement, driving directions, and code completion engines are simply a few of the thousands of applications that incorporate machine learning algorithms to improve service behavior [13]. For example, Booking [5] has deployed more than 150 machine learning-based services encompassing a broad range of recommendation and personalization capabilities [6]. Large-scale machine learning is fundamental to the prediction of home value at multiple real estate sites including Zillow [31, 32]. Designing, building, deploying, and evolving the software architecture of AI systems creates numerous challenges for the broad software engineering community [14, 30]. At one extreme, major technology companies such as Google [11], Meta [17], Uber [28], and Microsoft [18] are able to invest in custom, comprehensive software infrastructures that support all stages of model development and deployment. At the other extreme, organizations design and compose machine learning software architectures, commonly called pipelines, in an ad hoc manner. They leverage both home-grown and off-the-shelf components to realize application functional and quality requirements. The resulting software architectures compose machine learning models with various software components to effectively train and deploy the system. This is depicted in Fig. 1, as described below: • Application users, developers, and data scientists need interfaces—UIs and APIs—to interact with the system. • The model requires data for training and updates and features to enrich user requests for inference. • A software architecture composes the various data and processing components and augments the application with facilities for monitoring, ongoing validation, and run-time management and recovery. This includes support for heterogeneous and distributed hardware platforms.

Software Architectures for AI Systems: State of Practice and Challenges Fig. 1 Machine learning model as part of an application architecture

Application/User Interface

Data

27

Developer/Data Scientist Interface

Machine Learning Model

Features

Component integration, coordination, monitoring

In fact, model training and development in reality forms a relatively small fraction of the time spent designing and deploying intelligent systems [9]. Feature selection, training, and validation are complex, iterative, and creative activities performed by data scientists. However, at scale, data discovery, integration, cleaning, preprocessing, and deployment to meet quality attribute requirements often dwarf the effort of model development. This is where the impact of software architecture can be a major influence on high-priority quality requirements such as scalability, hardware heterogeneity, performance, interoperability, and interpretability [23]. In this chapter, we describe some of the software architecture challenges of building AI applications. We identified the challenges during brainstorming sessions held at the ICSA-Lite working conference: an in-person event associated with the 2022 edition of the International Conference on Software Architecture (ICSA) [1]. The brainstorming sessions involved five workshop participants (the authors of this book chapter) and took approximately 2 hours each. The brainstorming sessions were organized as follows: (1) Challenges proposal: each author could propose new challenges based on their experience. For example, a challenge proposed by one of the authors concerns the “composition of different data processing pipelines”. (2) Reflections: the group discussed and reflected on the challenge. (3) Creation of the mind map: the authors grouped the challenges that refer to similar aspects into high-level categories. Our chapter is organized as follows. Section “State of Practice” presents the state of practice, section “Challenges” summarizes and discusses a set of selected architectural challenges for AI systems, section “Research Directions” suggests and discusses future research directions, and section “Conclusion” presents our conclusions.

State of Practice To provide the reader with sufficient context to understand our research challenges, this section briefly summarizes the state of practice for developing AI software

28

I. Gorton et al.

Fig. 2 A reference ML/AI pipeline architecture

systems. The interested reader can refer to similar research agendas or surveys for in-depth discussions on the state of practice (e.g., [7, 8]). An AI-enabled software system incorporates AI components to provide intelligent capabilities that satisfy one or more business requirements. These intelligent systems must process data both for the training of AI models and subsequently to perform inferences on operational data. The resulting systems are often referred to as machine learning or AI pipelines, as they comprise multiple stages to process raw data, train models, and deploy and monitor intelligent components in production environments. A reference ML/AI pipeline showing the various stages of processing is depicted in Fig. 2. Increased data availability has been fundamental to the growing success of machine learning at scale. More and higher-quality data enables richer models to be constructed and more useful predictions to be made. Increased data volumes for training enable models to find more patterns in the data. As patterns in data change over time, models can be retrained to incorporate these changes without data scientists explicitly requiring to understand these pattern changes. More data does not always improve model performance however. Outdated or irrelevant data, or data that is incorrectly labeled, can degrade model performance. In summary, the importance of data in modern intelligent systems cannot be understated.

Key Roles and Activities Broadly, there are several roles involved in building AI systems: 1. Software architects: This role is responsible for designing the application, including traditional software components as well as AI components, and the communication between the components. Quality attributes such as performance and scalability are key considerations for software architects to address [24].

Software Architectures for AI Systems: State of Practice and Challenges

29

2. Data scientists/engineers: Select, train, and validate models based on appropriate proprietary and public data sets. This role is responsible for initial experimentation [15] in the problem domain and the subsequent design (data scientists) and the implementation (ML engineers) of AI components. It is the responsibility of a data scientist to ensure the inference stage produces acceptable results for the application domain. 3. Software engineers: Design, develop, deploy, and evolve the data processing pipeline that operationalizes the training of models and their deployment in production systems. The software engineers are responsible for the implementation of the various components, focusing on non-AI components and the overall integration with AI components. 4. DataOps expert: This role is responsible for the deployment process of the implemented application (with a focus on data and models) and monitoring of the application. 5. Application users: The end users of the developed application. 6. Domain expert: A person with expertise in the domain targeted by the software application. Software architects, as part of the software engineering role, are crucial to defining the qualities that the intelligent system must exhibit. Once model requirements are defined, the architect must work with data scientists and the engineering teams in three broad areas [2], namely, data engineering, model engineering, and model deployment, monitoring, and evolution, described in details in the following section.

Conceptual Architecture and the Overall Process for Developing AI Systems Throughout the development process, the abovementioned actors involved in the design, development, and deployment process interact in order to ensure a reliable implementation of the AI system. Figure 3 depicts the main artifacts and processes in the development of AI systems. As in traditional software engineering, the development of AI-driven applications starts with a set of requirements (functional and non-functional) for the application, typically described in natural language documents (possibly with a predefined structure). The requirements are used by the software architect as input to the software architecture which typically consists of the design of the application (components and their interactions), documented using various types of diagrams (e.g., components diagrams, interaction diagrams, etc.). In AI-driven applications, it is common to differentiate between AI components (typically ML models) and non-AI components (e.g., user interfaces, databases, etc.). Detailed designs of the components prescribe specific functions needed for the implementation. Software engineers are involved in the implementation of the components, usually, non-AI components, whereas data scientists/engineers are involved in the design and implementation of the AI components.

30

I. Gorton et al.

Design time

Run time Operational data

Software components (non-AI) Software architect DataOps Architecture design

Requirements

Deployed Application

Application Software engineer

Application User

Software components (AI models) Model refinement and application re-deployment

Model development and refinement

Prepare

Build

Data scientist/ engineer Raw data

Training data labeling

Data transformation and cleaning

Feature engineering

Algorithms

Train and Tune

Training

Tuning

Model refinement

Fig. 3 Conceptual architecture for AI systems and the overall process for developing such systems

The process of designing and implementing AI components (e.g., prediction models, classification models, etc.) is far from trivial and involves a number of aspects each with their corresponding components. The design and development of an AI component/model begins with data engineering activities. First, in the Prepare phase of the process from Fig. 3, data scientists work with the engineering teams to identify suitable data sets for model training and validation. Obtaining training data sets commonly requires the integration of multiple potentially massive, distributed data sets. These data sets represent raw data that is fed into the data preparation component whose responsibility is to ensure that data is labeled (in case the model uses supervised learning techniques) and is of high quality and that relevant features are managed in a suitable way (e.g., through use of feature stores). Typical functions in the data preparation process include data labeling, data cleaning/transformation, and feature engineering (managing/selecting features) and are usually carried out by data engineers. The role of the Build phase from Fig. 3 is to actually build the AI model. Data scientists may reuse existing ML algorithms, extend algorithms, or build new ones to meet application requirements. Finally, the Train and Tune phase of the process from Fig. 3 is training and tuning the actual model, and its purpose is to improve the accuracy and quality of the model. The model development process should be seen as an iterative process in which the model is continuously improved by iteratively going through the process of preparing data, building the model, and fine-tuning the model. These represent model engineering activities to train a model. The architect plays an essential role in the design and development process of AI components, ensuring the choice of suitable approaches to the full data pipeline (e.g., design of the data pipelines, ensuring parallel training data at scale, accounting for latency constraints, etc.).

Software Architectures for AI Systems: State of Practice and Challenges

31

The resulting AI and non-AI components are integrated with other generic components (e.g., access control) to create the application which is then deployed into production by the DataOps. The abovementioned process represents the designtime process for designing the application. At run-time, in production environments, AI-driven applications receive real data that is fed into the developed application to generate the desired outcomes used by the users of the application. A peculiarity of AI-driven applications is that, as new data is fed into the run-time system, through proper monitoring and evaluation of the outputs of the application, the AI models can be improved/refined, which in turn can trigger the redeployment of the application containing the improved models and its architecture. Models are typically retrained and redeployed at some frequency that is application dependent. For example, different software components’ (for AI and non-AI) functionalities might be plugged in and removed from the running system. A news article recommendation model needs retraining and deploying on a potentially daily basis to incorporate the latest news articles. A predictive typing model will need retraining and redeploying much less frequently as its knowledge domain and inputs are relatively stable. In either case, the results may drift and become of lower quality if the characteristics of the real-world data that the model processes change over time. To react to this change, the application may reconfigure the software architecture, for example, by changing the AI software component with another component that can account for lower-quality data. Thus, the development of AI-driven applications should be seen as an iterative process in which the quality of the AI models improves over time. Such activities represent model deployment, monitoring, and evolution activities. The whole process of data and model engineering as well as model deployment, monitoring, and evolution requires a high level of observability at the architectural level, and the architect plays an essential role. Furthermore, the complexity of AI systems inevitably requires the architect to consider and resolve trade-offs. Some examples of these trade-offs are as follows: • Latency versus throughput: Research has shown that many use cases are extremely latency sensitive. Increased latencies, even by a hundred milliseconds, can cause users to move to other products or diminish the yield (in terms of business benefits) of predictions. • Cost versus prediction quality: Models that are trained with more and more data can yield higher-quality predictions. This generally comes at the cost of computational time, requiring more powerful (and expensive) hardware for training and deployment, and longer latencies. For some applications, a 0.1% improvement in prediction quality can at scale be significant in business returns, bringing in new orders or bookings. However, in, for example, cloud-based language translation service, improving translation from 97.4% to 97.5% might have considerable training and deployment costs and have no discernible user benefit. • Fairness versus costs: Many well-known examples exist of unfair predictions based on machine learning algorithms. Prominent ones include loan application

32

I. Gorton et al.

denials and facial recognition. If unfair predictions manifest in only a small percentage of predictions, and the costs of retraining the model to fix these are high, organizations may not be motivated to remove this inherent unfairness.

Example: Development of a Recommender System A typical example of an AI system is a recommender system (e.g., an e-commerce website capable of recommending products for users with similar interests). In the following, we envision its development. The design of the system starts with the identification of requirements, both for non-AI components (e.g., UI for the recommender system, access to the application, response time, etc.) and AI components (e.g., the accuracy of recommendations, etc.). The software architect translates such requirements into functionalities for the components (e.g., UI features, database components, etc.). Data engineers start with raw data (e.g., users, products, past purchases, etc.) and prepare the data for training (they define, e.g., the situations in which a product should be recommended). Various ML algorithms (e.g., decision tree models) could be selected, and then the selected model is trained and refined based on the provided data. Once the data scientist is satisfied with the quality of the model, it is then integrated by software engineers with the rest of the components into a recommender system application. The DataOps deploys the application in production. As new product recommendations are generated at run-time and more data is collected by users’ purchases, these data can further be used to refine/improve the recommendation model, which can then be redeployed for the user to have access to better product recommendations.

Challenges Architecting AI systems introduces many software engineering challenges. These become especially problematic as the scale of the systems grows. Figure 4 presents a mind map representing the challenges we identified during the workshop. Related challenges are grouped within the same category. For example, the “composition of different data processing pipelines” and the “maintenance of the consistency of the training and production pipelines” are grouped into the “design” category, as they both are concerned with the conceptualization and design of AI systems. Notice that we did not compare these challenges with existing challenges from the literature nor verified their accuracy with industrial partners. However, they can be a helpful starting point for researchers and practitioners working in the field. The center node of the mind map diagram (“challenges”) is connected to the categories that group the different challenges. An edge connects each category to the corresponding challenges. In the following, we describe the categories and, for each category, give details of the corresponding challenges.

Software Architectures for AI Systems: State of Practice and Challenges

33

Pipeline Pipeline Composition Monitoring Pipeline Consistency

Pipeline Definition

Repairing Predictive Maintenance & Evolution

Data Provenance Selection

Design

Data Consistency Maintainance & Evolution

Challenges

Data Management Data Absence

Pipeline Reorganization

Data Security

Efficiency

Supporting new Data

Enable Adaptation

Enable Resource Scalability

Ensure Decentralize Computation Performance

Fig. 4 Architecting AI systems: challenges

Design This refers to challenges related to the conceptualization and design of the architecture for AI systems. • Pipeline Definition. This concerns the mapping of application requirements to the software pipeline that defines how the components process the data. It includes selecting the components for data processing and communications, including both software and hardware platforms. • Pipeline Composition. Defining the connections between different pipeline elements is the challenge of pipeline composition. This task includes investigating and specifying how data flows from a source to a destination pipeline, typically synchronously or asynchronously. It also requires intermediate storage elements to be designed, so that the necessary state and processed data can be shared across pipeline elements.

34

I. Gorton et al.

• Pipeline Monitoring. Monitoring, also known as observability, refers to strategies and software elements used to ensure that the pipeline elements meet application requirements at run-time. In general, this requires capturing and analyzing data that quantitatively characterizes the pipeline’s dynamic behavior. There are many aspects to monitoring. It may include, for example, evaluating the incoming data to assess quality and measuring the quality of the results produced by the pipeline to ensure that its outputs are not drifting from expected ranges. Data monitoring is also fundamental for predicting and preventing incidents and reducing costs. Finally, since the rate of change that impacts AI and nonAI components of an AI system can vary significantly (with AI components degrading at a different rate than other components), it is important to include monitoring components for both the system and the AI components. We also need to decouple routine changes from the highly coupled data changes and build selfadaptation capabilities in the system, to ease the deployment of retrained models. • Pipeline Consistency. Consistency refers to the process of maintaining consistent training and production pipelines. The training pipeline automatically creates a model based on identified training data. The production pipeline uses the model to generate predictions using the application’s run-time data. Consistency concerns the problem of maintaining the training and production pipelines in sync, such that the model trained on data is continually representative of the inputs the application pipeline will ingest. Data Management Intelligent AI systems create multiple challenges related to the collection and usage of data for training, validation, and inference. • Data Provenance. Provenance refers to the capture, identification, and management of the data sets and hyperparameter settings used for training and validation of the system’s inference capabilities. Provenance has many purposes, for example, enabling a previous version of a model to be reconstructed to investigate anomalous behavior. • Data Consistency. Consistency concerns the verification of whether the data are correctly and consistently formatted. • Data Absence. Some data might be unavailable for a specific time interval. Developing techniques enabling the system to operate even when access to specific data is temporarily impossible is challenging. • Data Security. Dealing with confidential data in ML pipelines is complex. It can include tasks such as making confidential data anonymous and replacing it with representative values for training purposes. Efficiency AI pipelines need to process data with performance and scalability characteristics that meet evolving application requirements. Efficient and timely processing creates challenges related to the ability of the software architecture to ensure that the applications achieve their performance goals while ideally minimizing compute, storage, and energy costs.

Software Architectures for AI Systems: State of Practice and Challenges

35

• Enable Adaptation. Pipeline requirements inevitably evolve as data sources, algorithms, and capabilities change over time. Adaptation refers to the capability of the architecture design to accommodate changes and dynamically adapt to different environmental conditions. These include new compute and data platforms and circumstances such as pipeline component and connector failures. • Ensure Performance. All AI applications have latency requirements. These may concern training, for example, a news article recommendation system needs to perform training with new data daily. Latency for inference capabilities is often equally important in systems that, for example, perform fraud detection on transactions. Central to this concern is the software architecture of the deployed pipeline, as well as specific algorithm selection. Meeting performance requirement requires ongoing system monitoring to detect and address issues in a timely fashion. • Decentralize Computation. Modern edge devices such as cell phones and workstations possess the computation power to perform local inference based on a copy of a global model. Similarly, federated learning exploits edge devices to perform training on a local copy of the model. Updated copies of the model are then merged and distributed to participating nodes to perform predictions. Both approaches require the software architecture to be designed to coordinate the model and corresponding updates across all nodes participating in the system. As models can be large, executing them efficiently to have the required low latency on constrained edge devices can be challenging. For example, a predictive typing application is only useful if it can provide suggestions before the user has had the time to type the following words. • Enable Resource Scalability. The software architecture should enable increasing or decreasing the resources available to the AI application on demand to match the application load. This property is known as elasticity. Elasticity requires the application pipeline to be composed based on design principles that enable scalability [12], for example, stateless services, load balancing, and distributed storage. Maintenance and Evolution Intelligent AI applications need to evolve to offer new features and support growing workloads. Challenges related to the enhancement and modification of systems are described below. • Supporting New Data. AI applications need to draw on various data sources to build powerful predictive capabilities. These inevitably change over time as new higher-quality resources become available and new features are incorporated into the model. The software architecture should be able to accommodate such changes with minimum effort and cost. • Pipeline Reorganization. It is common for pipeline components to be divided or merged to satisfy quality requirements. An application may also need to compose new components into the pipeline or change the pipeline topology. The software architecture and platforms should facilitate such changes with minimal effort and no disruption to operations.

36

I. Gorton et al.

• Predictive Maintenance and Evolution. This refers to the challenge of predicting the maintenance and evolution activities to be performed by analyzing runtime monitoring data and model outputs to support decision-making. Predictive maintenance helps engineers understand when and how to maintain and evolve the AI application. • Repairing. When hardware and software failures occur, restoring the application state and returning to operational status is a major challenge. The software architecture must be designed with features such as check-pointing and restarts to ensure that the system state can be accurately recovered. Architects need to define effective procedures that limit the consequences of problems and efficiently repair the software architecture after a failure occurs. The challenges identified during the ICSA-Lite working conference led to the roadmap presented in the next section.

Research Directions Building on the challenges presented above, this section summarizes some research directions identified in our breakout group. These research directions do not cover all the challenges from section “Challenges”. We focused on research challenges that stem from the data-driven nature of these systems. In particular, we focused on (i) observability, (ii) uncertainty and change management, and (iii) the need for tools to support the architecture development process of AI systems. • Observability Observability in AI systems is critical to monitor how changes in data distributions affect different quality attributes of the system. Although some frameworks like TFX [27] or SageMaker [21] support pipeline monitoring by computing specified metrics over distributions of ML features over time, they often produce false positives [4, 29] and report changes in distributions prematurely. To be able to proactively react to changes occurring in the behavior of ML pipelines and AI systems, we need mechanisms to collect and share information in real time. This requires developing novel reference architectures (e.g., [25]). The research community should invent novel architecture patterns that support observability of AI systems. We also need to develop monitoring techniques that impose minimum overhead on the monitored systems (in particular, those deployed on edge devices) and minimum data communication between edge devices and centralized servers. Such techniques will monitor both the behavior of the AI system and its operational environment, to ensure the health of the code, data, and model and provide feedback/alerts in cases of failures, anomalies, or data/model drifts. • Uncertainties and change management We need efficient techniques for identifying uncertainties in AI systems with provable guarantees. Because these uncertainties often require adapting the behavior of the system, we also need to develop efficient techniques and architecture patterns to support adaptations

Software Architectures for AI Systems: State of Practice and Challenges

37

and repairs at the algorithmic, code, model, and component levels, to ensure that the systems maintain their level of quality. The uncertainty, availability, and scalability of data should be key architecture drivers. This is important because software update cycles may not adequately address data changes and their impact. Novel change management and impact analysis techniques are needed. We also need novel architectural patterns that (a) decouple data stores and their interactions with other components of the systems, in order to isolate changes and updates to as few locations as possible. This is important to, for example, ease the implementation of privacy protections and (b) allow coordinating the model and corresponding updates across all nodes participating in the system. • Tools to support the architecture development process of AI systems One of the expected outputs of a software architecture process is a set of artifacts that captures the system’s architecture in a way that eases communications between different stakeholders. A standard set of processes and practices to support the architecture development process of AI systems is still missing [19]. More research is needed to understand how to better manage and document design decisions that consider the data quality, data volume, probabilistic nature of the learning algorithms, etc., as well as the constraints that they impose on the software systems. We also need to establish best practices for organizing the development teams such that architecture decisions can be communicated smoothly between the architect, data scientists, ML engineers, and DevOps engineers. Finally, we need to develop novel tools and techniques that provide rich visualizations of the data and models generated throughout the development life cycle and their interactions with non-AI entities of the system. These tools are important to help architects communicate design decisions more effectively. In summary, strong data dependency and the increased uncertainty are the two key factors we see that prevent traditional software architecture techniques from working effectively on AI systems.

Conclusion In this chapter, we reported on the software architecture challenges of building AI systems, identified in our breakout group during the ICSA-Lite working conference. From these challenges, we derive areas of future research for the software architecture community. The challenges and areas of future research reported in this chapter are useful references for researchers and practitioners interested in AI software architectures. They can also stimulate reflections from other researchers and practitioners. Acknowledgments Foutse Khomh acknowledges the support of the Canadian Institute for Advanced Research (CIFAR), the Fonds de Recherche du Quebec (FRQ), and NSERC. Claudio Menghi acknowledges the support of the Natural Sciences and Engineering Research Council of Canada (NSERC) [funding reference numbers sRGPIN-2022-04622 and DGECR-2022-0040].

38

I. Gorton et al.

Dumitru Roman acknowledges the support of the projects DataCloud (H2020 101016835), enRichMyData (HE 101070284), Graph-Massivizer (HE 101093202), and BigDataMine (NFR 309691).

References 1. International Conference on Software Architecture ICSA 2022 (2022). https://doi.org/10.1109/ ICSA53651.2022 2. Amershi, S., Begel, A., Bird, C., DeLine, R., Gall, H., Kamar, E., Nagappan, N., Nushi, B., Zimmermann, T.: Software engineering for machine learning: a case study. In: International Conference on Software Engineering: Software Engineering in Practice, pp. 291–300. IEEE/ACM, New York (2019). https://doi.org/10.1109/ICSE-SEIP.2019.00042 3. Apache Spark. https://spark.apache.org/ 4. Baylor, D., Breck, E., Cheng, H.T., Fiedel, N., Foo, C.Y., Haque, Z., Haykal, S., Ispir, M., Jain, V., Koc, L., et al.: TFX: A tensorflow-based production-scale machine learning platform. In: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1387–1395 (2017) 5. Booking: www.booking.com 6. Booking: 150 Successful Machine Learning Models: 6 Lessons Learned at Booking.com. https://blog.acolyer.org/2019/10/07/150-successful-machine-learning-models/ 7. Borges, A.F., Laurindo, F.J., Spínola, M.M., Gonçalves, R.F., Mattos, C.A.: The strategic use of artificial intelligence in the digital era: Systematic literature review and future research directions. Int. J. Inf. Manag. 57, 102225 (2021) 8. Collins, C., Dennehy, D., Conboy, K., Mikalef, P.: Artificial intelligence in information systems research: a systematic literature review and research agenda. Int. J. Inf. Manag. 60, 102383 (2021) 9. Domingos, P.: A few useful things to know about machine learning. Commun. ACM 55(10), 78–87 (2012) 10. Enholm, I.M., Papagiannidis, E., Mikalef, P., Krogstie, J.: Artificial intelligence and business value: a literature review. Inf. Syst. Front. 24(5), 1709–1734 (2022) 11. Google: www.google.com 12. Gorton, I.: Foundations of Scalable Systems. O’Reilly Media, California (2022) 13. Jordan, M.I., Mitchell, T.M.: Machine learning: trends, perspectives, and prospects. Science 349(6245), 255–260 (2015) 14. Lewis, G.A., Ozkaya, I., Xu, X.: Software architecture challenges for ML systems. In: International Conference on Software Maintenance and Evolution, pp. 634–638. IEEE, New York (2021) 15. Lwakatare, L.E., Raj, A., Bosch, J., Olsson, H.H., Crnkovic, I.: A taxonomy of software engineering challenges for machine learning systems: an empirical investigation. In: Agile Processes in Software Engineering and Extreme Programming, pp. 227–243. Springer, New York (2019) 16. Meske, C., Bunde, E., Schneider, J., Gersch, M.: Explainable artificial intelligence: objectives, stakeholders, and future research opportunities. Inf. Syst. Manag. 39(1), 53–63 (2022) 17. Meta: www.meta.com 18. Microsoft: www.microsoft.com

Software Architectures for AI Systems: State of Practice and Challenges

39

19. Muccini, H., Vaidhyanathan, K.: Software Architecture for ML-based Systems: What Exists and What Lies Ahead. In: Workshop on AI Engineering—Software Engineering for AI, pp. 121–128. IEEE/ACM, New York (2021) 20. Pytorch: https://pytorch.org/ 21. Sagemaker: aws.amazon.com/sagemaker/ 22. Schneider, J., Abraham, R., Meske, C., Vom Brocke, J.: Artificial intelligence governance for businesses. In: Information Systems Management, pp. 1–21 (2022) 23. Serban, A., Visser, J.: Adapting software architectures to machine learning challenges. In: International Conference on Software Analysis, Evolution and Reengineering, pp. 152–163. IEEE, New York (2022). https://doi.org/10.1109/SANER53432.2022.00029 24. Serban, A., Visser, J.: Adapting software architectures to machine learning challenges. In: International Conference on Software Analysis, Evolution and Reengineering, pp. 152–163. IEEE, New York (2022) 25. Shankar, S., Parameswaran, A.G.: Towards observability for machine learning pipelines. ArXiv abs/2108.13557 (2022) 26. TensorFlow: www.tensorflow.org 27. TFX: www.tensorflow.org/tfx 28. Uber: www.uber.com 29. Whang, S.E., Lee, J.G.: Data collection and quality challenges for deep learning. Proc. VLDB Endow. 13(12), 3429–3432 (2020) 30. Woods, E.: Software architecture in a changing world. IEEE Software 33(6), 94–97 (2016) 31. Zillow: https://www.zillow.com/ 32. Zillow: Introduction to Recommendations at Zillow. https://www.zillow.com/tech/ introduction-recommendations-zillow/

Architecting and Engineering Value-Based Ecosystems Patrizio Pelliccione, Barbora Buhnova, Sebastian Gottschalk, Ingo Weber, and Gregor Engels

Abstract As the digitalization of the human world progresses, people, systems, machines, and other entities are forming dynamic and unpredictable ecosystems. Such ecosystems emerge from the mutual interaction of constituent and individual systems, which might be driven by collaborative, competitive, or even malicious goals. In these ecosystems, enhanced by artificial intelligence, humans might find themselves unprotected in their interaction with the digital world, not only in terms of their safety but also in terms of other societal values, such as fairness and privacy. The future of a digital society, where individual systems are incentivized mainly by profit, is unlikely to serve humanity best. As a counter-point, this chapter invites the research community to direct its focus toward methods and techniques of architecting and engineering digital ecosystems where values such as safety, fairness, ethics, and privacy are built in by design. Specifically, we illustrate the problem and related challenges, and we outline the steps that community members can take to lead society toward a better path.

P. Pelliccione () Gran Sasso Science Institute (GSSI), L’Aquila, Italy e-mail: [email protected] B. Buhnova Faculty of Informatics, Masaryk University, Brno, Czech Republic e-mail: [email protected] S. Gottschalk Paderborn University, Paderborn, Germany e-mail: [email protected] I. Weber Technical University of Munich, School of CIT, and Fraunhofer Gesellschaft, Munich, Germany e-mail: [email protected] G. Engels Paderborn University, Paderborn, Germany e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 P. Pelliccione et al. (eds.), Software Architecture, https://doi.org/10.1007/978-3-031-36847-9_3

41

42

P. Pelliccione et al.

Introduction The scope of today’s software systems is expanding beyond our control, both in terms of directing their actions and predicting the outcomes. The complexity of the operating context of each software system, which is only partially predictable, generally uncontrollable, and changing dynamically during runtime, constitutes the main driver toward transitioning from standalone system and systems-of-systems architectures to dynamic software ecosystems [23]. Collaborative Dynamic Ecosystems Within dynamic software ecosystems, the individual systems and other entities are engaged in mutual interactions, which might be driven by collaborative, competitive, or even malicious goals [26]. This pushes the interacting entities to distinguish among the systems they do and do not trust, forming friendships and social relationships to support their interests and needs [58]. With the growing digitalization and expansion of concepts such as Industry 4.0, the ecosystems have effectively left cyberspace and entered the cyber-physical world, engaging in physical interaction with humans and exposing them even more to the risks of technology. Dynamic ecosystems nowadays consist of software systems, possibly mobile and largely autonomous things and machines, and humans, all in an interplay with uneven stakes. Individual and Social Values in Dynamic Ecosystems In a digital society, where the relationship between humans and machines is uneven, whether due to humans feeling unprotected or manipulated, ethical values such as individuality and responsibility are at risk. This is further amplified by the tendency of each ecosystem entity to act in its own interest rather than the interest of the whole. At the European level, the European Group on Ethics in Science and New Technologies recommends rethinking the values around which the digital society is to be structured [33] and highlights the importance of recognizing that a person is worthy of respect in her interaction with autonomous technologies. Also, the EU High-Level Expert Group on AI in [3] identifies a number of requirements that an AI system should satisfy. According to them, besides respecting the law, (i) AI systems should be aligned with agreed ethical principles and values, including privacy, fairness, and human dignity, (ii) the humans should be able to take control when needed, and (iii) these systems should remain trustworthy even when failures occur. Japan is proposing to achieve a human-centered society, called Society 5.0 [50],1 which aims at integrating the cyberspace and the physical space while balancing economic advancement with the resolution of social problems. The need to act has been raised, e.g., by Webb [73] who concludes that tech companies need to move from profit centricity to pursuing human and societal interests. And the same needs to be done for ecosystems in general, including but

1

https://www8.cao.go.jp/cstp/english/society5_0/index.html, accessed 2023-04-14.

Architecting and Engineering Value-Based Ecosystems

43

not limited to the AI-enhanced parts of them, where the risks of local optimization harming the global values are at the highest risk. Empowering Humans and Their Values in Digital Ecosystems It is then time to better balance the forces of humans and digital entities, by promoting social and human values as drivers when engineering and building systems, by protecting and/or empowering humans, by not accepting any more digital entities that expose opaque behaviors, and by shaping technology toward ethical perception and needs of humans, instead of the other way around. The work in [76] highlights the importance of considering human values as first-class entities in software engineering, since neglecting them can also cause devastating economic harm. There is a large body of research within the social sciences studying human values, as testified by the most prominent theory of universal values of Schwartz [62]. This study identifies ten universal values that are recognized by most countries and cultures, i.e., security, conformity, tradition, benevolence, universalism, self-direction, power, achievement, hedonism, and stimulation. These are the universal values, and each individual, organization, or society holds a different subset of these values. The work in [46] identifies 11 ethical principles in 84 AI guidelines [46], i.e., transparency, justice and fairness, non-maleficence, responsibility, privacy, beneficence, freedom and autonomy, trust, sustainability, dignity, and solidarity. Subjective Nature of Social and Human Values It is also important to highlight that social and human values might have a subjective nature. In fact, as highlighted in [35], digital ethics, which aim at studying how technology is shaping and will shape our political, social, and moral existence, is composed of two main components, hard ethics and soft ethics. Hard ethics, which represents values collectively accepted, e.g., defined and enforced by legislation and institutional bodies, is insufficient since it does not cover the preferences of individuals, which are instead covered by soft ethics. Moreover, as we learn from the moral machine experiment of MIT, values collectively accepted are anchored to and depend on specific cultures and/or countries [8]. This also implies that within a valuebased ecosystem, there can be multiple and even conflicting values that should be guaranteed. This might call for a tradeoff and/or negotiation of values among the involved humans and digital entities. Engineering Value-Based Ecosystems In this chapter, we argue that to make the interaction between humans and digital entities beneficial and sustainable, these ecosystems need to have built-in mechanisms that are protecting social and human values in their mutual interactions as well as in individual behaviors. These mechanisms can be various and of different natures, including (i) the mediation of interactions of humans with digital entities to protect and preserve societal values [7, 44], (ii) pricing and/or incentive mechanisms or algorithms to promote socially desirable and ethical behaviors [34], (iii) solutions for optimization problems that take into account fairness criteria [12], (iv) methods for transparency and explainability of behaviors [40], and (v) approaches to build, manage, and maintain trust [4].

44

P. Pelliccione et al.

Example of a Value-Based Ecosystem Value-based dynamic ecosystems can be illustrated on a mobility scenario within a smart city, where humans (i.e., vulnerable road users), vehicles with different importance for the city (e.g., cars, buses, ambulances), and entities of road infrastructure engage in mutual interactions. While some of the interactions follow clear traffic rules, there are many ungoverned situations where altruistic behavior (e.g., informing other road members about a danger they cannot yet see), or a demonstration of solidarity with surrounding entities (e.g., freeing a parking slot for a vehicle transporting a disabled person), can be examples of values supporting overall ecosystem well-being. Scope of Value-Based Ecosystems Today’s software systems might be characterized by two sets of ecosystems, those that are critical from the point of view of, e.g., safety, mission, and business, as well as those that impact our everyday life and are not considered to be critical in any way. These two categories of ecosystems are built in very different ways: systems in the first category are typically controlled by standards and regulations, and the other ones have basically time to market and user satisfaction as main drivers. However, many of the systems belonging to the second category have access to and manage our personal data and might touch social and human values in various ways. Examples of these systems and ecosystems are very easy to find. In fact, AI can be used to, e.g., (i) analyze CVs to suggest the appropriate salary or to automatically identify a short list during recruitment, (ii) suggest suspected persons to police via analysis of images or videos, or (iii) decide whether the bank loans money to you or not. Also, the majority of systems make use of our personal data and/or share them within the ecosystem to which they belong. Data can be collected from our profiles and cookies, but also through the use of microphones and cameras or even through our actions, e.g., when we use digital systems to enter buildings or open doors, to pay for services or products, etc. Moreover, our personal data are often made available to third-party libraries or software, e.g., for image recognition or speech-to-text facilities. Although it is less obvious that ecosystems belonging to the second set are critical, we argue that, when engineering them, there should be a careful analysis to identify the values that should be guaranteed and preserved. The analysis can identify those ecosystems that are value-critical according to some specific values; then, there should be instruments to show confidence that social and human values are seriously taken into account while engineering systems. We believe that when society starts requiring evidence about compliance to social and human values, the values will also become business values; this could help increase the attention of companies to instruments able to provide evidence of compliance to social and human values. Structure of This Chapter The remainder of this chapter is organized as follows: section “State of the Art” describes the state of the art in value-based systems and their engineering. Section “Problem Description” highlights valuebased systems’ challenges and open issues. Section “Value-Based Ecosystems” explains the various facets of value-based ecosystems. Section “Aspects and Architectural Implications of Solution Approaches” discusses aspects and architectural

Architecting and Engineering Value-Based Ecosystems

45

implications of approaches to engineering and managing value-based ecosystems. Section “Research Opportunities” discusses research opportunities, and, finally, Sect. “Summary” summarizes the chapter.

State of the Art Currently, different research disciplines take values into account for the design and development of their software and systems. In the following, we give an overview of the engineering principles connected to human and social values and the application of those values to concrete use cases.

Engineering of Value-Based Systems Researchers developed different concepts of dealing with human and social values in the engineering process in the past. In the following, we provide an overview of the work for using values during the design and runtime of those systems, divided into two main categories. Values During Design Time of a System In this context, value-based software engineering [16] describes the integration of value considerations by all involved stakeholders, like the developers and potential users of the system, over the whole development of the system. This, in turn, often involves situations where a tradeoff between different values needs to be made that might also directly or indirectly affect the involved stakeholders. This concept is also supported by agile development methods like SCRUM [41], which test the value of the system for the user in an iterative manner. In SCRUM, small parts of the system are developed in sprints in a predefined short time range and cross-checked afterward for value creation with the users of the system. To support software engineering, the work in [43] develops a value framework in the developer’s perception of values, their usage in the actual systems, and challenges during the development. Within a case study, they received the results that the organizational culture of the company has a major impact on how values are addressed in the developed software. A special subpart of value-based software engineering is value-based requirements engineering [70], which aims to consider stakeholder values, motivations, and emotions during the requirements engineering process. This, in turn, might uncover conflicts between the goals of different stakeholders. To model the values as nonfunctional requirements of a single system during the process, modelling languages like i* [78] allow the visualization of different non-functional goals related to values. In i*, those goals are structured through hierarchies (e.g., subgoal), and relationships between the goals (e.g., supporting, conflicting) can be modelled with the focus of a single system. Moreover, modelling languages like e3-value [38] support the

46

P. Pelliccione et al.

visualization of a network of collaborating systems. Here, e3-value allows the flexible modelling of a system’s network together with the possibility to calculate the outcome of different scenarios. With this, those systems tend to create value networks where different companies build inter-firm relationships to maximize the value creation, delivery, and capturing for the users [48]. Here, a recent work builds a value network of intelligent autonomous vehicles for the provision of sustainable supply networks [11]. Based on the concept, the authors developed a preliminary simulation tool for the testing of those vehicle networks. Values During Runtime of a System Here, autonomous systems with an additional adaption layer can be used to manage a system using sensors and effectors [47]. That adaption layer, in turn, provides an internal logic to measure the sensors, analyze the measurements, plan corresponding adjustments, and execute them through the effectors. This provides reasoning for certain goals of the companies behind the systems. Those goals can also contain different kinds of types like fulfilling specific purposes or supporting an independent collaboration without restrictions [75]. The fulfillment of values (e.g., fairness, privacy) might be also considered as a purpose to accomplish by those autonomous systems. Moreover, multiple systems can be combined into systems-of-systems in order to foster collaboration between them. Those systems-of-systems can be found in various domains like connected cars, air defense networks, or healthcare applications [55] where their collaboration increases the combined value for the user. To support the development of those collections, the work in [51] proposes a clear definition of systems-of-systems, a limited taxonomy of features, and a set of basic architectural principles to assist their design. Moreover, those collections might also provide an adaption mechanism to support autonomous and self-adapting behavior toward certain goals, including potential values. While, in recent years, different kinds of those systems-of-systems have been studied, they often operate on a much lower scale than traditional systems-of-systems [57]. Here, an existing work [77] provides a reconfiguration of these systems based on internal system changes or external environment changes. For that, the authors present a novel service-oriented approach with a decentralized reconfiguration.

Application to Value-Based Systems Researchers applied the usage of concrete values to different kinds of systems in the past. In the following, we provide an overview of work in the areas of digital systems and digital ecosystems. Ethics and Privacy in Digital Systems In this area, the ethics value can be used in two distinct broad senses: descriptive and normative. In its descriptive sense, ethics refers to certain codes of conduct actually put forward by a particular group or society. In this respect, a descriptivist has no moral obligation to follow the norms endorsed by that group or society. In the normative sense, instead, ethics refers to a

Architecting and Engineering Value-Based Ecosystems

47

code of conduct that, given specified conditions, should be pursued by all rational persons. For this reason, a normativist would believe that ethics essentially consists in following, without exception, rules or norms of conduct by means of which one can judge all possible actions. In the context of digital society, an example of a normative framework is represented by machine ethics [20, 54]. When focusing on autonomous cars, there is no general consensus on which ethical principles need to be embedded, and how, in the control software of the vehicle, e.g., [27, 37, 42]. In 2016, the German Federal Ministry of Transport and Digital Infrastructures appointed an ethical committee that produced a recommendation report resulting in 20 ethical rules for automated and connected vehicular traffic [1]: rules 4 and 6 mention the ethical principle of safeguarding the freedom of individuals to take responsible decisions and the need to balance it with the freedom and safety of the others; this appears to be an attempt of defining a hard ethics setting [35]. Other works also address various hard ethics problems on human interactions with AI, robotic, and bionic systems [6, 15, 61, 69], concerning the analysis of conflicts between competing normative ethics approaches and the development of public ethical policies to defuse those conflicts. Various approaches have been proposed to describe and implement ethics [72], including (i) logic formalisms like deontic logic and logic programming [13, 20, 56], (ii) those that cast general ethical categories (e.g., deontology and consequentialism) into elements of a mathematical problem (e.g., constraints and costs in an optimization problem) [71], and (iii) experimental philosophy approaches to collect user preferences and build/refine ethical models out of them [9, 68]. However, the characterization and specification of soft ethics and, consequently, its seamless and consistent combination with hard ethics still require further investigation. The work in [7] proposes a software personalized exoskeleton that enhances and protects human beings by mediating their interactions with the digital world according to their own ethics of actions and privacy of data. In [74], value-based methods are mentioned in the context of digital money, to incentivize spending of money that is in line with societal values, e.g., on fair-trade organic produce. From a general philosophical perspective, privacy is a dimension of ethics [32]. User privacy focuses on (personal) data on which we want to exercise direct and unconditional control concerning its diffusion and disclosure. Examples of such information are physical characteristics, sexual orientation, economic status, health conditions, and the like (see GDPR [2]). Ethics instead concerns actions of digital and autonomous entities. Considering the privacy dimension, a descriptive view states what is in fact protected as private, while a normative view takes privacy to be a value that should be protected. Data are traditionally considered passive entities that are created and manipulated by heterogeneous hardware and software devices. The logic that controls the life cycle of such data is decoupled from the data itself, which is thus unprotected against behaviors that might be inconsistent with the original uses the data was created for. So far, such problems have been mitigated by means of regulatory and technical attempts. GDPR is an example of regulatory intervention, which has been recently issued by the European Commission to protect personal data. In the literature, we can find a number of technical approaches that

48

P. Pelliccione et al.

have been proposed for addressing a variety of issues, such as focused on providing support to (i) specify what can be done with the data being modelled [54], (ii) enforce privacy concerns [66] or access control policies [54], (iv) act on the system architecture by distributing it [60], etc. However, they can only partially solve the mentioned issues about data management. The work in [64] proposes an approach that defines and realizes the novel notion of active data, which is intended to be a software module that wraps, encapsulates, and protects personal data. Active data, equipped with security mechanisms, mediate the access to personal data via welldefined interfaces and forbid any different and direct access while preventing any unauthorized use. Active data embody monitor and enforcer technology to both guarantee the preservation of privacy policies specified by the owner and enforce actions that are needed in order to satisfy their privacy desiderations. Security and Safety in Digital Ecosystem With the increasing dynamism of digital ecosystems, the current standards and certification schemes are lacking behind in reflecting the technological advancement [18, 49], namely, in connection to the cybersecurity threats [22] and safety concerns [29, 79]. Let us consider as an example the advancing adoption of autonomous driving. This is one of the scenarios being discussed in connection to the gaps in existing certifications and standards when it comes to their compliance with the dynamic ecosystems. To address the drawbacks of existing standards and certification procedures, existing public bodies and the scientific community are discussing new ways of validating autonomous ecosystems, together with examining digital certificates that could be aligned with policies for ensuring safety and cybersecurity [10, 36]. While these approaches present fair attempts to account for the technological changes facing the transition toward dynamic ecosystems, the scope of the solutions is so far limited to the context of safety and security and lacks to consider other values, such as fairness and ethics.

Problem Description The digitalization of our society is bringing individual systems together; connecting them into dynamic collaborative ecosystems, with individual entities—whether digital, mechanical, or human—dynamically joining and leaving the ecosystems; and adapting their behavior to the complexity of their surroundings [23]. In such an environment characterized by high unpredictability, limited controllability, and natural motivation of the individual entities to optimize their individual outcome, the ecosystem can lose its grip on crucial societal values, such as fairness or ethics. This section studies the problem of architecting and engineering value-based ecosystems, listing the key characteristics of the problem, examined in the rest of the chapter.

Architecting and Engineering Value-Based Ecosystems

49

Motivating Scenarios Dynamic ecosystems can be observed in various contexts, which are expanding with the growing level of digitalization of our society. Examples span from ecosystems with high degrees of mechanization and automatization (e.g., smart manufacturing and Industry 4.0), high degree of human aspects (e.g., smart work and business process management), to examples in between where humans interact largely with the digital and mechanical systems (e.g., smart city, intelligent transportation). Scenario 1: Smart City Mobility Consider a city ecosystem with an emphasis on the scenario of mobility and transportation, as shown in Fig. 1. Let us characterize the entities, interactions, and values that might be relevant in such a context. • Entities In the context of city mobility, there are three main types of entities that engage in mutual interaction: (a) vulnerable road users, (b) vehicles, and (c) road infrastructure. – The vulnerable road users include pedestrians but might also include cyclists or animals on the road. – The vehicles might include cars, motorbikes, busses, trams, as well as emergency vehicles such as ambulances or police cars. – The road infrastructure can include traffic lights, road construction signalling, tollgates, and others. • Interactions One can imagine numerous ecosystem interactions among the listed entities that require collaboration. Examples include:

Fig. 1 Smart city mobility scenario

50

P. Pelliccione et al.

– Information exchange in coordinated moves such as one vehicle overtaking another one and one vehicle giving priority to another one when entering a crossroad. This situation can be specifically challenging if the vehicle one needs to coordinate with is intentionally misbehaving (e.g., to gain some speed benefit from the interaction) and others need to think of collision avoidance with such a misbehaving vehicle. A further challenge in such an interaction is the priority management of emergency vehicles, which need to be given priority. Then values such as transparency, fairness, precaution, safety, responsibility, liability, trust, and solidarity apply to this scenario. – Protection of vulnerable road users, which includes not only the protection of humans from harm by vehicles but also protection by various connected members via sharing information about the presence of the vulnerable road user on the road. This situation is challenging given that vulnerable road users are unlikely to be connected to the ecosystem for bidirectional communication and they might misbehave (e.g., cross the road at the red light), putting other road users in danger. Values such as transparency, disclosure, protection, safety, responsibility, liability, freedom, autonomy, trust, and solidarity apply to this scenario. – Humans act as passengers in vehicles, which also puts them in a vulnerable position. In the case of driverless vehicles, the interaction between the vehicle and its passengers is crucial for building the trust of the passengers in the car and its actions. The same might apply to the passengers of public transportation, such as buses or trams. Values such as transparency, explainability, understandability, communication, security, safety, privacy, freedom, consent, choice, and dignity apply to this scenario. • Values Values belonging to various categories, such as transparency (disclosure, explainability, understandability, communication), justice and fairness, nonmaleficence (precaution, safety, protection, security), responsibility (liability), privacy, freedom and autonomy (consent, choice), trust, dignity, and solidarity, have been listed in the examples above (according to the classification by Jobin et al. [46]). Scenario 2: Business Ecosystems Within business ecosystems, different organizations create and share collective value for specific end users. Here, the concept introduces a shift from traditional value chains (e.g., selling products in retail) to complex value networks (e.g., authorizing financial transactions for credits). Those ecosystems can be characterized by different entities, interactions, and values which highly depend on the application domain. Here, Entities might be divided into the actual end customer who uses the collective value of the ecosystem, aggregators who accumulate the value and stand in direct contact which the end customers, and providers who offer specific products or services during the valuecreation process. Interactions are used as a connection between the different entities. Those interactions range from the provision of products and services to the exchange of information. Moreover, those ecosystems might provide different

Architecting and Engineering Value-Based Ecosystems

51

Values like the protection of privacy through decentralized data management or the seamless user experience through the aggregation of different services. Scenario 3: Smart Fabrics Within smart fabrics, factory workers collaborate with various machines in order to build products for specific customers. Smart fabrics can be characterized by different entities, interactions, and values that depend heavily on the products to build. Entities might be divided into human workers who perform manual steps, assistant machines that support the conducting of steps, and AI-based machines that automate some steps without the integration of the human. Those entities are connected through different Interactions that are mainly characterized as flows in the manufacturing process. Those interactions might be the physical product that is currently built, over data exchange between different machines, to adjustment on the manufacturing line. Moreover, smart fabrics are made of the Values around safety and security. Here, it is important that single entities, regardless of whether humans or machines, are safe from getting hurt by the other entities together with the overall security of the systems against internal failures or external misuse.

The Problem and Related Challenges The problem of architecting and engineering value-based ecosystems calls for addressing a number of aspects that characterize the problem and influence the solution being discussed later in the chapter. The Scope of Value-Based Ecosystems In the abovementioned motivating scenarios, we discuss the entities of the ecosystems, their interactions, and the values involved. However, the scope of a value-based ecosystem can be much wider than that, considering different stakeholders, including the role of authorities responsible for value governance. Furthermore, one shall consider the role of the goals of the individual entities that might be guiding the interaction. Besides individual goals, one can also consider collaborative or competitive goals, which might lead to the creation of coalitions among the entities. How to Engineer Value-Based Ecosystems This aspect calls for exploring the architectural and software engineering strategies facilitating value management and governance in value-based ecosystems. These strategies might include methods for value assurance by design, design-time preparedness for runtime value guarantees (e.g., data to be stored, mechanisms to be enabled), as well as runtime value assessment and mechanisms for corrective actions in case of value violations. How We Can Assess the Values One of the crucial aspects of value governance is the ability to track the current state of value compliance across the ecosystem. This aspect calls for appropriate metrics to assess the values and methods of values verification and validation. At runtime, we might need ways to track the values to understand if an action needs to be taken to guarantee the values. And an interesting

52

P. Pelliccione et al.

research problem is to understand whether it is even realistic to predict the values at system design time, given that the ecosystem is composed of many third-party systems with high unpredictability. How to Guarantee Values at Runtime Despite our best efforts to engineer individual systems for value compliance, there will always be situations when the system will have the tendency to lean out of value alignment. In such cases, it is important to know that we have mechanisms in place that help the system to adapt and repair, possibly via incentive mechanisms that stimulate the system to gravitate back to value alignment. Furthermore, there might be conflicts between individual values (e.g., protection and autonomy, fairness and solidarity), which might be hard to resolve, given varied priorities by different ecosystem stakeholders. Software engineering analogies, such as the value debt, might be useful in addressing this aspect, as some actions might lead to compromising some values and it is important to understand to which degree this is happening and how this can be compensated. The Role of Central Authority and Certification For a long time, critical systems have been subject to certification, which needs to be revisited in the context of value-based ecosystems. With the high degree of dynamism in these ecosystems, the role and format of certification need to be reconsidered. Furthermore, an important element of value-based ecosystems is a certain (central) authority that can become the ground of truth about the state of values in the ecosystem (e.g., collecting the evidence of the reputation of the individual ecosystem entities, the rules that the entities need to adhere to).

Value-Based Ecosystems Before moving on to architecting and engineering future value-based ecosystems, this section aims to clarify the actual meaning of value-based ecosystems. Within the recent literature, various definitions have been proposed for the value and ecosystem terms, both within and outside software engineering. The term value comes originally from psychology and defines “(i) concepts or beliefs, (ii) pertain to desirable end states or behaviours, (iii) transcend specific situations, (iv) guide selection or evaluation of behaviour and events, and (v) are ordered by relative importance” [63]. Moreover, a simplified version of valuebased software engineering describes those values as “relative worth, utility, or importance” [16]. The term ecosystem comes originally from biology and describes it as consisting “of all the organisms and the physical environment with which they interact” [24]. In software engineering, software ecosystems are defined as “a software platform, a set of internal and external developers and a community of domain experts in service to a community of users that compose relevant solution elements to satisfy their needs” [19]. Building on these existing definitions of value and ecosystem, we aim at defining value-based ecosystems. Before providing this definition, we

Architecting and Engineering Value-Based Ecosystems

53

provide other definitions. We start with a definition of the ecosystem and dynamic ecosystem; see Definitions 1 and 2, respectively. Definition 1 (Ecosystem) An ecosystem is a system composed of a set of constituent entities, which can be digital systems or human beings; constituent systems interact or collaborate with each other according to their individual goal as well as common and shared goals. Definition 2 (Dynamic Ecosystem) An ecosystem is dynamic when constituent entities can join and leave the ecosystem during its life cycle, according to a set of defined rules. A dynamic ecosystem needs to be properly managed by the governance of the ecosystem, as defined in Definition 3. Definition 3 (Ecosystem Governance) A long-lived ecosystem needs governance mechanisms to control how decisions regarding changes to values, goals, and rules are made. Governance itself relies on rules, which are, in turn, subject to possible changes. Ecosystem values, goals, and rules should be in line with relevant laws, regulations, and rules of the specific context in which the ecosystem exists. Before providing the definition of a value-based ecosystem, in Definition 4, we provide a general definition of social and human values; to have more concrete examples of those values, we invite the reader to refer to [46, 62, 76]. Definition 4 (Social and Human Values) Social and human values are those values that are recognized as important for the sustainability and long-term wellbeing of the ecosystem as a whole, especially in terms of the outcomes for the humans, defending and systematizing the concepts of right and wrong behaviour. Definition 5 (Value-Based Ecosystem) A value-based ecosystem is a dynamic ecosystem that comprises values, goals, and constituent entities, and is subject to governance. The interactions and behaviours of constituent systems need to comply with the values and goals of the ecosystem and the values and goals of individual constituent entities, under suitable tradeoffs. Figure 2 shows an example of a value-based ecosystem composed of four entities, Entity A, Entity B, Entity C, and Entity D. Each entity, according to the definitions above, has its own goals and values. Entities can interact with each other, and interactions can change over time. Entities can also join and/or leave the ecosystem according to Definition 2. The dynamic aspects are not visible in the figure since it shows the value-based ecosystem at a specific time t. The ecosystem itself has also its own goals and values, which constrain the behavior of the overall ecosystem as well as of its entities, as described in Definition 5. Finally, the figure also shows the governance of the ecosystem, which controls how decisions regarding changes to values, goals, and rules are made (see Definition 3). Ecosystems can be classified as dynamic if they allow flexible modification of the entities or static if they support just a fixed set of entities. This also directly impacts the underlying structures, which can be fixed for the behavior of the entities

54

P. Pelliccione et al.

Fig. 2 Example of a value-based ecosystem at time t

or flexible during the evolution of the ecosystem. While for the fixed entities, the ecosystem might also be prepared against predictable situations, for the flexible entities, the ecosystem also needs to be able to manage uncertainty. Moreover, the ecosystem might provide mechanisms for regulations or allow unregulated usage by the entities. An ecosystem has rules, goals, and values: • Ecosystem Rules: The ecosystem rules are overall guidance mechanisms that all entities need to follow during their participation in the ecosystem. Those rules might be also combined with rights for entities among the other entities. This could be, for example, the frictionless joining of new entities to the ecosystems in a smart city or the usage of existing computing resources within a connected factory. • Ecosystem Goals: The ecosystem goals describe the superior goals that all entities want to achieve in common. This could be, for example, the flexible management of a supply chain and the provision of failsafe equipment in a factory. • Ecosystem Values: The ecosystem values are connected to specific ecosystem goals and describe the superior values of the ecosystem entities. This could be, for example, the saving of privacy of all supply chain actors or ensuring the quality of the factory equipment. According to Definition 1, an ecosystem is composed of constituent entities. Entities can be classified into different groups like humans, which manually make interactions, machines whose interactions are fully automated, or hybrid AI-based

Architecting and Engineering Value-Based Ecosystems

55

entities that interactions are mixed or just supported. Here, those groups can also influence if the entity can automatically or manually be adapted within the environment. The entities can be providers, which offer products or services to other entities, or aggregators that offer products or services to the actual end users. At any moment, entities can exclusively belong to a single ecosystem, also called singlehomed, or they can belong to more than one ecosystem, also called multi-homed. Each entity has goals and values: • Entity Goals: The entity goals describe the goals that a specific entity aims to achieve within the ecosystem. This could be, for example, the maximization of the utilization of specific equipment or the maximization of the revenue from a certain service. • Entity Values: The entity values are connected to specific entity goals and describe the specific values of a single entity. This could be, for example, the reliability of a specific service or the usability of a specific product. According to Definition 1, the various entities can interact and/or collaborate among themselves. Interactions and collaborations can be within the ecosystem or among entities of the ecosystem and entities that do not belong to the ecosystem. Interactions can be cooperative, competing, or conflicting with each other. With this, those interactions might have direct or indirect effects on the involved entities. This, in turn, might also lead to manual or automated reactions of the entities based on the interaction of other entities within the ecosystem. Each interaction provides an offering or a value between the entities: • Interaction Offerings: The interaction offerings describe the provided offerings between some entities in the ecosystem. Those interactions might have different characteristics like unidirectional, bidirectional, or complex offerings between the entities. That could be, for example, the provision of a product or a service. • Interaction Values: The interaction values describe the proposed values between some entities in the ecosystem. Those interactions might have different characteristics like unidirectional, bidirectional, or complex value propositions between the entities. That could be, for example, a product’s quality or service’s reliability. With the switch from classical ecosystems to value-based ecosystems, we also see certain benefits for the whole ecosystem as well as single entities themselves. In the following, we list some of those benefits: • Facilitation of Collaboration: Value-based ecosystems promote trust building among individual entities and thus facilitate the formation of coalitions, in which the individual entities can rely on each other. At the same time, untrustworthy entities can be quickly identified, allowing the other entities to protect themselves from them. • Improved Sustainability: By providing value-based ecosystems, we support the improvement of the sustainability of the ecosystem as well as the specific entities. Here, instead of just covering the financial profit of each entity, those ecosystems include additional non-financial values.

56

P. Pelliccione et al.

• Simplified Entrance: By providing value-based ecosystems, we support the simplified entrance of new entities. Here, instead of a dedicated ecosystem provider, new entities are able to join the ecosystem by accepting the governance mechanisms. • Decentralized Regulation: By providing value-based ecosystems, we can engage all the jointly working entities in value governance. Instead of an ecosystem provider with predefined regulations, the regulations might get adjusted through voting mechanisms by the entities. • Accountability for Value Violation: By having clear mechanisms of value governance, involving all ecosystem entities, trusted evidence of value violation can be collected and used for ensuring accountability and reparation on the side of the violator.

Aspects and Architectural Implications of Solution Approaches Architecting and engineering of value-based ecosystems should start from the identification of the main functionalities, requirements, and constraints these systems should include and guarantee. In the following, we discuss several important aspects that these systems could address. A cross-cutting concern for these aspects is the complexity in the ecosystem: the set of aspects could result in a very complex system, and interacting with it might be burdensome for human participants. This concern should be kept in mind when designing ecosystems and addressed by suitable means to limit or reduce complexity such that humans are willing to participate and able to make informed decisions.

Human- and Values-Centered Systems Humans should be at the center of the system, as well as at the center of the development process. Humans at the Center of the System Systems should be engineered and architected to include humans in the loop. This means that, when needed, part of the control and the decision should be delegated to humans. This concerns the decision of which tasks to allocate to humans and which, instead, to systems. In specific systems, it could be also required to redistribute control and responsibility of the different tasks. It can be required to define a negotiation step, e.g., by enabling voting democratically, or based on individual preferences how to distribute and redistribute control over time. For some systems, it could even make sense to have humans in the loop, meaning that humans have the last word in terms of important decisions or always monitoring

Architecting and Engineering Value-Based Ecosystems

57

the work of the system being ready to take over. Indeed, there are side effects of this solution since the system will be not robust with respect to human foibles, like timing, distraction, bias, etc. Hybrid solutions can be considered, like permitting in specific situations human-initiated detection and correction of erroneous or suboptimal system actions. Another hybrid solution is framed autonomy [30], where the autonomy of an AI is bound inside a frame; any decisions that exceed the frame require human intervention. Overall, there is a need for effective protocols of interaction between humans and systems. The above points are architectural challenges that should be properly discussed and addressed. Another architectural challenge concerns allowing humans to understand what the system is doing and why. This concerns the theme of explainable AI. Humans at the Center of the Development Process Systems should be engineered and architected for humans. This strongly influences the protocol of interaction between humans and systems, as mentioned above. Nowadays, many systems are constraining and limiting the interaction humans have with them. This should change, and the entire development process should have values, like ethics, privacy, and fairness, considered as traditionally done for quality aspects like availability, maintainability, robustness, etc. As normally done for non-functional aspects, it will become important to define suitable tradeoff analysis among these values, as well as with traditional quality aspects. This will force us to explicitly discuss and identify priorities among the values and qualities. To effectively and properly perform this task, it will become important to include expertise that goes beyond computer science [28]. Another type of tradeoff concerns the possibility of conflicting values among different individuals as well as digital systems. This should be resolved via a negotiation protocol suitably defined.

Dealing with Fuzziness One concern about values, goals, and digital systems concerns the divide between the fuzzy nature of the former and the precise nature of computation, algorithms, and program code that make up digital systems. Regulation and law are examples of natural language formulation of rules, obligations, permissions, sanctions, etc. and have long been known to consist of parts with a clear meaning as well as parts which are subject to considerable interpretation [31]. Similarly, we can expect values and goals to have both types of parts. In contrast, digital systems are driven by programs, which are typically much more precise than natural language. This difference in precision, and specifically the fuzziness of goals and values, needs to be handled well in value-based ecosystems. Not only will the ecosystem need mechanisms to make decisions based on fuzzy values and goals, but it also requires mechanisms to handle a potentially diverse understanding of values and goals within the group of human users. A possible mechanism to this end could be

58

P. Pelliccione et al.

voting: human members of the ecosystem vote on specific issues, in order to inform the digital systems about concrete decisions and interpretations. A challenge hereby concerns the selection of a few issues to vote on—there should be few, as to not overburden the users, but the information which the digital systems gain from them should be maximized.

Dealing with Uncertainty These systems should be able to deal with the unpredictability of the entities, taking into account that there are also humans involved, which can have high degrees of unpredictability. Another dimension of complexity is represented by the controllability of these entities, as well as humans. Entities that are part of the system can be only partially controllable or completely uncontrollable. When looking at the overall ecosystem, an overall organizational structure might not exist a priori. Instead, it might emerge from the interactions of the entities. There can be rules which need to be obeyed. For instance, AI-based entities could be subjected to bounded autonomy: the (humans in the) ecosystem set the boundaries within which AI-based entities can operate. Moreover, a solution should take into account the dynamicity of entities, which can join, leave, change, or evolve at any moment. Another aspect of uncertainty to be considered deals with the fact that some entities can have hidden intentions. This implies that a model or representation of the behavior of entities might be not completely aligned with their actual behavior. In some cases, this can be identified with monitoring approaches able to compare the actual behavior with the specification. There can be more complex situations where even though every single action seems to appropriate, the emerging behavior, which might be desired, is unacceptable with respect to the agreed values. These situations are more complex to be prevented.

Quality of Data/Bias Removal The violation of social and human values might be also caused by the data used to train machine learning components [52, 67]. Data can already contain bias due to historical aspects or socio-cultural circumstances. This is, for instance, the case of a large majority of men in STEM (science, technology, engineering, and mathematics) jobs. Bias on data can be added when designing a machine learning model. Biases can be introduced by the collection process, e.g., because of biases of the humans in charge of collecting data or because of errors in the software or hardware. Finally, biases can be introduced during the transformation of data to prepare for training the ML model.

Architecting and Engineering Value-Based Ecosystems

59

In general, machine learning and AI might inherit biases from data, and, depending on the domain, this can cause harm in various ways and/or take unfair decisions that can lead to discrimination. This explains the need for bias detection and mitigation approaches in machine learning [52]. Approaches can be defined at different stages: (i) pre-processing, i.e., before training the ML model; (ii) in-processing, i.e., during the model training; and (iii) post-processing, i.e., after the model training.

Ensuring Values by Design In value-based ecosystems, multiple entities with different propositions interact to increase the overall value of the whole ecosystem. Here, it is hard to change the values of the ecosystem and its entities if they were not initially considered during their creation. Therefore, it would be beneficial to ensure the usage of those values due to the design of the ecosystems as well as the entities. The usage might be supported by providing a value-by-design concept. Here, ethics by design is a related concept to systematically include ethical values and principles in the design and development processes for software. The approach can be applied to traditional as well as agile software development methods [80]. Moreover, it is used by AI-based systems to reduce the bias of the predictions. To apply that concept to value-based ecosystems, we need to generalize the used values and split the design process. First, for the generalization, we need to consider also values apart from ethical considerations to increase flexibility in the design of the ecosystems. Second, for the design process, we need to consider the initial design of the ecosystem by multiple entities as well as the design process for those initial entities and new entities themselves. Here, values need to be considered from the derivation of the requirements to the continuous evolution. Moreover, both design and development processes need to be intertwined to support a value alignment. Additionally, it is important to control the compliance of the values of new entities before they join the ecosystem to prevent initial conflicts and support the internal governance mechanisms.

Verification and Validation As anticipated, there will be the need for verification and validation (V&V) techniques able to identify, prevent, or block behaviors that are unacceptable with respect to agreed values and defined tradeoffs among them and qualities as well. There exist various runtime verification approaches [60] that study the dynamic analysis of execution traces against formal specifications. These approaches normally include two steps, which are the creation of a monitor from the specification and the definition of algorithms for the evaluation of traces against the generated

60

P. Pelliccione et al.

monitors. Some approaches only focus on identifying problems; other approaches include also enforcers to solve the problem when identified or performing actions able to prevent the error to occur. Another notable technique for a value-based system is field-based testing, which operates in the field to reveal those faults that escape in-house testing [14]. The work in [66] focuses on the important notion of self-adaptivity of testing in the field; this is an important aspect of test approaches in the field since these techniques need to adapt in many ways their strategy to the context and the emerging behaviors of the system under test. Value-based systems might also require new types of V&V techniques that are more tailored to catch behaviors related to values. An example of such techniques is social testing introduced by Facebook [5], which aims at identifying social bugs, i.e., those bugs that arise and emerge through community interactions. As suggested in [5], social testing is another (abstraction) level of testing above unit, integration, and system testing.

Value-Based Supervisor To guarantee the correct behavior of value-based ecosystems, also according to social and human values, it is promising and effective the use of value-based supervisors. By supervisors, we mean software entities that, first, are able to monitor the behavior of the ecosystem, both for what concerns the behavior of individual consistent entities and the behavior that is emerging from the collaboration and interaction of the constituent entities of the ecosystem. Then, they should be able to actively block bad behavior by mediating and filtering interactions and collaborations leading to behavior which violates social and human values. Supervisors can also be exploited by the ecosystem governance to enforce rules, as well as an instrument to collect useful data and instruct stakeholders involved in the governance to change rules, values, and/or goals. Supervisors can be external entities, as it is often done in safety-critical domains [17], or can be part of the ecosystem, e.g., wrapped around digital constituent entities or their components [7]. Since the various constituent entities of the value-based ecosystem might have different values, there is a need to find a tradeoff among the various values in order to satisfy each value specification. However, a tradeoff cannot be always mitigated or solved. This calls for a negotiation that should necessarily involve stakeholders. For instance, the decision might be taken democratically by running a poll to collect the opinion of the various stakeholders.

Architecting and Engineering Value-Based Ecosystems

61

Governance Mechanisms For any long-lived value-based ecosystem, the rules, goals, and values are more than likely to require adaptation over time. Rules may include prerequisites that systems need to meet before being permitted to join an ecosystem. They may also specify levels of access and privileges and could possibly benefit from being specified in deontic logic [39] capturing permitted and forbidden actions, obligations, and more. In the smart city example, some entities—like the police—might be granted with privileged access to both systems in the ecosystem to control they behaviour, and data. New systems wishing to join the ecosystem (e.g., by entering the city) may have to implement interfaces that grant such access. Such a mechanism could then be used to enable the police to stop a harmfully or even maliciously behaving system with an emergency stop switch, figuratively speaking. However, in some social contexts, such access levels might be seen as overly far-reaching and possibly violating the privacy of human users of systems. Naturally, societal views with respect to such rules can change over time, for instance, in cases like privacy violation scandals. Hence, an ecosystem architecture requires governance mechanisms to update rules, goals, and values, e.g., to define and evolve levels of privileges and authority. Such mechanisms need to specify aspects such as who is allowed to propose changes, who is eligible for voting about proposed changes, and which type of quorum is necessary, e.g., a unanimous vote by at least 80% of the eligible voters within 10 days. The governance mechanism can be seen in analogy to the constitution of democracy, along with the institutions that implement it. An instance of a digital governance mechanism that might be suitable for the context of value-based ecosystems can be found in the EOS blockchain platform:2 a “constitution” encodes rules and permissions and updates logic. The constitution is versioned, and individual transactions on the blockchain refer to a particular version of the constitution. Hence, it is transparent which set of rules applies (or is applied) to a given transaction. That being said, implementing such a governance mechanism could make use of a blockchain, but could well be achieved without blockchain technology.

Reputation and Incentives One of the key prerequisites of value governance is the ability of the ecosystem to incentivize behaviors that contribute to promoting the agreed values in the ecosystem. This can be done by the mechanisms of reward and punishment, which are however only possible if there is indisputable evidence of good/bad behavior.

2 https://www.leewayhertz.com/what-is-eos-blockchain/,

accessed 2023-04-14.

62

P. Pelliccione et al.

One of the strategies to understand the tendencies of the ecosystem entities to act well/badly over time is based on the concept of reputation [21]. Reputation can be fed by the direct experience of the individual entities interacting with each other, and shall be managed by the ecosystem governance mechanism, which needs to guarantee the validity of the evidence that the reputation is based on, and shall be responsible for managing the reputation scores over time (e.g., them fading away in the absence of recent interactions).

Usage of Digital Twins In value-based ecosystems, many entities are interacting with each other based on loosely coupled connections which enable a dynamic rise and fall of the individual entities. Here, it is hard to determine the effect of new entities or changed interactions on the whole ecosystem. Therefore, it would be beneficial to determine those effects on single entities or parts of the ecosystem in advance. This determination might be supported by the usage of digital twins that create a virtual representation of the actual entities. Toward the integration of digital twins, we can divide between digital models, which provide a manual data flow between a digital object and a physical object; digital shadows, which provide the digital object automatically; and digital twins, which automatically link the data flow back to the physical object. With this, digital twins have the advantage of simulating the behavior of an entity based on existing data in the ecosystem together with predicting the value outcome in different scenarios [5, 25, 45]. In some cases, those predictions can also lead to an automatic adaption in terms of changed interactions or values of an entity based on the comparison of those scenarios with its own goals. With this, entities might focus on increasing their own value within the ecosystem or the overall ecosystem value. Moreover, instead of a single entity, digital twins can be also organized in network structures to simulate and predict parts of the ecosystem. However, the usage of digital twins comes with the disadvantage of the overhead in creating and maintaining them. Here, it is a crucial step to determine the provided data within the data model and regularly rethink those choices to avoid the omission of new effects and the changing impact of existing effects in the ecosystem.

Research Opportunities Value-based ecosystems have to handle standard issues in software engineering, such as scalable architectures or incremental development processes, as well as the issues connected to the increased integration of AI in the digital systems around us, such as biases in training data or limiting the scope in which the system can make decisions autonomously. In addition, there are numerous challenges specific to this context, including the following:

Architecting and Engineering Value-Based Ecosystems

63

• Creating models, methods, and tools for architecting and engineering value-based ecosystems, addressing the aspects outlined in the previous section. For instance, how to handle the split and interaction between entities and the ecosystem, in terms of architectural design and technical mechanisms of software systems? It might be necessary to create and apply value-by-design methods, but can those be flexible enough to adapt to changing values? • How to validate and verify the values, which is challenging due to the fact that the individual involved systems are developed by different parties, and so we might be limited to runtime V&V methods, which need to be rethought for the context of value-based ecosystems. The methods need to further work with the fact that the individual values might be conflicting (e.g., protection and autonomy, fairness and solidarity) and hard to improve without negatively influencing others. • How to specify, integrate, and effectively implement values, constraints and boundaries (e.g., for AI autonomy), governance, and control mechanisms. For instance, how to compare ecosystem values with the behavior of an entity, or how to ensure (or, better yet, technically guarantee) that required control can indeed be exercised—e.g., that the police can remote-disable a misbehaving self-driving car? Which methods of verification and validation can ensure compatibility or adherence with the basic rules of an ecosystem? How should supervisor software, which checks adherence to rules and values, be designed and integrated into an ecosystem? When are incentives, positive or negative, the best way to achieve adherence, and when is direct control required? • How to devise solutions that put humans at the center, yet are not overly complex or time-demanding? How to handle the fuzziness of goals and values in precise computer systems in a satisfactory way? How to handle unpredictability, uncontrollability, and other forms of uncertainties of entities in an ecosystem, e.g., to ensure safety of all participants? Can networks of digital twins provide high-quality predictions? • How to ensure that the ecosystem is able to self-correct? Much in the sense of democracy’s ability to self-correct and even question and revise its very foundation, value-based ecosystems need to be able to evolve and react to flaws as they are discovered. The assumption is that we cannot build a flawless system; hence, it is essential for any solution and its socio-technical context to be able to self-correct even on core aspects. Can suitable governance mechanisms provide sufficient means for self-correcting behavior? This list of questions spawns a wide array of possible research directions and can hopefully be a valuable input for selecting research topics for years to come.

64

P. Pelliccione et al.

Summary In order to shape the ecosystems of the future to be friendly toward humans and society, significant efforts are required. In this chapter, we propose to channel these efforts into the creation of value-based ecosystems, which are guided by goals, societal values, and rules. After motivating the proposal and reviewing the state of the art, we analyze the problems and challenges. We then describe the vision of value-based ecosystems and provide definitions and examples, before discussing various aspects that need to be considered when working toward the vision. Finally, we review the resulting research challenges and questions, which we hope will inform the next generation of software architecture research works on this important and highly relevant topic. Acknowledgments The work of B. Buhnova was supported by ERDF “CyberSecurity, CyberCrime and Critical Information Infrastructures Center of Excellence” (No. CZ.02.1.01/0.0/0.0/16_019/0000822). The work of S. Gottschalk was supported by the German Research Foundation (DFG) within the CRC “On-The-Fly Computing” (CRC 901, Project Number: 160364472SFB901) and the German Federal Ministry of Education and Research (BMBF) through Software Campus grant (Project Number: 01IS17046). The work of P. Pelliccione was supported by the Centre of EXcellence on Connected, Geo-Localized and Cybersecure Vehicles (EX-Emerge), funded by the Italian government under CIPE resolution n. 70/2017 (Aug. 7, 2017).

References 1. Ethics Commission Automated and Connected Driving, Appointed by the German Federal Minister of Transport and Digital Infrastructure, Report (extract) (2017). https://bmdv.bund.de/ SharedDocs/EN/publications/report-ethics-commission-automated-and-connected-driving. html. Accessed: 2023-04-14 2. European Commission. General Data Protection Regulation (2018). https://gdpr.eu/. Accessed: 2023-04-14 3. European Commission, High-Level Expert Group on AI: The Ethics Guidelines for Trustworthy Artificial Intelligence (2019) 4. Abdelghani, W., Zayani, C.A., Amous, I., Sèdes, F.: Trust management in social internet of things: a survey. In: Dwivedi, Y.K., Mäntymäki, M., Ravishankar, M., Janssen, M., Clement, M., Slade, E.L., Rana, N.P., Al-Sharhan, S., Simintiras, A.C. (eds.) Social Media: The Good, the Bad, and the Ugly, pp. 430–441. Springer International Publishing, Cham (2016) 5. Ahlgren, J., Berezin, M.E., Bojarczuk, K., Dulskyte, E., Dvortsova, I., George, J., Gucevska, N., Harman, M., Lämmel, R., Meijer, E., Sapora, S., Spahr-Summers, J.: Wes: Agent-based user interaction simulation on real infrastructure. In: Proceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops, ICSEW’20, pp. 276–284. Association for Computing Machinery, New York (2020) 6. Amoroso, D., Tamburrini, G.: The ethical and legal case against autonomy in weapons systems. Global Jurist 17, 20170012 (2017) 7. Autili, M., Di Ruscio, D., Inverardi, P., Pelliccione, P., Tivoli, M.: A software exoskeleton to protect and support citizen’s ethics and privacy in the digital world. IEEE Access 7, 62011– 62021 (2019)

Architecting and Engineering Value-Based Ecosystems

65

8. Awad, E., Dsouza, S., Bonnefon, J.F., Shariff, A., Rahwan, I.: Crowdsourcing moral machines. Commun. ACM 63(3), 48–55 (2020). https://doi.org/10.1145/3339904 9. Awad, E., Dsouza, S., Kim, R., Schulz, J., Henrich, J., Shariff, A., Bonnefon, J.F., Rahwan, I.: The moral machine experiment. Nature 563, 59–64 (2018). https://doi.org/10.1038/s41586018-0637-6 10. Bakirtzis, G., Carr, S., Danks, D., Topcu, U.: Dynamic Certification for Autonomous Systems (2022) 11. Bechtsis, D., Tsolakis, N., Vlachos, D., Srai, J.S.: Intelligent autonomous vehicles in digital supply chains: a framework for integrating innovations towards sustainable value networks. J. Clean. Prod. 181, 60–71 (2018) 12. Becker, R., D’Angelo, G., Ghobadi, S., Gilbert, H.: Fairness in influence maximization through randomization. J. Artif. Intell. Res. 73, 1251–1283 (2022). https://doi.org/10.1613/jair.1.13367 13. Benzmüller, C., Parent, X., van der Torre, L.W.N.: Designing normative theories of ethical reasoning: Formal framework, methodology, and tool support. Artif. Intell. 287, 103348 (2020) 14. Bertolino, A., Braione, P., Angelis, G.D., Gazzola, L., Kifetew, F., Mariani, L., Orrù, M., Pezzè, M., Pietrantuono, R., Russo, S., Tonella, P.: A survey of field-based testing techniques. ACM Comput. Surv. 54(5), 1–39 (2021). https://doi.org/10.1145/3447240 15. Bicchi, A., Tamburrini, G.: Social robotics and societies of robots. Inf. Soc. 31(3), 237–243 (2015). https://doi.org/10.1080/01972243.2015.1020196 16. Biffl, S.H.: Value Based Software Engineering. Springer, Berlin; Heidelberg [u.a.] (2006) 17. Bombarda, A., Bonfanti, S., Galbiati, C., Gargantini, A., Pelliccione, P., Riccobene, E., Wada, M.: Guidelines for the development of a critical software under emergency. Inf. Softw. Technol. 152, 107061 (2022). https://doi.org/10.1016/j.infsof.2022.107061 18. Bonnin, H.: The certification challenges of connected and autonomous vehicles. In: 9th European Congress on Embedded Real Time Software and Systems (ERTS 2018). Toulouse, France (2018) 19. Bosch, J., Bosch-Sijtsema, P.: From integration to composition: on the impact of software product lines, global development and ecosystems. J. Syst. Softw. 83(1), 67–76 (2010) 20. Broersen, J.M., Cranefield, S., Elrakaiby, Y., Gabbay, D.M., Grossi, D., Lorini, E., Parent, X., van der Torre, L.W.N., Tummolini, L., Turrini, P., Schwarzentruber, F.: Normative reasoning and consequence. In: Normative Multi-Agent Systems, Dagstuhl Follow-Ups, vol. 4, pp. 33– 70. Schloss Dagstuhl—Leibniz-Zentrum für Informatik (2013) 21. Buhnova, B.: Trust management in the Internet of Everything. In: Proceedings of the 16th European Conference on Software Architecture-Companion Volume, pp. 1–13. Springer, Berlin (2023). Preprint at http://arxiv.org/abs/2212.14688 22. Burzio, G., Cordella, G.F., Colajanni, M., Marchetti, M., Stabili, D.: Cybersecurity of Connected Autonomous Vehicles: A ranking based approach. In: 2018 International Conference of Electrical and Electronic Technologies for Automotive, pp. 1–6 (2018). https://doi.org/10. 23919/EETA.2018.8493180 23. Capilla, R., Cioroaica, E., Buhnova, B., Bosch, J.: On autonomous dynamic software ecosystems. IEEE Trans. Eng. Manag. 69(6), 3633–3647 (2022) 24. Chapin, F.S., Matson, P.A., Mooney, H.A.: Principles of Terrestrial Ecosystem Ecology. Springer, New York (2002). https://doi.org/10.1007/b97397 25. Cioroaica, E., Kuhn, T., Buhnova, B.: (do not) trust in ecosystems. In: 2019 IEEE/ACM 41st International Conference on Software Engineering: New Ideas and Emerging Results (ICSENIER), pp. 9–12. IEEE, New York (2019) 26. Cioroaica, E., Purohit, A., Buhnova, B., Schneider, D.: Goals within trust-based digital ecosystems. In: 2021 IEEE/ACM Joint 9th International Workshop on Software Engineering for Systems-of-Systems and 15th Workshop on Distributed Software Development, Software Ecosystems and Systems-of-Systems (SESoS/WDES), pp. 1–7. IEEE, New York (2021) 27. Contissa, G., Lagioia, F., Sartor, G.: The ethical knob: ethically-customisable automated vehicles and the law. Artif. Intell. Law 25(3), 365–378 (2017). https://doi.org/10.1007/s10506017-9211-z

66

P. Pelliccione et al.

28. Cooper, A.: The Inmates Are Running the Asylum: Why High Tech Products Drive Us Crazy and How to Restore the Sanity, 2nd (edn.) SAMS Publishing, New York (2004) 29. Cummings, M.: Adaptation of Human Licensing Examinations to the Certification of Autonomous Systems, pp. 145–162. Springer International Publishing, Cham, Switzerland (2019). https://doi.org/10.1007/978-3-319-97301-2_8 30. Dumas, M., Fournier, F., Limonad, L., Marrella, A., Montali, M., Rehse, J.R., Accorsi, R., Calvanese, D., Giacomo, G.D., Fahland, D., Gal, A., Rosa, M.L., Völzer, H., Weber, I.: Augmented business process management systems: a research manifesto. arXiv:2201.12855 [cs.AI] (2022) 31. Easterbrook, F.H.: Legal interpretation and the power of the judiciary. Harvard Journal of Law and Public Policy, 7, 87 (1984) 32. EDPS: Opinion 4/2015, towards a new digital ethics—data, dignity and technology. https://edps.europa.eu/sites/edp/files/publication/15-09-11_data_ethics_en.pdf (2015). Accessed: 2023-04-14 33. European Group on Ethics in Science and New Technologies and others: statement on artificial intelligence, robotics and ‘autonomous’ systems: Brussels, 9 March 2018. EU: European Union (2018) 34. Flammini, M., Kodric, B., Varricchio, G.: Strategyproof mechanisms for friends and enemies games. Artif. Intell. 302, 103610 (2022). https://doi.org/10.1016/j.artint.2021.103610 35. Floridi, L.: Soft ethics and the governance of the digital. Philos. Technol. 31(1), 1–8 (2018) 36. García-Magariño, I., Sendra, S., Lacuesta, R., Lloret, J.: Security in vehicles with IoT by prioritization rules, vehicle certificates, and trust management. IEEE Internet Things J. 6(4), 5927–5934 (2019). https://doi.org/10.1109/JIOT.2018.2871255 37. Gogoll, J., Müller, J.F.: Autonomous cars: In favor of a mandatory ethics setting. Sci. Eng. Ethics 23(3), 681–700 (2017). https://doi.org/10.1007/s11948-016-9806-x 38. Gordijn, J., Akkermans, J.M.: Value-based requirements engineering: exploring innovative ecommerce ideas. Requir. Eng. 8(2), 114–134 (2003) 39. Governatori, G., Milosevic, Z.: A formal analysis of a business contract language. International Journal of Cooperative Information Systems 15(04), 659–685 (2006). https://doi.org/10.1142/ S0218843006001529 40. Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., Pedreschi, D.: A survey of methods for explaining black box models. ACM Comput. Surv. 51(5), 1–42 (2018). https://doi. org/10.1145/3236009 41. Highsmith, J., Fowler, M.: The agile manifesto. Software Development Magazine 9(8), 29–30 (2001) 42. Holstein, T., Dodig-Crnkovic, G., Pelliccione, P.: Steps towards real-world ethics for selfdriving cars: Beyond the trolley problem. In: Thompson, S.J. (ed.) Handbook of Research on Machine Ethics and Morality (2021) 43. Hussain, W., Perera, H., Whittle, J., Nurwidyantoro, A., Hoda, R., Shams, R.A., Oliver, G.: Human values in software engineering: Contrasting case studies of practice. IEEE Trans. Softw. Eng. 48(5), 1818–1833 (2022). https://doi.org/10.1109/TSE.2020.3038802 44. Inverardi, P.: The european perspective on responsible computing. Commun. ACM 62(4), 64 (2019) 45. Iqbal, D., Buhnova, B.: Model-based approach for building trust in autonomous drones through digital twins. In: 2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 656–662. IEEE, New York (2022) 46. Jobin, A., Ienca, M., Vayena, E.: The global landscape of AI ethics guidelines. Nature Machine Intelligence 1(9), 389–399 (2019) 47. Kephart, J.O., Chess, D.M.: The vision of autonomic computing. Computer 36(1), 41–50 (2003). https://doi.org/10.1109/MC.2003.1160055 48. Kothandaraman, P., Wilson, D.T.: The future of competition: value-creating networks. Ind. Mark. Manag. 30(4), 379–389 (2001)

Architecting and Engineering Value-Based Ecosystems

67

49. Kusnirakova, D., Buhnova, B.: Rethinking certification for higher trust and ethical safeguarding of autonomous systems. In: Proceedings of the 18th International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE). Scitepress, Portugal (2023) 50. Laboratory, H.: Society 5.0: A People-centric Super-smart Society. Springer Nature, Singapore (2020) 51. Maier, M.W.: Architecting principles for systems-of-systems. Syst. Eng. 1(4), 267–284 (1999) 52. Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., Galstyan, A.: A survey on bias and fairness in machine learning. ACM Comput. Surv. 54(6), 1–35 (2021). https://doi.org/10.1145/3457607 53. OASIS: Privacy Management Reference Model and Methodology (PMRM) version 1.0 (2016) 54. Parent, X., van der Torre, L.: Handbook of Deontic Logic and Normative Systems. College Publication, Marshalls Creek (2013) 55. Pelliccione, P., Knauss, E., Ågren, S.M., Heldal, R., Bergenhem, C., Vinel, A., Brunnegård, O.: Beyond connected cars: a systems of systems perspective. Sci. Comput. Program. 191, 102414 (2020). https://doi.org/10.1016/j.scico.2020.102414 56. Pereira, L.M., Saptawijaya, A.: Programming machine ethics. In: Studies in Applied Philosophy, Epistemology and Rational Ethics, vol. 26. Springer, Berlin (2016). https://doi.org/10. 1007/978-3-319-29354-7 57. Romay, M.P., Cuesta, C.E., Fernández-Sanz, L.: On self-adaptation in systems-of-systems. In: Proceedings of the First International Workshop on Software Engineering for Systemsof-Systems, SESoS ’13, p. 29–34. Association for Computing Machinery, New York (2013). https://doi.org/10.1145/2489850.2489856 58. Sagar, S., Mahmood, A., Sheng, Q.Z., Pabani, J.K., Zhang, W.E.: Understanding the trustworthiness management in the social internet of things: a survey. arXiv preprint arXiv:2202.03624 (2022) 59. Salve, A.D., Mori, P., Ricci, L.: A survey on privacy in decentralized online social networks. Computer Science Review 27, 154–176 (2018) 60. Sánchez, C., Schneider, G., Ahrendt, W., Bartocci, E., Bianculli, D., Colombo, C., Falcone, Y., Francalanza, A., Krsti´c, S., Lourenço, J.M., Nickovic, D., Pace, G.J., Rufino, J., Signoles, J., Traytel, D., Weiss, A.: A survey of challenges for runtime verification from advanced application domains (beyond software). Formal Methods in System Design 54(3), 279–335 (2019). https://doi.org/10.1007/s10703-019-00337-w 61. Santoro, M., Marino, D., Tamburrini, G.: Learning robots interacting with humans: from epistemic risk to responsibility. AI & Soc. 22(3), 301–314 (2008). https://doi.org/10.1007/ s00146-007-0155-9 62. Schwartz, S.: Basic human values: theory, measurement, and applications. Revue Francaise de Sociologie 47(4), 929 (2006) 63. Schwartz, S.H.: Universals in the content and structure of values: theoretical advances and empirical tests in 20 countries. Adv. Exp. Soc. Psychol. 25, 1–65 (2009) 64. Scoccia, G.L., Autili, M., Pelliccione, P., Inverardi, P., Fiore, M.M., Russo, A.: Hey, my data are mine! active data to empower the user. In: 2020 IEEE/ACM 42nd International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER), pp. 5–8 (2020) 65. Shishkov, B., Janssen, M.: Enforcing context-awareness and privacy-by-design in the specification of information systems. In: Shishkov, B. (ed.) Business Modeling and Software Design, pp. 87–111. Springer, Berlin (2018) 66. Silva, S., Bertolino, A., Pelliccione, P.: Self-adaptive testing in the field: are we there yet? In: Proceedings of the 17th Symposium on Software Engineering for Adaptive and Self-Managing Systems, SEAMS ’22, pp. 58–69. Association for Computing Machinery, New York (2022). https://doi.org/10.1145/3524844.3528050 67. Stoyanovich, J., Abiteboul, S., Howe, B., Jagadish, H.V., Schelter, S.: Responsible data management. Commun. ACM 65(6), 64–74 (2022) 68. Sütfeld, L.R., Gast, R., König, P., Pipa, G.: Using virtual reality to assess ethical decisions in road traffic scenarios: applicability of value-of-life-based models and influences of time pressure. Front. Behav. Neurosci. 11, 122 (2017)

68

P. Pelliccione et al.

69. Tamburrini, G.: On the ethical framing of research programs in robotics. AI Soc. 31(4), 463– 471 (2016). https://doi.org/10.1007/s00146-015-0627-2 70. Thew, S., Sutcliffe, A.: Value-based requirements engineering: method and experience. Requir. Eng. 23(4), 443–464 (2018). https://doi.org/10.1007/s00766-017-0273-y 71. Thornton, S.M., Pan, S., Erlien, S.M., Gerdes, J.C.: Incorporating ethical considerations into automated vehicle control. IEEE Trans. Intell. Transp. Syst. 18(6), 1429–1439 (2017). https:// doi.org/10.1109/TITS.2016.2609339 72. Tolmeijer, S., Kneer, M., Sarasua, C., Christen, M., Bernstein, A.: Implementations in machine ethics: A survey. CoRR abs/2001.07573 (2020). https://arxiv.org/abs/2001.07573 73. Webb, A.: The Big Nine: How the Tech Titans and Their Thinking Machines Could Warp Humanity. PublicAffairs (2019) 74. Weber, I., Staples, M.: Programmable money: next-generation blockchain-based conditional payments. Digital Finance (DFIN) 4(2), 109–125 (2022). https://doi.org/10.1007/s42521-02200059-5 75. Werkhoven, P., Kester, L., Neerincx, M.: Telling autonomous systems what to do. In: Proceedings of the 36th European Conference on Cognitive Ergonomics, ECCE ’18. Association for Computing Machinery, New York (2018). https://doi.org/10.1145/3232078.3232238 76. Whittle, J., Ferrario, M.A., Simm, W., Hussain, W.: A case for human values in software engineering. IEEE Softw. 38(1), 106–113 (2021). https://doi.org/10.1109/MS.2019.2956701 77. Wudka, B., Thomas, C., Siefke, L., Sommer, V.: A reconfiguration approach for open adaptive systems-of-systems. In: 2020 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW), pp. 219–222 (2020). https://doi.org/10.1109/ISSREW51248. 2020.00076 78. Yu, E.S.: Social modeling and i ∗ . In: Borgida, A.T., Chaudhri, V.K., Giorgini, P., Yu, E.S. (eds.) Conceptual Modeling: Foundations and Applications. Essays in Honor of John Mylopoulos, LNCS, vol. 5600, pp. 99–121. Springer, Berlin (2009). https://doi.org/10.1007/978-3-64202463-4_7 79. Zhao, T., Yurtsever, E., Paulson, J., Rizzoni, G.: Formal certification methods for automated vehicle safety assessment. IEEE Trans. Intell. Veh. 8(1), 232–249 (2022) 80. Zuber, N., Gogoll, J., Kacianka, S., Pretschner, A., Nida-Rümelin, J.: Empowered and embedded: ethics and agile processes. Humanities and Social Sciences Communications 9(1), 1–13 (2022). https://doi.org/10.1057/s41599-022-01206-4

Continuous Alignment Between Software Architecture Design and Development in CI/CD Pipelines Davide Taibi, Yuanfang Cai, Ingo Weber, Mehdi Mirakhorli, Michael W. Godfrey, John T. Stough, and Patrizio Pelliccione

Abstract Oftentimes, architectural information is not well-maintained, while software is developed and thus is out of sync with the product. Development of architectural models and documentation is therefore frequently done (i) before development starts and quickly becomes outdated, or (ii) as an afterthought or step taken when, e.g., significant refactoring becomes necessary, or (iii) a combination of the two. In order to resolve this dilemma and maintain better clarity and reasoning on the system, we propose to ensure continuous alignment between software architecture design and the development process itself, by suitable extension and use of CI/CD pipelines. Specifically, methods from architecture recovery should serve as a good basis for achieving such continuous alignment, e.g., when focused D. Taibi () University of Oulu, Finland and Tampere University, Oulu, Finland e-mail: [email protected] Y. Cai Drexel University, Philadelphia, PA, USA e-mail: [email protected] I. Weber Technical University of Munich, School of CIT, and Fraunhofer Gesellschaft, Munich, Germany e-mail: [email protected] M. Mirakhorli Rochester Institute of Technology, Rochester, NY, USA e-mail: [email protected] M. W. Godfrey University of Waterloo, Waterloo, ON, Canada e-mail: [email protected] J. T. Stough JHNA, Washington, DC, USA e-mail: [email protected] P. Pelliccione Gran Sasso Science Institute (GSSI), L’Aquila, Italy e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 P. Pelliccione et al. (eds.), Software Architecture, https://doi.org/10.1007/978-3-031-36847-9_4

69

70

D. Taibi et al.

or projected on specific critical user journeys. The alignment will also allow the detection of architectural decay and deviation from targeted non-functional properties. In this chapter, we outline the vision and open research topics toward continuous alignment.

Introduction and Motivation It is postulated that architectural information has historically been expensive to produce and maintain. Thus, it is often out of sync with the actual developed product, at best, or even ignored entirely, at worst. This leads to a lack of architectural clarity in large, complex, and rapidly evolving systems. Additionally, architectural design is often considered unclear and thus perhaps does not justify documentation delays early in system development until the point there is a working system. At that point, design has essentially already occurred and architectural information is historical, documenting the existing system, rather than a conceptual model directed at the future state and providing a roadmap of the system development. The creation of architectural models and documentation is therefore potentially an afterthought or step taken when a problem that requires significant refactoring occurs, and not as a normal aspect of the system life cycle concurrently developed with rapidly evolving agile systems. In order to resolve this dilemma and maintain better clarity and reasoning on the system, better methods are necessary to ensure that there is continuous alignment between software architecture design and the development process itself. The vision of this chapter is the incorporation of architectural information directly into the code repository so that it is both maintained and measured during runs of the continuous integration/continuous deployment (CI/CD) pipeline [1]. This vision spans an area of research and could lead to very real benefits in overall system-problem-fit and development team productivity. The motivation is to close the gap between architectural design and rapid agile development methods so that the value of architectural reasoning increases over the system life cycle precisely because the architectural information is always current with the system itself. Narrowing the scope of the architectural information is important—if the intent does not allow for modularity in the architectural information, then the view of “architecture” by default can be considered to represent the whole system, often expressed in a single viewpoint such as a very large and unreadable wall chart. In such a case, architectural documentation is less likely to be maintained or understood as the actual system is developed and maintained. Instead, it must be focused on specific high-value elements that are modular, in the same way, that the envisioned system components are modular, so that the various architecture views can be consumed by development teams as they focus on specific iterations. In this way, architectural information (representing both static and dynamic aspects of the components under revision) exists for the entire system and is maintained in the concurrent development of system components; therefore, in the envisioned

Continuous Alignment Between Architecture and Development

71

future state proposed by this approach, architectural views of highest value can be developed, analyzed, validated, and maintained with the code that is of the highest value and that is actually changing while being processed repeatedly by the CI/CD pipeline. This chapter will outline the relationships between architecture and agile/DevOps and will explore the specific architectural information necessary for integration into the CI/CD pipeline. Additionally, solution components and open research topics will be explored for agile architecting, continuous architecting, and other approaches to architecting large complex systems that evolve continuously. We will put a specific focus on capturing architecture knowledge and ensuring the maintenance of that data, including architecture decision records (ADRs) and other key documentation that is not normally available from the source code, and providing a framework plan for traceability among artifacts. The implications of this research are significant not only to bottom-line industry concerns expressed in normal quality attributes and business drivers but also in emerging topics such as sustainability. For an example of the latter, consider the ability to identify and consider the implications of power drain due to massive, previously ignored bottlenecks and resulting workarounds with large, expensive, and power-intensive solutions; this is in contrast to identifying and executing a refactoring plan based on concurrent architectural information.

Background and State of the Art Architectural tools and integrated development environments are often not directly connected [2, 3]. In some cases, such as complex cyber-physical systems, there are code generation tools that can be used to develop system-level models and to autogenerate code [4]. However, in many large complex systems, and in particular, in those relying on scalable cloud services or other distributed environments, tools are developed independently and in many cases are integrated into proprietary DevOps stacks. Decades of training in architecture modeling and architectural documentation have, in many cases, not resulted in the adoption of architectural design best practices [5]. Few of the reasons for the latter might be the lack of automated tools and the long manual process required both for modeling and documenting the architecture. Another possible reason is the rush to implement new features, which often requires applying quick patches to deliver a feature to the customer quickly. Another set of reasons concerns short-term economic considerations: the customer might only want to pay for visible results, and anything that does not directly result in features in the system might be discounted as “nice-to-have”. The vision of integrating a set of automated tools for architectural reconstruction and documentation into developers’ DevOps pipeline might be beneficial for companies, to keep track of their architecture with minimal overhead. There have been some attempts to the integration of design thinking and practices into developers’ daily activities [2, 6]. For instance, researchers have used Plugins for popular programming integrated development environments (IDEs) as a mechanism

72

D. Taibi et al.

to bring architecture reverse engineering [7] or design documentation tools [8] into developers’ IDEs. However, overall architecture analysis tools have been largely missing from developers’ daily pipelines. Existing architectural languages such as Architecture Analysis and Design Language (AADL) [9] provide some structural information that may allow for the formal structure to include architectural element measures. However, such tools and languages are neither broadly adopted nor understood in terms of organizational value for investment into the CI/CD pipeline. Therefore, a clear explanation of value delivery for architectural development for inclusion into the CI/CD pipeline is postulated as of equal importance to the continued development of the tools themselves (e.g., aiming at explaining the value to the organization, the line of thinking might be similar to the history of building the business case to invest in unit testing frameworks in the early days of test-driven design). Therefore, we believe that architectural measures, constraints, qualities, and other information must be available to be checked in, compiled, and tested along with the code in a way that is not overly cumbersome for the development teams. Recent studies emphasize the complex interaction of CI/CD pipelines and software architecture itself. In particular, there is a common understanding that organizations cannot realize DevOps in a scalable manner without incorporating that into the architectural considerations [10–14]. On the other hand, studies have demonstrated that software architecture itself is one of the biggest bottlenecks on CI/CD pipelines [13]. Similarly, other researchers have attempted to identify architectural design patterns that support seamless CI/CD [15]. These prior works all further demand the need for the integration of software architecture analysis tools and practices in CI/CD pipelines to help provide a better assessment of architectural issues. There are numerous architecture analysis tools that can be integrated into CI/CD pipelines, for instance, tools for tracking architectural decay and architectural smell detection [16–23], tools for detecting design flaws in the code [24, 25], and tools that can enable model-based DevOps and consistency checking [26–28]. With the aim to reduce irrelevant documentation in continuous development settings, the work in [29] identifies architecture descriptions as boundary objects, i.e., as objects that are able to adapt to actual and current needs and constraints and yet robust enough to maintain common identity. The work proposes an approach to identify boundary objects and three practices to manage systems engineering artifacts in industry. Then, the work in [30] proposes to treat architectural interfaces as boundary objects, since they determine and regulate the exchange of information between components, subsystems, and systems, which are often developed by multiple teams. The work also identifies how interfaces are changed, what categories of interfaces exist along these dimensions, how categories of interfaces change over time, and, finally, practices to manage the different categories of interfaces in largescale agile development. Since the role of architecture changes in current iterative and agile development methods, there is a need for continuous architecture evaluation as a means to provide useful feedback during the development of continuously evolving systems [31]. After investigating the use of the architecture tradeoff analysis method (ATAM) [32]

Continuous Alignment Between Architecture and Development

73

in a continuous development setting, the authors propose principles of continuous evaluation and evolution of architecture, together with a roadmap for future research. To achieve the vision, researchers may in the future make use of existing methods from fields like model-driven engineering (MDE, e.g., [33, 34]) or process mining [35], as we discuss toward the end of this chapter. In line with this work, the work in [36] proposes a method to align and realign business goals and the architecture in just-in-time architecting contexts.

Motivational Examples To motivate the problem addressed in this chapter and to provide deeper background and relation to state-of-the-art research, this section comprises descriptions of practical experiences from industrial settings.

Practical Experience at Google Software architecture modeling techniques, such as Unified Modeling Languages (UML) [37] and quality attribute (QA) analysis, have been researched and taught in classrooms for decades, but their applications in practice are very limited [38– 41]. Developers usually consider that creating a “big picture view” of a large-scale system is difficult, not worthwhile, or not feasible [38, 39]. Researchers have revealed that applying partial models within a limited context could improve software quality [38, 41], but there is no guidance on how to leverage these models, especially when the designer proposes architecture-level changes for large-scale legacy systems and needs to analyze and justify their decisions. These were the challenges faced by the tech leads (TLs) of Monarch [42]—a large-scale in-memory database and one of the largest infrastructure software systems within Google. Given the rapid growth in the number of users and workloads, the tech leads of Monarch proposed to redesign the system and adopt a more modularized architecture. Since the impact of the proposed change will be significant, influencing most of the existing infrastructure, the TLs found it difficult to convince the team that the new design was worthwhile: the number of servers will increase, and the team was concerned that several key qualities, especially performance, will be degraded, but they had no way to precisely estimate the severity of the impact and the trade-offs among maintainability, performance, availability, etc. As a monitoring infrastructure, Monarch not only collects performance and reliability data for other major Google products, but the performance data of Monarch itself, such as response time, are also continuously collected and monitored in real time. Each quality attribute has a corresponding service-level objective (SLO). For example, the availability SLO for the query function of Monarch is defined as “The system returns correct responses for .≥ 99.99% of the queries”. If the real-time

74

D. Taibi et al.

performance data reveal that from time to time, the SLOs cannot be met, it means that the performance of the system is at risk. Although the performance measures are available for the current system, the challenge is that there was no way for the team to rigorously assess how performance and other important quality attributes will be impacted in the proposed new design or what the trade-offs are among maintainability, availability, and resource efficiency. In a recent paper [43], the authors report on their experiences of addressing this challenge: using lightweight, abstract, and partial models to efficiently analyze and communicate trade-offs among multiple quality attributes. They combined multiple lightweight models that are abstract enough to model specific scenarios only and analyzed the proposed re-architecting plan of Monarch in a rigorous and precise way. As illustrated in Fig. 1, they first identified a few critical user journeys (CUJs) and associated quality attributes (QAs) that would be affected by the proposed new architecture, such as performance and availability. They further modeled each attribute using a quality attribute scenario (QAS), specifying the concrete components (binaries) involved, the responses expected, and the target measures of these responses. After that, they used UML component diagrams to model the static structure among these components and used UML sequence diagrams to model their run-time behaviors so that the remote procedure call (RPC) routes needed to accomplish these scenarios could be visualized. The authors modeled the current architecture design and the proposed new design using both component and sequence diagrams, respectively. These models enabled comparisons between the current design and the proposed new design: in order to realize the same user scenario, how many new components need to be added/changed, how are the RPCs routed through these components in the current design, and how the routes would change in the proposed new design. These sequence diagrams provided the concrete visualization for the team to assess how latency and availability would be affected in the proposed new design. The component diagrams clearly revealed how existing components would be decoupled to simplify the APIs and improve maintainability. The diagrams in Fig. 1 were all created manually based on the selected CUJs. There are a number of existing tools designed to recover high-level models from source code [45–48], but these tools tend to create a big picture for the overall system, and the results from different tools are drastically different [49]. Given that Monarch is a large system with hundreds of components, the recovered model will not be specific to the selected CUJs. There is no existing work, to the best of our knowledge, that can recover a partial architecture model for selected functions only. With the quantitative target measures in QAS, and the abovementioned comparison, the authors of [43] were able to tell, in the new design, exactly how many more binary instances will be added on the RPC paths. Comparing with the actual performance data collected from the current infrastructure, the team recognized that the system can still meet performance service-level objectives (SLOs) [50] with the new design, even though the query latency will increase to some extent. On the other hand, in the new design, availability and maintainability would be greatly improved,

Component Diagram

Component Diagram

Sequence Diagram

Sequence Diagram

Proposed New Architecture

Fig. 1 Google experience: supporting re-architecting decisions using scenario-based design models based on [44]

User Scenario + Quality Attribute Scenarios

Current Architecture

Trade-off Analysis

Continuous Alignment Between Architecture and Development 75

76

D. Taibi et al.

addressing the most prominent risks that are hindering the evolution of the overall system. The scenario-based lightweight modeling and trade-off analysis techniques were well-received by the team. The comments the researchers received indicated that the team agreed that UML component and sequence diagrams are effective ways to illustrate static and run-time designs, while most developers were not familiar with, or even aware of, these models. The Monarch tech leads recommended the process to other Monarch projects and other Google product teams. A few days after they presented and shared this process, two new Monarch projects adopted component and sequence diagrams in their design documents, and two other Google teams started to adopt the modeling process as well. In the domain of software performance engineering, using UML and other models to analyze and predict software performance has been widely studied [51– 60]. Most of these works focus on assessing performance only, rather than analyzing the trade-offs among multiple quality attributes. The time and effort needed to learn and create more sophisticated performance models was another obstacle. It wasn’t clear to the Monarch team how to create a performance model for the existing design to start with. Deriving an effective performance model from source code automatically could be a new research direction. To summarize, the following two factors ensured the success of this process: First, making concise models for concrete scenarios only. Since a QAS lists the specific components involved in these scenarios, the models only need to include these components. It is critical that these models are high-level abstractions to avoid the unnecessary complexity of modeling the overall “big picture” [38, 39, 41]. Second, making precise model by separating static and run-time views. It has been well-recognized that a single diagram cannot precisely capture the multiple structures within a software system [14, 61]: static module structures, run-time component-connector structures, and allocation structures. In practice, however, designers usually model both static and run-time views in one box-and-line-style diagram, making it impossible to avoid ambiguity. With these high-level design models, the whole team, including senior and junior developers, as well as managers, could understand the differences between the existing and newly proposed designs, even though most of them do not have extensive prior knowledge of software modeling. This inspiring experience indicates the possibility and benefits of integrating abstract, partial, lightweight models, quality attribute analysis, and trade-off analysis into software development processes, rather than creating a complex full model before the implementation started. Since this process starts from user scenarios, it can be applied whenever a new feature is added, a refactoring plan is proposed, or a quality attribute needs to be evaluated.

Continuous Alignment Between Architecture and Development

77

Practical Experience in Various Domains The work in [62] discusses the software development challenges of Volvo cars and, specifically, the architecture degradation problem [63] caused primarily by (i) important directives of the as-intended architecture that are violated due to time constraints, mistakes, and misunderstanding, (ii) architectural choices of the asintended architecture that are based on assumptions (might be implicit) that then later identified as imprecise or wrong, and/or (iii) architectural choices of the asintended architecture that are made under uncertainty and then later judged as suboptimal. The work in [62] also identifies “just-in-time architecture” or agile architecting as a potential solution to these challenges: in fact, stakeholders different from the architects, such as developers, are enabled to improve the architecture, for instance, by fixing wrong assumptions or deliberately postponing decisions. On the one side, potential misalignments between software architecture design and implementation are avoided since decisions are taken later, during implementation, and when there is a better understanding. On the other side, the software architecture design is a living object that can be updated according to those decisions taken during development that introduce discrepancies between the architecture description and the implementation. Similar work in the avionics domain [64] has been instrumental in leading to the need to work on common architectural standards to allow for the rapid integration of real-time safety-critical applications derived from different vendors [65]. In many cases, this requires a continuous refactoring of legacy systems in complex environments. Unlike automotive “year model” revisions, in aviation systems, there are regular upgrades throughout the airframe life cycle (leveraging the existing investment rather than procuring a new model every few years). Similar situations exist within the domains of industrial controls and complex real-time systems that have a decade or more of life for the physical system with the need to upgrade software on a much faster timeline. One of the challenges in this environment is that the agile method in fast-moving commercial software spaces does not typically align well with the systems engineering methods that require complex requirements traces (e.g., for system safety, airworthiness, HIPAA, or other stakeholder concerns). The works in [66, 67] testify that also in the healthcare domain, where typically the adopted development methodologies are traditional, there is often the need of mixing plan-driven/predictive and agile methodologies. These two works report the experience of an international and heterogeneous team of scientists who, during the first wave of the COVID-19 pandemic, collaborated on the production of a mechanical ventilator for intensive care units (MVM) [68]. Among other guidelines and lessons learned, they highlight that there was the need to integrate plan-driven/predictive processes with agile practices, to combine rigorousness with efficiency and flexibility [67]. About the role of architecture, they highlight also that a clear identification of components and interfaces might promote agility since it helps development teams to work independently and in parallel. However, it can be difficult to define all the components upfront; also, the architecture can become

78

D. Taibi et al.

obsolete and misaligned with the implementation. Therefore, it becomes important to enrich the upfront architecture with aspects emerging during development and update the architecture accordingly. Establishing a community of practices (CoP)1 may enable architects to reason about changes and to reduce assumptions that can become inconsistencies.

Vision and Proposed Solution One of the promises of explicitly creating and maintaining software architectural models of large systems is that it can help to guide development practices toward desired goals and serve as a check if the system starts to diverge from the original plan, say, by permitting components to interact without using the expected interfaces. The problem of trying to keep the actual design of a system consistent with its planned design has been the subject of a lot of research over the last few decades; while many compelling techniques and ideas have emerged from this line of investigation, adoption of them by industry has remained elusive. There are many reasons for this, but the main ones seem to be that while managers are often open to the idea of explicitly modeling the software architecture of systems, the reality of adopting such an approach has rarely been popular with front-line developers, who often find the tools heavyweight, intrusive to their preferred workflows, unduly constraining, hard to use, uninformative, or having relatively little value relative to the work required to use them. If the tooling required to extract and model software architectural relationships can be made lightweight and non-intrusive, and if it can be integrated into developer workflows, then there is a much better chance of adoption. The growing availability and adoption of tools such as distributed version control, issue tracking, and CI/CD tooling presents a tremendous opportunity: these tools can instrument developer processes to verify not only properties of the source code but also of other development artifacts and system properties. Since these tools are already in widespread use, adding instrumentation to them in order to extract models of the system under development is within reach and comes at a relatively low cost. This seems to be the future of supporting architectural compliance as development progresses. Previously, researchers asked managers and developers to adopt a large (research) tool wholesale—that was unrealistic, especially when the developers often saw a little concrete benefit to this added a heavy layer of documentation and technical obligations. But developers do see benefits in this new generation of tools, and building validation layers by instrumenting these tools is non-invasive, is relatively cheap to implement, and therefore has a much higher chance of success.

1 https://www.scaledagileframework.com/communities-of-practice/,

accessed 2023-02-28

Continuous Alignment Between Architecture and Development

79

Legend CI / CD Pipeline

Code

Build artefact

Process

Activity Deviations found?

Architecture models

Create model from scratch Yes

No

OK Decision gateway

Analysis of deviations Deviation 1

Inputs and outputs

Deviation 2 Deviation 3 Change code or models

Flow: control and data Alternative flow

Fig. 2 Overview of the envisioned method for continuous alignment. Conventional CI/CD pipeline with inputs and outputs is shown in yellow (simplified); new proposed parts are shown in blue

An overview of the proposed approach is shown in Fig. 2. As can be seen, the conventional CI/CD pipeline operates as before, while mechanisms for achieving alignment with architectural models are add-ons. To achieve these goals while being minimally invasive to existing practices, some assumptions must be made, some requirements must be fulfilled, and some desirable functionality should be provided. We list such aspects below, including manual and automated ones. Fulfilling the complete vision includes finding solutions for all automated steps; however, to start making headway toward the vision while realizing some of the benefits already, relying on more manual steps initially is more realistic. 1. The continuous alignment solution should be seamlessly integrated with the conventional CI/CD pipeline operation (shown with yellow elements and solid black arrows in Fig. 2). 2. The models may cover only a subset of the code. Ideally, mechanisms will allow focusing on coherent subsets, such as a set of critical user journeys (CUJs), so that each recovered architecture model can be created for the codebase implementing the CUJs only. As we demonstrated in the Monarch experience, this way we can generate precise and concise models to avoid complexity. In practice, a CI/CD pipeline is often set up with a user story, a feature description, or a quality attribute scenario (QAS) as a starting point. From such a point, architects could manually model a feature using a CUJ. If this feature has high performance or other quality requirements, the architects could also model its quality attribute scenarios. For a system with many CUJs, we may end up having a number of high-level models. We can use the CUJs as indexes to these models and create tools to maintain the consistency of these models.

80

D. Taibi et al.

3. Static and dynamic model creation and recovery. In order to communicate the design of a CUJ, the designer could create minimal static models (either a component diagram or a class diagram) and a minimal sequence diagram to analyze run-time-related quality attributes. It is important that these models are simple enough to represent the entities involved in the CUJ only, just sufficient to communicate the key design decisions. 4. Managing scenarios and models. All the models and scenarios should be indexed and assigned a ticket number. Implicit or explicit mechanisms for linking code and models are required, so that the relation between existing models and changed code can be traced, which in turn enables further traceability analysis. 5. After the scenarios are implemented, the actual static models and run-time models could be recovered automatically from execution logs. The automated method needs to be able to operate in a setting where architecture models do not yet exist; in that case, initial models are created from scratch, indicated by the dashed black arrow. Approaches from architecture recovery form a solid base for this functionality; however, current tooling (such as “Architecture 101”) is not designed for use in a continuous setting, and adaptation is most likely required. 6. Automatically created or updated models can be compared with earlier design model, regardless of whether they were created manually or automatically from an earlier version of the code. For measuring the alignment (or compliance or conformance) between models and code, suitable mechanisms need to be available. Measurements can be taken directly between models and code or between design models and automatically derived models. If deviations are found, it must be possible to display these in a user-understandable way. Particular challenges might arise from the goal to detect architectural decay and accumulation of technical debt, even when they occur very gradually over time. 7. If there are deviations and alignment is not achieved, the reason may be that the desired architecture is reflected in the models and hence the code needs to be changed or that the desired architecture is reflected in the code, i.e., the models are outdated and need to be updated. 8. Leveraging design models for design improvement. If part of the design needs to be refactored, as demonstrated in the Google experience, it is possible to use these static and dynamic models to analyze the trade-offs of the proposed refactoring. Changes to models or code may even be proposed by the tooling, e.g., by a suitable generative artificial intelligence. 9. For instances of possibly desirable functionality, one example may be to collect metric data for quality attributes of the system under development during runs of the CI/CD pipeline and compare the measured data with the requirements. QASs may also be related to the sustainability goals of an organization, such that unanticipated negative impact (e.g., a large increase in electricity consumption) of a change in code could be detected as deviation. Further, it might be useful to connect automated quality gates in the pipeline with measured alignment, such that non-compliance results in code being blocked from automatic deployment to production.

Continuous Alignment Between Architecture and Development

81

The above list covers a variety of desired features and properties of mechanisms for achieving architectural alignment. It is sorted to reflect an order in which these aspects could be investigated, which in part implies starting with less ambitious, manual points and moving to more ambitious automatic approaches subsequently. To achieve the vision, part of the methods and tooling may be based on or inspired by approaches from model-driven engineering (MDE [33, 34]) or process mining [35]. Architecture recovery can be seen as a useful mechanism from MDE. Also, if code generation is used during development, it may be easier to draw the link back from newly written code to models. As an alternative from the field of business process management, process mining can roughly be understood as data mining methods with a business process lens. Process mining includes among others methods for process discovery (creating process models from execution data; coarsely related to architecture recovery) and conformance checking [69, 70] (comparing execution log data with existing models). Process mining has been applied to system logs without a well-defined business process [71, 72]. One class of conformance checking approaches is rule-based, i.e., conformance of an execution log is assessed concerning adherence to rules. Similarly, the adherence of the code with architectural constraints could be checked by the envisioned method. Generally speaking, the mechanisms for realizing the vision of continuous alignment could be inspired by various process mining mechanisms, some of which are performed as post hoc analysis, while others are designed for online use.

Summary and Outlook In this chapter, we outlined a vision for the continuous alignment of architecture models and code. After describing the motivation and background, we covered practical examples from Google and settings from the automotive, avionics, and healthcare domains. In summary, often architecture models are either created initially and outdated quickly, or they are not created at all. However, based on the practical examples, we observe that much can be achieved by creating architecture models for particular contexts and tasks, like refactoring. In the previous section, we presented a vision for integrating methods for continuous architecture alignment into CI/CD pipelines. Aspects of the envisaged solution include manual and/or automatic model creation and updates, traceability, focus, critical user journeys, and quality attributes and even AI approaches for automatic improvements. We sorted these and more aspects into an ordered list of steps, starting with more manual steps that presumably can already deliver some anticipated benefits. Achieving all aspects of the vision would likely take a number of years, and we encourage other members of the software architecture research community to join us in this endeavor.

82

D. Taibi et al.

Acknowledgments The work of P. Pelliccione was supported by the Centre of Excellence on Connected, Geo-Localized, and Cybersecure Vehicles (EX-Emerge), funded by the Italian government under CIPE resolution n. 70/2017 (Aug. 7, 2017).

References 1. Bass, L., Weber, I., Zhu, L.: DevOps: A Software Architect’s Perspective. Addison-Wesley Professional, Reading (2015) 2. Galster, M., Mirakhorli, M., Medvidovic, N.: Bringing architecture thinking into developers’ daily activities. SIGSOFT Softw. Eng. Notes 41(6), 24–26 (2017). Available: https://doi.org/ 10.1145/3011286.3011290 3. Mehrpour, S., LaToza, T.D., Sarvari, H.: RulePad: interactive authoring of checkable design rules. In: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ACM, New York (2020). Available: https://doi.org/10.1145%2F3368089.3409751 4. Lasnier, G., Zalila, B., Pautet, L., Hugues, J.: Ocarina: An Environment for AADL Models Analysis and Automatic Code Generation for High Integrity Applications, pp. 237–250 (2009). Available: https://doi.org/10.1007%2F978-3-642-01924-1_17 5. Garcia, J., Mirakhorli, M., Xiao, L., Zhao, Y., Mujhid, I., Pham, K., Okutan, A., Malek, S., Kazman, R., Cai, Y., Medvidovi´c, N.: Constructing a shared infrastructure for software architecture analysis and maintenance. In: 2021 IEEE 18th International Conference on Software Architecture (ICSA), pp. 150–161 (2021) 6. Mirakhorli, M., Shin, Y., Cleland-Huang, J., Çinar, M.: A tactic-centric approach for automating traceability of quality concerns. In: Glinz, M., Murphy, G.C., Pezzè, M. (eds.) 34th International Conference on Software Engineering, ICSE 2012, June 2–9, 2012, Zurich, Switzerland, pp. 639–649. IEEE Computer Society, New York (2012). Available: https://doi. org/10.1109/ICSE.2012.6227153 7. Mirakhorli, M., Fakhry, A., Grechko, A., Wieloch, M., Cleland-Huang, J.: Archie: a tool for detecting, monitoring, and preserving architecturally significant code. In: Cheung, S., Orso, A., Storey, M.D. (eds.) Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, (FSE-22), Hong Kong, China, November 16–22, 2014, pp. 739–742 (2014). Available: https://doi.org/10.1145/2635868.2661671 8. LaToza, T.D.: Connecting design to code. IEEE Softw. 40(01), 94–97 (2023) 9. Feiler, P.H., Gluch, D.P., Hudak, J.J.: The Architecture Analysis and Design Language (AADL): An Introduction. Carnegie-Mellon University Pittsburgh PA Software Engineering Institute, New York. Tech. Rep. (2006) 10. Chen, H.-M., Kazman, R., Haziyev, S., Kropov, V., Chtchourov, D.: Architectural support for devops in a neo-metropolis bdaas platform. In: 2015 IEEE 34th Symposium on Reliable Distributed Systems Workshop (SRDSW), pp. 25–30 (2015) 11. Pahl, C., Jamshidi, P., Zimmermann, O.: Architectural principles for cloud software. ACM Trans. Internet Technol. 18(2), 1–23 (2018). Available: https://doi.org/10.1145/3104028 12. Taibi, D., Lenarduzzi, V., Pahl, C.: Continuous architecting with microservices and devops: a systematic mapping study. In: Muñoz, V.M., Ferguson, D., Helfert, M., Pahl, C. (eds.) Cloud Computing and Services Science, pp. 126–151. Springer International Publishing, Cham (2019) 13. Shahin, M., Babar, M.A., Zhu, L.: The intersection of continuous deployment and architecting process: Practitioners’ perspectives. In: Proceedings of the 10th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (2016) 14. Bass, L., Clements, P., Kazman, R.: Software Architecture in Practice, 4th edn. AddisonWesley, Reading (2021)

Continuous Alignment Between Architecture and Development

83

15. Helwani, F., Jahi´c, J.: Acia: a methodology for identification of architectural design patterns that support continuous integration based on continuous assessment. In: 2022 IEEE 19th International Conference on Software Architecture Companion (ICSA-C), pp. 198–205 (2022) 16. Garcia, J., Popescu, D., Edwards, G., Medvidovic, N.: Toward a catalogue of architectural bad smells. In: QoSA ’09: Proceeding of the 5th International Conference on Quality of Software Architectures (2009) 17. Garcia, J., Popescu, D., Edwards, G., Nenad, M.: Identifying Architectural Bad Smells. In: 13th European Conference on Software Maintenance and Reengineering (2009) 18. Le, D.M., Link, D., Shahbazian, A., Medvidovic, N.: An empirical study of architectural decay in open-source software. In: 2018 IEEE International Conference on Software Architecture (ICSA). IEEE, New York, pp. 176–17609 (2018) 19. Garcia, J.: A Unified Framework for Studying Architectural Decay of Software Systems,” Ph.D. dissertation. University of Southern California, Southern California (2014) 20. Mo, R., Cai, Y., Kazman, R., Xiao, L.: Hotspot patterns: the formal definition and automatic detection of recurring high-maintenance architecture issues. In: Proceedings of the 12th Working IEEE/IFIP International Conference on Software Architecture (2015) 21. Tokuda, L., Batory, D.: Evolving object-oriented designs with refactorings. Autom. Softw. Eng. 8(1), 89–120 (2001). Available: http://link.springer.com/article/10.1023/A:1008715808855 22. Philipps, J., Rumpe, B.: Refinement of information flow architectures. In: Proceedings of the First IEEE International Conference on Formal Engineering Methods 1997, pp. 203–212. IEEE, New York (1997) 23. Mens, T., Tourwé, T.: A survey of software refactoring. IEEE Trans. Softw. Eng. 30(2), 126– 139 (2004) 24. Tuma, K., Scandariato, R., Balliu, M.: Flaws in flows: unveiling design flaws via information flow analysis. In: 2019 IEEE International Conference on Software Architecture (ICSA), pp. 191–200 (2019) 25. Mendoza, C., Garcés, K., Casallas, R., Bocanegra, J.: Detecting architectural issues during the continuous integration pipeline. In: 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C), pp. 589–597 (2019) 26. Santos, J.C.S., Suloglu, S., Ye, J., Mirakhorli, M.: Towards an automated approach for detecting architectural weaknesses in critical systems. In: ICSE ’20: 42nd International Conference on Software Engineering, Workshops, Seoul, Republic of Korea, 27 June–19 July, 2020, pp. 250– 253. ACM, New York (2020). Available: https://doi.org/10.1145/3387940.3392222 27. Guerriero, M., Ciavotta, M., Gibilisco, G.P., Ardagna, D.: A model-driven DevOps framework for QoS-aware cloud applications. In: 2015 17th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing (SYNASC), pp. 345–351 (2015) 28. Ivkovic, I., Kontogiannis, K.: A Framework for Software Architecture Refactoring Using Model Transformations and Semantic Annotations. In: Proceedings of the Conference on Software Maintenance and Reengineering, ser. CSMR ’06, pp. 135–144. IEEE Computer Society, Washington, DC, USA (2006). Available: http://dl.acm.org/citation.cfm?id=1116163. 1116398 29. Wohlrab, R., Pelliccione, P., Knauss, E., Larsson, M.: Boundary objects and their use in agile systems engineering. J. Software: Evol. Process 31(5), e2166 (2019). Available: https:// onlinelibrary.wiley.com/doi/abs/10.1002/smr.2166 30. Wohlrab, R., Pelliccione, P., Knauss, E., Heldal, R.: On interfaces to support agile architecting in automotive: An exploratory case study. In: 2019 IEEE International Conference on Software Architecture (ICSA), pp. 161–170 (2019) 31. Ågren, S.M., Knauss, E., Heldal, R., Pelliccione, P., Alminger, A., Antonsson, M., Karlkvist, T., Lindeborg, A.: Architecture evaluation in continuous development. J. Syst. Softw. 184, 111111 (2022). Available: https://www.sciencedirect.com/science/article/pii/ S0164121221002089 32. Kazman, R., Klein, M., Clements, P.: ATAM: Method for Architecture Evaluation. CarnegieMellon Univ Pittsburgh PA Software Engineering Inst. Tech. Rep. (2000)

84

D. Taibi et al.

33. Abrahão, S., Bourdeleau, F., Cheng, B., Kokaly, S., Paige, R., Stöerrle, H., Whittle, J.: User experience for model-driven engineering: challenges and future directions. In: 2017 ACM/IEEE 20th International Conference on Model Driven Engineering Languages and Systems (MODELS), pp. 229–236 (2017) 34. Schmidt, D.: Guest editor’s introduction: model-driven engineering. Computer 39(2), 25–31 (2006) 35. van der Aalst, W.M.P.: Process Mining: Data Science in Action. Springer, Heidelberg (2016) 36. Bucaioni, A., Pelliccione, P., Wohlrab, R.: Aligning architecture with business goals in the automotive domain. In: 2021 IEEE 18th International Conference on Software Architecture (ICSA), pp. 126–137 (2021) 37. Booch, G., Rumbaugh, J., Jacobson, I.: Unified Modeling Language User Guide, 2nd edn. Addison-Wesley, Reading (2005) 38. Júnior, E., Farias, K., Silva, B.: A survey on the use of UML in the brazilian industry. In: Proceedings of the XXXV Brazilian Symposium on Software Engineering, ser. SBES ’21, pp. 275–284. Association for Computing Machinery, New York (2021). Available: https://doi.org/ 10.1145/3474624.3474632 39. Guimaraes, E., Manica, M., Gonçales, L., Bischoff, V., da Silva, B., Farias, K.: On the UML use in Brazilian Industry: A State of the Practice Survey (2018) 40. Störrle, H.: How are conceptual models used in industrial software development? a descriptive survey. In: Proceedings of the 21st International Conference on Evaluation and Assessment in Software Engineering, ser. EASE’17, pp. 160–169. Association for Computing Machinery, New York (2017). Available: https://doi.org/10.1145/3084226.3084256 41. Fernández-Sáez, A.M., Chaudron, M.R., Genero, M.: An industrial case study on the use of UML in software maintenance and its perceived benefits and hurdles. Empirical Softw. Engg. 23(6), 3281–3345 (2018). Available: https://doi.org/10.1007/s10664-018-9599-4 42. Adams, C., Alonso, L., Atkin, B., Banning, J.P., Bhola, S., Buskens, R., Chen, M., Chen, X., Chung, Y., Jia, Q., Sakharov, N., Talbot, G.T., Tart, A.J., Taylor, N. (eds.) Monarch: Google’s Planet-Scale In-Memory Time Series Database (2020) 43. Jia, Q., Cai, Y., Çakmak, O.: A model-based, quality attribute-guided architecture re-design process at google. In: 2023 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) (2023) 44. Jia, Q., Cai, Y., Çakmak, O.C.: A model-based, quality attribute-guided architecture redesign process at Google. In: 2023 IEEE/ACM 45th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP), Melbourne, Australia, pp. 61–73 (2023). https://doi.org/10.1109/ICSE-SEIP58684.2023.00011 45. Tzerpos, V., Holt, R.C.: ACCD: an algorithm for comprehension-driven clustering. In: Proceedings of the Seventh Working Conference on Reverse Engineering, pp. 258–267 (2000) 46. Mancoridis, S., Mitchell, B.S., Chen, Y., Gansner, E.R." Bunch: a clustering tool for the recovery and maintenance of software system structures. in Proceedings of the IEEE International Conference Software Maintenance, pp. 50–59 (1999) 47. Maqbool, O., Babri, H.A.: The weighted combined algorithm: a linkage algorithm for software clustering. In: Proceedings of the Eighth European Conference on Software Maintenance and Reengineering, 2004 (CSMR 2004), pp. 15–24 (2004) 48. Andritsos, P., Tzerpos, V.: Information-theoretic software clustering. IEEE Trans. Softw. Eng. 31(2), 150–165 (2005) 49. Lutellier, T., Chollak, D., Garcia, J., Tan, L., Rayside, D., Medvidovi´c, N., Kroeger, R.: Measuring the impact of code dependencies on software architecture recovery techniques. IEEE Trans. Softw. Eng. 44(2), 159–181 (2018) 50. Rastegari, Y., Shams, F.: Optimal decomposition of service level objectives into policy assertions. Sci. World J. 2015, 465074 (2015) 51. Smith, C.U.: Performance engineering of software systems, ser. Software Engineering Institute Series in Software Engineering. Addison-Wesley, Reading (1990) 52. Smith, C.U., Lladó, C.M., Cortellessa, V., Marco, A.D., Williams, L.G.: From UML models to software performance results: an SPE process based on XML interchange formats. In:

Continuous Alignment Between Architecture and Development

85

Proceedings of the Fifth International Workshop on Software and Performance, WOSP 2005, Palma, Illes Balears, Spain, July 12–14, 2005, pp. 87–98. ACM, New York (2005). Available: https://doi.org/10.1145/1071021.1071030 53. Melià, M., Lladó, C.M., Smith, C.U., Puigjaner, R.: Experimentation and output interchange for petri net models. In: Avritzer, A., Weyuker, E.J., Woodside, C.M. (eds.) Proceedings of the 7th International Workshop on Software and Performance, WOSP 2008, Princeton, NJ, USA, June 23–26, 2008, pp. 133–138. ACM, New York (2008). Available: https://doi.org/10.1145/ 1383559.1383576 54. Rapp, M., Scheerer, M., Reussner, R.: Design-time performability optimization of runtime adaptation strategies. In: Companion of the 2022 ACM/SPEC International Conference on Performance Engineering, ser. ICPE ’22, pp. 113–120. Association for Computing Machinery, New York (2022). Available: https://doi.org/10.1145/3491204.3527471 55. Requeno, J.I., Gascón, I., Merseguer, J.: Towards the performance analysis of Apache Tez applications. In: Companion of the 2018 ACM/SPEC International Conference on Performance Engineering, ser. ICPE ’18, pp. 147–152. Association for Computing Machinery, New York (2018). Available: https://doi.org/10.1145/3185768.3186284 56. Petriu, D.C., Woodside, C.M., Petriu, D.B., Xu, J., Israr, T., Georg, G., France, R., Bieman, J.M., Houmb, S.H., Jürjens, J.: Performance analysis of security aspects in UML models. In: Proceedings of the 6th International Workshop on Software and Performance, ser. WOSP ’07, pp. 91–102. Association for Computing Machinery, New York (2007). Available: https://doi. org/10.1145/1216993.1217010 57. Canevet, C., Gilmore, S., Hillston, J., Kloul, L., Stevens, P.: Analysing UML 2.0 activity diagrams in the software performance engineering process. SIGSOFT Softw. Eng. Notes 29(1), 74–78 (2004). Available: https://doi.org/10.1145/974043.974055 58. Petriu, D.: Integrating the analysis of multiple non-functional properties in model-driven engineering. Softw. Syst. Model. 20, 12 (2021) 59. Altamimi, T., Petriu, D.C.: Incremental change propagation from UML software models to LQN performance models. In: Proceedings of the 27th Annual International Conference on Computer Science and Software Engineering, ser. CASCON ’17, pp. 120–131. IBM Corporation, USA (2017) 60. Li, C., Altamimi, T., Zargar, M., Casale, G., Petriu, D.: Tulsa: a tool for transforming UML to layered queueing networks for performance analysis of data intensive applications. In: Proceedings of the Quantitative Evaluation of Systems: 14th International Conference, QEST 2017, Berlin, Germany, September 5–7, 2017, pp. 295–299. Springer International Publishing, Berlin (2017) 61. Clements, P., Bachmann, F., Bass, L., Garlan, D., Ivers, J., Little, R., Merson, P., Nord, R., Stafford, J.: Documenting Software Architectures: Views and Beyond. The 2nd Edition. Addison-Wesley, Reading (2010) 62. Pelliccione, P., Knauss, E., Heldal, R., Magnus Ågren, S., Mallozzi, P., Alminger, A., Borgentun, D.: Automotive architecture framework: the experience of volvo cars. J. Syst. Archit. 77, 83–100 (2017). Available: https://www.sciencedirect.com/science/article/pii/ S1383762117300954 63. Wohlrab, R., Eliasson, U., Pelliccione, P., Heldal, R.: Improving the consistency and usefulness of architecture descriptions: guidelines for architects. In: 2019 IEEE International Conference on Software Architecture (ICSA), pp. 151–160. IEEE, New York (2019) 64. Barbacci, M., Clements, P., Lattanze, A., Northrop, L., Wood, W.: Using the Architecture Tradeoff Analysis Methodsm (ATAMSM) to Evaluate the Software Architecture for a Product Line of Avionics Systems: A Case Study. Tech. Rep. (2003) 65. Hyatt, L., Stough, J., Hammond, R.A., Kellow, C., DuBois, T.: A holistic approach to open systems architecture for army aviation. In: Vertical Flight Society’s 75th Annual Forum and Technology Display, Philadelphia, PA, USA (2019)

86

D. Taibi et al.

66. Bombarda, A., Bonfanti, S., Galbiati, C., Gargantini, A., Pelliccione, P., Riccobene, E., Wada, M.: Lessons learned from the development of a mechanical ventilator for covid-19. In: 2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE), pp. 24–35 (2021) 67. Bombarda, A., Bonfanti, S., Galbiati, C., Gargantini, A., Pelliccione, P., Riccobene, E., Wada, M.: Guidelines for the development of a critical software under emergency. Inf. Softw. Technol. 152, 107061 (2022). Available: https://www.sciencedirect.com/science/article/pii/ S0950584922001707 68. Abba, A. et al.: The novel mechanical ventilator milano for the covid-19 pandemic. Phys. Fluids 33(3), 037122 (2021). [Online]. https://doi.org/10.1063/5.0044445 69. Carmona, J., van Dongen, B., Solti, A., Weidlich, M., Conformance Checking: Relating Processes and Models. Springer International Publishing, Berlin (2018) 70. Weber, I., Rogge-Solti, A., Li, C., Mendling, J.: CCaaS: online conformance checking as a service. In: BPM’15: International Conference on Business Process Management, demo track, Innsbruck, Austria (2015) 71. Xu, X., Zhu, L., Weber, I., Bass, L., Sun, D.: POD-Diagnosis: Error diagnosis of sporadic operations on cloud applications. In: DSN’14: 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, Atlanta, GA, USA, pp. 252–263 (2014) 72. Pecchia, A., Weber, I., Cinque, M., Ma, Y.: Discovering process models for the analysis of application failures under uncertainty of event logs. Knowl.-Based Syst. 189, 105054 (2020)

An Empirical Basis for Software Architecture Research Rick Kazman

, Roberto Tonelli

, and Cesare Pautasso

Abstract Despite the clear need for and importance of performing empirical studies as part of software architecture research, there is still a lack of curated, standardized, clean, well-maintained, documented, easily accessible, reusable, and shared datasets. In this chapter, we provide an overview of the problems, of the motivations, and of the opportunities currently related to mining and sharing datasets for researchers in software architecture. We first explore and describe which artifacts should be included into such datasets, such as code, documentation, and requirements, but also including other architecturally relevant artifacts, such as architectural decision records, models, and other kinds of documentation. This information can be complemented with revision history logs, social metadata, and email or chat discussion archives. The availability of such datasets would enable not only architectural reconstruction studies but would also help to catalyze broader and more ambitious program of empirical studies in software architecture research.

Introduction Shared, large-scale datasets are necessary to perform meaningful empirical research and for replications in software architecture research, as they are in most empirically based sciences. These kinds of datasets have become the norm in many fields

R. Kazman () University of Hawaii, Honolulu, HI, USA e-mail: [email protected] R. Tonelli University of Cagliari, Cagliari, Italy e-mail: [email protected] C. Pautasso University of Lugano, Lugano, Switzerland e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 P. Pelliccione et al. (eds.), Software Architecture, https://doi.org/10.1007/978-3-031-36847-9_5

87

88

R. Kazman et al.

of science: engineering, sociology, medicine, political science, and so forth, for example: – – – – –

https://aimi.stanford.edu/shared-datasets https://www.healthit.gov/data/data https://www.earthdata.nasa.gov/eosdis https://www.prio.org/journals/jpr/replicationdata https://researchdata.ntu.edu.sg/dataverse/CEE

The MSR (mining software repositories) and PROMISE (predictor models in software engineering) communities have led the efforts to make software engineering datasets widely accessible. But these efforts have not been focused on software architecture and the specific needs of the software architecture research community. Currently, datasets in software architecture are difficult to acquire, and when they are created, they are typically not updated and maintained. In this chapter, we put forth a vision for addressing this shortcoming, to make datasets more broadly available and accessible. We aspire to fulfill the FAIR principles: https://force11. org/info/the-fair-data-principles/. These principles are: – Findable: datasets should be persistent and easy to find, with a locator (e.g., URI) that never changes, and should be annotated with metadata. – Accessible: it should be easy for humans or computers to access the data, and any metadata, using the locator. – Interoperable: the representation employed should have a broadly accepted and used syntax and semantics. – Reusable: datasets should be sufficiently well described that they can be easily linked with other datasets. In practice, if and when such datasets are created, they are often not shared. And even when they are shared, they are typically in idiosyncratic data formats, often requiring specialized tooling, significant effort, and arcane knowledge, also due to the lack of standardized metadata. All of these represent high barriers to entry, which limits: – The size and scope of empirical studies that can be undertaken – The ease with which researchers can perform replication studies In software architecture research, this situation is particularly dire. Datasets are hard to come by and require significant effort to collect and curate. The little tooling that does exist to help marshal and manipulate the necessary data is complex, requiring a steep learning curve, and poorly supported [18]. And existing datasets typically have impoverished data. For example, in a study on the Software Architecture Documentation Practices and Maturity in Open-Source Software Development [14], “Out of the five maturity levels, most projects did not get past the first level”.

An Empirical Basis for Software Architecture Research

89

In this chapter, we lay out the kinds of artifacts that one might want to collect and share, the existing work in this area, challenges and opportunities, and a path forward.

Motivation Empirical research in software architecture is hard. While there are many rich datasets with information about real-world (mostly open-source) software projects, this data is typically messy. Every software project’s repositories were created for the needs of the project itself, and so they focus on their own stakeholders and their needs. These needs include a record of code and code versions as a series of commits often bundled into releases and a record of issues past and present along with accompanying discussions, email repositories, and other kinds of discussion forums and project documentation. Thus, we have a vast set of data of many types, in idiosyncratic formats, using various tools and repositories, with lots of variability along many dimensions. To make matters worse, the content, format, and structure of the data often change over time as the project adopts new tools and new conventions. Also, as just noted, most of the data that is available is open source. But for research to be relevant—for it to be thorough and balanced—both open- and closedsource projects should be analyzed. Closed-source projects, however, are typically accompanied by proprietary restrictions, making the availability of such datasets rare and making their sharing even rarer. For all of these reasons, replication is difficult in software architecture research, as replication requires precise descriptions of the experimental process and good control over all experimental data. In addition, replications are rare—because it is hard and because of the lack of strong incentives for researchers to create and share datasets. There have been few software engineering and software architecture research outlets that solicit replication studies. Without such incentives, however, these studies will rarely be done. This is a shame and a problem for our research community as it means that the validity and generalizability of the research are harder to argue for. And without replicability, we are not a true engineering discipline; we are not doing science, which is the foundation for engineering. Right now, most studies create and curate their own datasets. In addition to the replicability challenges that this approach fosters, it is a huge collective waste of effort. We feel that we can and we must do better as a research community. The challenges for our community are thus: – How to standardize on data formats for the data and metadata that we share? – How to encourage and incentivize data curation and sharing? – How to move our research from a basis in anecdotes to a basis in theory, patterns, and principles supported by empirical evidence?

90

R. Kazman et al.

Artifacts Complementing traditional software engineering artifacts, such as the ones found in code repositories, revision history logs, issue or bug trackers, as well as email/chat archives, software architecture datasets also include additional types of information sources. Natural language documentation covers both low-level fine-grained code comments, but also additional design artifacts describing the software and its architecture. For example, see the http://www.aosabook.org series. Architectural decisions are also semi-formally documented using architectural decision records (ADRs). These are meant to record the decisions together with their rationale and the considered alternatives. In practice, they can be stored along with the code in the same repository. This offers the opportunity to mine frequently occurring decisions and track how they evolve during a project lifecycle. Such documentation can also be represented more formally in terms of architectural models, for example, using architectural description languages or visual diagrams (e.g., C4, UML). These notations have been around for some time, and some large model collections are starting to appear [20]. Another formal, architectural-level specification concerns software interfaces and APIs. These artifacts abstract the underlying component implementation as they document the operations, events, and properties published to ease the integration of individual reusable components or entire software systems. Many interface description languages have been standardized and widely adopted in practice, resulting in a growing corpus of real-world API descriptions. Requirement specifications are also relevant artifacts when creating or assessing the corresponding software architecture. They cover functional requirements (e.g., use cases, feature models) as well as quality attribute scenarios. We are not aware of any dataset collecting software requirements, neither described using natural language nor with some semi-formal notation. Execution traces are often collected for performance and availability monitoring, legal/auditing purposes, post-mortem analysis, intrusion, and anomaly detection. They can help to recover behavioral design models by process mining techniques. As they often contain sensitive usage traffic details, they tend to be difficult to obtain for individual projects. Anonymized datasets of execution traces collected under real-world usage conditions from multiple systems are still hard to find. While all of the above artifacts focus on describing or prescribing the software architecture, they may need to be complemented by socio-technical observations about the architects and developers involved in their production and consumption. Capturing the rationale of design decision in ADRs may be sufficient to justify them, but information on who collaborated, who reviewed, and who was driving the decision-making process (not to mention who was affected by the decision) may be necessary to properly track the social context in which the architectural decision was born. Such observations may be buried in formal meeting minutes mixed with informal chat logs aggregated from countless communication channels. They may

An Empirical Basis for Software Architecture Research

91

require to hold explicit team retrospective discussions or individual interviews with each stakeholder. They may be revealed on social media or blog posts. And additional outcome artifacts might also be included for analysis purposes, such as performance and scalability benchmark results. A further challenge with these kinds of information is that they seldom come appropriately labeled or validated. If your only source of information comes from examining a software repository, it may be unclear what the ground truth is for that project. Even in the rare cases where the project contains some architectural documentation, how do we know that the information is accurate or up to date? There has, however, been some research effort aimed at producing “ground truth” architectures [4], and this information can be a significant asset for any empirical research.

Related Work Empirical research works in software architecture strongly rely in mining (of) software repositories. We have identified five main topics of interest to the purposes of this chapter in creating an empirical basis for architectural research.

Mining Software Repositories Mining software repositories dates back to the 1980s and 1990s where it began on repositories of commercial software systems and was restricted to private companies. It gained popularity in 2004 when the first edition of the MSR conference was established relying on the—at the time—recent availability of BTS, ITS, and CMS repositories from open-source projects [24]. Several historically well-known resources available are the works of Xie and Hassan (https://sites.google.com/site/asergrp/dmse/) and Hassan [6], and over the last two decades, three main international conferences in the field—MSR, PROMISE, and SANER—have gained popularity and collected hundreds of works in this research area. Given the variety of research works on mining software repositories, we refer the reader to those conferences and focus our attention here on a restricted subset of works and topics of interest for this chapter. In one study [7], the authors performed a survey of the best practices developed in mining software repositories using data from previous MSR conferences and workshops, analyzing and reviewing 117 papers published in the MSR proceedings between 2004 and 2012. From the comments mined from the dataset of papers, the authors extracted four main themes of data acquisition, synthesis, analysis, and replication and identified common recommendations. Nguyen et al. in [15] elaborate on the reasons why practitioners often do not trust results coming from empirical studies due to problems in finding data from

92

R. Kazman et al.

updated repositories, in developing new data extractors, and in integrating data from different platforms. To solve these issues, they present a tool, PANDORA, offering the capabilities to automatically and continuously mine data by means of different tools and platforms which also allowed them to update the results from MSR studies. In [17], the authors show that, contrary to what previous studies on mining software repositories claimed on the correlation between smells and code change/fault-proneness, the empirical evidence of such claims is quite limited due to various reasons such as limited size of the studies, unreliable data obtained from automatic smell detectors versus manually validated smells, missing analysis of the magnitudes of the observed smells, missing analysis on the magnitudes of the defects eventually introduced by smells in classes, lack of the analysis on intrinsically smelly classes, and lack of a temporal relation analysis.

Mining Software Repositories for Software Architecture Stepping forward to more recent studies in the literature, for what concerns software repositories devoted to software architecture analysis, Keim et al. [8] observe the difficulty in finding information in multiple artifacts in software architecture documentation made by informal textual documents and by formal systems’ models. They propose SWATTR, a framework based on agents built for creating trace links between the different artifacts in software architecture documentation (SAD), obtaining very good results compared to baseline approaches. In [18], the authors concentrate on tools that support empirical studies of software projects. The key observation is that literature works continuously propose new tools (Codeface, Codeface4Smells, GrimoireLab, and SmartSHARK) caring little or nothing about design decisions, so that functionalities already implemented in other tools are overlapped or replicated. Based on the observation on good and bad design decisions in these tools, the authors present “Kaiaulu”, a new tool implemented as an R package, as the outcome of the analysis of a set of key design decisions extrapolated by studying previous existing tools, which can be fed by common software development infrastructure logs to produce commonly used software engineering metrics, socio-technical metrics, and a network representation of such data sources. Smells at the architecture level are investigated by Chaniotaki and Sharma in [3]. These occur whenever best practices recommended for software architecture are ignored, causing a negative impact on the software system’s quality attributes. The authors analyzed 750 Java and 361 C# repositories for a total of more than 50 million lines of code to detect architecture smells in the context of the Pareto principle. An interesting result for the purposes of this chapter is that results indicate that the size of the repositories matters since it shows a small negative correlation with the Pareto categories. The authors were able to detect seven kinds of architecture smells, and among the other results, they found that around 45% of the Java repositories follow the Pareto principle and that for C# repositories, the adherence is higher, up to 66%.

An Empirical Basis for Software Architecture Research

93

Tingtin et al. use the mailing list of ArgoUML and Hibernate developers to investigate communications about architecture information in OSS projects [2]. The goals are to understand why such information is communicated, who are the main actors in such communication process, when they do share architecture information, and which specific information is mainly exchanged. Their data collection approach is based on different steps to extract from the mailing list repositories architectural posts, architectural threads, and data items related to the two subject projects. Their results show that the reasons for architecture communications relate to gathering suggestions to guide the work of developers, announcing architectural updates or modifications, or negotiating architectural choices and solutions. They also conclude that there are only few developers (“core” developers) involved in architecture information communication out of the many involved into an OSS project. Interestingly, they find that the frequency of architecture communication is project dependent. Finally, their conclusions suggest that the elements most frequently used in communications are architecture rationale and architecture models. Another interesting study in OSS targets decision-making process [10]. The used repository is once again the Hibernate developers’ mailing list. Authors investigate the motivations behind decision-making, the related artifacts, the approaches adopted along the process, and the trends over time. They mapped linguistic patterns of all identified decisions to three types: information giving, solution proposal, and feature request. They also identified five categories of decisions dominated by two: design decisions and requirement decisions. Furthermore, three types of rationales associated with decision-making are revealed in the work: non-functional requirement, functional requirement, and managing requirement. Another recent approach exploits Web search engines to gather architectural knowledge to get relevant and up-to-date information [22]. This empirical study involved 53 software engineers accustomed to the adoption of Google searches to make design decisions using the attribute-driven design (ADD) method. As its main contributions, the work provides a full overview of the various Web sources of architectural knowledge and their associated concepts and highlights the differences in the effectiveness of Google searches when proceeding with ADD’s architectural steps. A relatively recent tool for mining software repositories, namely, ModelMine, is presented in [19]. ModelMine targets model-based artifacts and designs from open-source repositories. It is best tuned to mine artifacts and commit history specifically related to designs and practices in OSS. Its creators introduced a six-phased approach based on indexing, paging, query reduction, querying, data representation, and results ranking and show that performance and usability are better than PyDriller’s ones, another commonly used Python framework for mining software repositories.

94

R. Kazman et al.

Pattern Mining A large number of software repositories are publicly shared on GitHub and can be particularly useful for research in software mining. Authors in [21] leveraged these repos to collect a dataset of 6919 real-world API description documents in search of recurring structures that can be thought of as pattern primitives according to the standard OpenAPI specification language in Web API accessible form. Such commonly occurring building blocks can be composed to obtain API design patterns that may be useful to designers willing to introduce collections for a specific class of items in HTTP-based APIs. The authors’ contributions provide a method to detect similar recurring or different API structures and provide two collections: the first about widely used API fragments and about how frequently they occur across realworld APIs and the second about structural pattern primitives used as building blocks for HTTP-based APIs. Further contributions are a classification of design smells traced to Web APIs and two composition operators for composing the pattern primitives in larger API structures. A general framework for detecting design patterns is presented in [12]. The framework is based on the open-source process mining toolkit ProM and attempts to address the problem of detecting design patterns with existing tools and approaches. The tool integrates static techniques, which only consider structural connections among classes of patterns on source code input, and dynamic techniques, which analyze sequences of methods calls and the interaction among object on an input made by execution data, to detect observer, state, singleton, and strategy patterns. It has been tested and evaluated on three open-source software systems (Lexi 0.1.1, JUnit 3.7, and JHotDraw 5.1) on a global dataset containing more than one million methods calls, and the results show higher precision and recall with respect to other existing approaches.

API Evolution Analytics Even if architecture degradation along a software system’s lifetime is a widely recognized phenomenon related to changes applied to the system, its full understanding has been hindered by a lack of empirical data regarding the nature of architectural changes. Behnamghader and coauthors in [1] introduced ARCADE, an architecture recovery framework enabling the conduct of replicable large-scale empirical studies on architectural changes. The authors applied the framework to hundreds of version of 23 open-source software systems. They also focused their attention on the introduction of new versions when APIs start losing compatibility with previous system versions which often brings substantial changes to the system’s architecture. As a result, the chapter provides a comprehensive discussion of two newly introduced similarity architectural metrics, a2a (architecture to architecture),

An Empirical Basis for Software Architecture Research

95

a system-level metric to quantify architectural changes, and cvg (cluster coverage), a component-level metric. An empirical study on the evolution of a large OpenAPI collection is presented in [9] to detect API evolution over time and how developers manage the introduction of breaking changes. The authors targeted in particular changes for operations removal. Their findings show that only a very small fraction (0.6%) of removals are tagged as deprecated before being removed, while a larger proportion (24.6%) of removals directly occur without any preliminary warning about the potentially breaking changes. In [11], Lima and coauthors target the PROMISE database, expanding the corpus with the introduction of new software requirements and evaluating the obtained dataset by machine learning algorithms.

Public Repositories for Datasets We conclude this chapter by discussing some popular public repositories enabling the recovering of data useful for architecture research and illustrating a framework that allows the integration of empirical studies and the assembling of tools for supporting architecture-based studies. Dataverse (https://dataverse.org) is a global research data management consortium originally developed as a collaborative open-source project by the Institute for Quantitative Social Science (IQSS) at Harvard. It is currently an open-source research data repository addressed to researchers, developers, institutions, and journals. It currently holds 91 installations, mainly in Europe and the USA, and allows to set and manage personal data collections. It is also a powerful instrument for journals to manage submissions, to review and link data related to articles. It allows institutions to participate in a consortium of worldwide shared repositories. It also provides guidelines for dataset replication with regard to descriptive metadata, files to include in a Dataverse repository, code, files, and scripts to make possible replications, to remove confidential information from shared datasets, and so on. Zenodo (https://zenodo.org/) is another open repository used by, but not specifically addressed at, software architecture. It has a broader purpose and audience. It is maintained by CERN (European Organization for Nuclear Research) and open to researchers worldwide. A digital object identifier (DOI) is associated with any upload so that all items are identified and tracked. It provides specifications for file size limits (50 GB), as well as for multiple datasets (50 GB). It accepts all file formats and classifies them into categories: publications, posters, presentations, datasets, images, software, videos/audio, and lessons. It is linked to GitHub so that it is possible to directly deploy GitHub repositories ensuring long lifetimes for datasets. Users have different access policy options to choose from for their datasets, such as open, embargoed, restricted, or closed access. It features free-text searches and offers metadata for automated machine reads.

96

R. Kazman et al.

The International Software Benchmarking Standards Group maintains a repository with data about completed software projects reported from different organizations [13]. For each project, the records include aspects related to their context, size, and qualitative factors influencing project execution as well as development and product characteristics. The architecture is unfortunately not explicitly considered beyond basic deployment platform choices as a way to characterize the projects. We conclude this section by introducing SAIN. To tackle the problem of uncoordinated and disjoint researches, infrastructures, and tools in the field of software engineering build in the past in the effort of understanding the architectures of large and complex systems, Garcia et al. created SAIN, an extensible infrastructure enabling the sharing, the replication, and the advancement of software architecture research [5]. This framework was built with the aim of enabling the integration of empirical studies and the technology transfer faced by disjoint research and different development environments. This research project created SAIN (Software Architecture INstrument), a framework enabling the integration and assembly of different tools to support architecture-based analysis. SAIN is provided as a Web platform with three principal components: a catalogued library of cutting-edge tools for reverse engineering, a plug-and-play instrument for integrating the tools, and a reproducibility wizard. The results presented in [5] show that this framework can help software architects with maintainability easing their analysis tasks and allowing them to highlight a small set of architectural elements to focus on.

Problems There are a number of inherent problems in creating an empirical basis for software architecture research. There is, or at least has been, a high bar to entry to do research in this area. First of all, it is often difficult to get access to closed-source data, and so many researchers have turned to open source. This, by itself, is not necessarily a problem, but it might be a threat to validity. Once a project has been identified, you need to obtain or create crawlers to extract the raw data. If you are examining multiple projects, then you need to address differences in what data projects collect and how they choose to represent and label this data. You may need to integrate data from multiple sources (e.g., a long-lived project may change issue trackers or revision control systems or email distribution mechanisms over its lifetime). You may need to build tools to track information over time (e.g., file names may change). And you may need to resolve multiple identities for contributors (contributor’s email addresses may change over time, or they may simply have and use multiple identities) [18]. And you often need to deal with outlier removal (e.g., removing files created by test scripts or a commit made to change the license in every file in the system).

An Empirical Basis for Software Architecture Research

97

The pure sizes of the datasets involved also present challenges. Many opensource code repository providers limit the rate at which their servers respond to requests, which limits the rate at which you can collect data. Assuming that the data has been collected and cleaned and stored and shared, what do we do with it? Other challenges for the research community have been a lack of standard definitions of architecture quality and a lack of suitable metrics over the raw data to measure this quality. For example, we could measure developer productivity per unit time: number of commits, number of lines of code committed, numbers of bugs resolved, etc. We could measure project-level characteristics, such as release frequency or number of likes or stars or downloads in a marketplace. We could compare the speed at which consensus is reached while making different kinds of architectural decisions [16]. And we could measure the health of the community of developers supporting the architecture [23]. Finally, once the data is collected and suitably analyzed and measured, you need to think about how to publicize and share this data. Without this final step, the data that you collect may benefit you, but it does not benefit the software architecture community.

Opportunities Creating large-scale curated software architecture datasets opens up many opportunities for the research and practitioner communities. If we had a large corpus of curated datasets and robust tools, we could envision more ambitious, larger studies. Ideally, we would have a collection of (at least) business goals, requirements, architecture documentation, revision history, issues, discussions (perhaps over a mailing list), and code. And we would not have just a single snapshot, but a record of these artifacts evolving over time. And in these studies, we could maintain more precise control over the many variables involved. This would result in more confident conclusions, deeper understanding of the relationship between the architecture of a project and the resulting outcomes. This, in turn, would give the research community the ability to generalize results—something that has traditionally been a challenge in software architecture research. Due to the inherent complexities in studying large, complex systems (and their communities), generalizability has always been a challenge. The creation of robust sharable datasets holds the promise to address this challenge, and this is an enormous opportunity for the research community. With such a broad and validated empirical basis, it becomes possible to envision projects making “just-in-time” predictions, based on leading indicators, and using these predictions to guide project effort, the architecture of the project, and the evolution of the architecture. This would also bring up opportunities to analyze a project’s evolution and opportunities for action research where there is an intervention (motivated by an

98

R. Kazman et al.

architecture analysis) followed by a period of observation and data collection to determine the outcomes of the intervention and whether these match the predictions. Further opportunities are related to specific architectural patterns. Larger and more curated datasets may be used to best tune architectural choices according to project-specific patterns or, on the contrary, to timely highlight critical or dangerous choices that could be avoided. An example may be the application to migration strategies to understand which microservices patterns may be more useful or more dangerous when extracting components from legacy monolithic systems. While such forms of research are possible now, they require heroic efforts on the part of the research team, and so they are seldom done and never replicated.

Summary and Outlook In this chapter, we have outlined the promise and challenges in creating an empirical basis for software architecture research. The good news is that there are already some efforts underway to ease the collection and dissemination of software architecture tools and datasets. But there is much remaining to be done. Assuming data sources are available and there is a consensus on standardized data and metadata formats, sharing datasets requires a suitable infrastructure for advertising and discovering them. Also, to break the stalemate (there is no data to be found, why should I look for some; if nobody is looking, why should I contribute my dataset?) and start a virtuous cycle, the right incentives and rewards should be established.

References 1. Behnamghader, P., Le, D.M., Garcia, J., Link, D., Shahbazian, A., Medvidovic, N.: A largescale study of architectural evolution in open-source software systems. Empir. Softw. Eng. 22(3), 1146–1193 (2017) 2. Bi, T., Ding, W., Liang, P., Tang, A.: Architecture information communication in two oss projects: The why, who, when, and what. J. Syst. Softw. 181, 111035 (2021) 3. Chaniotaki, A.M., Sharma, T.: Architecture smells and pareto principle: a preliminary empirical exploration. In: 2021 IEEE/ACM 18th International Conference on Mining Software Repositories (MSR), pp. 190–194 (2021). https://doi.org/10.1109/MSR52588.2021.00031 4. Garcia, J., Krka, I., Mattmann, C., Medvidovic, N.: Obtaining ground-truth software architectures. In: 2013 35th International Conference on Software Engineering (ICSE), pp. 901–910 (2013). https://doi.org/10.1109/ICSE.2013.6606639 5. Garcia, J., Mirakhorli, M., Xiao, L., Zhao, Y., Mujhid, I., Pham, K., Okutan, A., Malek, S., Kazman, R., Cai, Y., Medvidovi´c, N.: Constructing a shared infrastructure for software architecture analysis and maintenance. In: 2021 IEEE 18th International Conference on Software Architecture (ICSA), pp. 150–161 (2021). https://doi.org/10.1109/ICSA51549.2021. 00022

An Empirical Basis for Software Architecture Research

99

6. Hassan, A.E.: The road ahead for mining software repositories. In: 2008 Frontiers of Software Maintenance, pp. 48–57. IEEE, New York (2008) 7. Hemmati, H., Nadi, S., Baysal, O., Kononenko, O., Wang, W., Holmes, R., Godfrey, M.W.: The msr cookbook: mining a decade of research. In: 2013 10th Working Conference on Mining Software Repositories (MSR), pp. 343–352 (2013). https://doi.org/10.1109/MSR.2013. 6624048 8. Keim, J., Schulz, S., Fuchß, D., Kocher, C., Speit, J., Koziolek, A.: Trace link recovery for software architecture documentation. In: European Conference on Software Architecture, pp. 101–116. Springer, Berlin (2021) 9. Lauro, F.D., Serbout, S., Pautasso, C.: To deprecate or to simply drop operations? an empirical study on the evolution of a large openapi collection. In: 16th European Conference on Software Architecture (ECSA). Prague, Czech Republic (2022) 10. Li, X., Liang, P., Liu, T.: Decisions and their making in oss development: an exploratory study using the hibernate developer mailing list. In: 2019 26th Asia-Pacific Software Engineering Conference (APSEC), pp. 323–330 (2019). https://doi.org/10.1109/APSEC48747.2019.00051 11. Lima, M., Valle, V., Costa, E., Lira, F., Gadelha, B.: Software engineering repositories: expanding the promise database. In: Proceedings of the XXXIII Brazilian Symposium on Software Engineering, pp. 427–436 (2019) 12. Liu, C.: A general framework to detect design patterns by combining static and dynamic analysis techniques. Int. J. Softw. Eng. Knowl. Eng. 31(01), 21–54 (2021) 13. Lokan, C., Wright, T., Hill, P., Stringer, M.: Organizational benchmarking using the isbsg data repository. IEEE Softw. 18(5), 26–32 (2001). https://doi.org/10.1109/52.951491 14. Muszynski, M., Lugtigheid, S., Castor, F., Brinkkemper, S.: A study on the software architecture documentation practices and maturity in open-source software development. In: 2022 IEEE 19th International Conference on Software Architecture (ICSA), pp. 47–57 (2022). https://doi.org/10.1109/ICSA53651.2022.00013 15. Nguyen, H., Lomio, F., Pecorelli, F., Lenarduzzi, V.: Pandora: continuous mining software repository and dataset generation. In: 2022 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER), pp. 263–267 (2022). https://doi.org/10. 1109/SANER53432.2022.00041 16. Nowak, M., Pautasso, C.: Goals, questions and metrics for architectural decision models. In: Proceedings of the 6th International Workshop on SHAring and Reusing Architectural Knowledge (SHARK), pp. 21–28. ACM, New York (2011). https://doi.org/10.1145/1988676. 1988682 17. Palomba, F., Bavota, G., Penta, M.D., Fasano, F., Oliveto, R., Lucia, A.D.: On the diffuseness and the impact on maintainability of code smells: a large scale empirical investigation. Empir. Softw. Eng. 23(3), 1188–1221 (2018) 18. Paradis, C., Kazman, R.: Building the MSR tool kaiaulu: design principles and experiences. In: Scandurra, P., Galster, M., Mirandola, R., Weyns, D. (eds.) Software Architecture, pp. 107– 129. Springer International Publishing, Cham (2022) 19. Reza, S.M., Badreddin, O., Rahad, K.: Modelmine: a tool to facilitate mining models from open source repositories. In: Proceedings of the 23rd ACM/IEEE International Conference on Model Driven Engineering Languages and Systems: Companion Proceedings, pp. 1–5 (2020) 20. Robles, G., Ho-Quang, T., Hebig, R., Chaudron, M.R., Fernandez, M.A.: An extensive dataset of uml models in github. In: 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR), pp. 519–522 (2017). https://doi.org/10.1109/MSR.2017.48 21. Serbout, S., Pautasso, C., Zdun, U., Zimmermann, O.: From openapi fragments to api pattern primitives and design smells. In: European Conference on Pattern Languages of Programs (EuroPLoP). ACM, Virtual Kloster Irsee, Germany (2021). https://doi.org/10.1145/3489449. 3489998. https://zenodo.org/record/5727094#.YZ97mFMo-0o

100

R. Kazman et al.

22. Soliman, M., Wiese, M., Li, Y., Riebisch, M., Avgeriou, P.: Exploring web search engines to find architectural knowledge. In: 2021 IEEE 18th International Conference on Software Architecture (ICSA), pp. 162–172 (2021). https://doi.org/10.1109/ICSA51549.2021.00023 23. Tamburri, D., Palomba, F., Kazman, R.: Exploring community smells in open-source: an automated approach. IEEE Trans. Softw. Eng. 47(3), 630–652 (2021) 24. Xie, T., Zimmermann, T., van Deursen, A.: Introduction to the special issue on mining software repositories. Empir. Softw. Eng. 18, 1043–46 (2013)

A Better Way to Teach Software Architecture Rick Kazman , Yuanfang Cai , Michael W. Godfrey Cesare Pautasso , and Anna Liu

,

Abstract Software architecture education is a weak spot in many undergraduate programs in computer science and software engineering. While the concepts and practices used by software architects in industry are rich and varied, transferring this expertise into a university classroom has proved problematic. Bridging the gap between industry and academia requires ongoing, often heroic, effort. This is a “chicken and egg” problem: Because there is a lack of good teaching materials, architecture is seldom taught, and because it is seldom taught, there has been little incentive to create good materials. We would like to change that. Our goal is to establish guidelines for how software architecture practices should be taught—both technical and non-technical topics—and to suggest appropriate teaching methods to best prepare students to be software architects in practice.

R. Kazman () University of Hawaii, Honolulu, HI, USA e-mail: [email protected] Y. Cai Drexel University, Philadelphia, PA, USA e-mail: [email protected] M. W. Godfrey University of Waterloo, Waterloo, ON, Canada e-mail: [email protected] C. Pautasso University of Lugano, Lugano, Switzerland e-mail: [email protected] A. Liu Amazon Web Services, Seattle, WA, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 P. Pelliccione et al. (eds.), Software Architecture, https://doi.org/10.1007/978-3-031-36847-9_6

101

102

R. Kazman et al.

Introduction Software architecture concerns the design of a software-intensive system at the highest level of abstraction. It is typically first addressed early on in development, as the decisions made at this level will pervade the design of the rest of the system and are thus the most difficult and costly to change later on. Of course, change is inevitable in most technical endeavors, and the architecture will also have to evolve as knowledge about the system and its operational context become clearer. However, understanding the costs and benefits of the architectural-level decisions can help to mitigate the overall risk of developing the software system. The chosen architecture of a software system will exert strong forces on the system’s quality attributes, such as modifiability, extensibility, availability, performance, scalability, sustainability, and security. It also determines the division of functionality in the system and consequently the structure of the development team. A large-scale system without a consciously developed and managed architecture can be disastrous, resulting in unhappy stakeholders—because their quality attribute goals are not being met—and an ever-increasing load of technical debt as the system evolves in an unprincipled and ad hoc way. Software architectural expertise is vital to practicing software developers, but the topic is seldom taught in undergraduate computer science (CS) or software engineering (SE) programs. More often, it is found within MSc-level curricula [12], if it is found at all. The typical undergraduate CS/SE student is often a “good” programmer, but has little experience with developing or maintaining systems larger than a few hundred lines of code. It can be difficult to motivate students to care about the high-level design of the system—they don’t know what they don’t know—and it can be difficult to convey the importance of architecture and the risks of not paying attention to it, especially when dealing with large and long-lasting systems involving multiple developers and stakeholders. While some MSc-level students might have industry experience, this is not guaranteed. So anyone who is teaching software architecture to students at either level must grapple with their lack of real-world experience. Furthermore, software architecture design is more of an engineering discipline than a mathematical one. Design is a “wicked” problem: there are rarely right/wrong answers, but rather, there are better/worse approaches, each with their own set of contextual tradeoffs. There are always many ways of designing an architecture and many ways of designing satisfactory solutions. However, there are many more ways of designing unsatisfactory solutions—it is an inherently complex problem, requiring experience and domain knowledge to address. Complicating the problem further is that undergraduate CS/SE education is often taught in conceptual “silos”, with classes and projects that are isolated from each other. For this reason, students have become accustomed to wrestling with problems of limited scope and scale; problems whose solutions have short timeframes; problems that can be solved in isolation, ignoring pre-existing solutions; and

A Better Way to Teach Software Architecture

103

problems with well-defined, stable requirements. None of these are typical features of real-world software systems. A successful software architect needs to have a broad range of both technical and non-technical skills. However, there is often little attention paid to so-called “soft” skills in software engineering education. Our vision is that a course in software architecture must emphasize soft skills such as arbitration, brainstorming, facilitation, negotiation, listening and eliciting, delivering clear and convincing presentations, receiving and providing feedback [10], translating between technical and business worlds, leadership, decision-making, and risk analysis. Developing these skills will empower software architects to devise and communicate highquality large-scale system designs. To achieve all of these goals, we may need to expand on traditional thinking and go beyond the usual scope of classroom-based education, engaging in (or at least simulating the engagement in) large-scale real-world industrial or open-source projects. Only in these settings, students will experience how to work under a largely unknown context.

Teaching Resources For many educators, finding a supporting textbook is a must before embarking on the design of a new course. Fortunately, there are already several good options for software architecture: – – – – –

Software Architecture in Practice [2] Patterns of Software Architecture [3] Software Architecture: Foundations, Theory, and Practice [15] Just Enough Software Architecture: A Risk-Driven Approach [8] Software Systems Architecture: Working with Stakeholders Using Viewpoints and Perspectives [16] – Design It! From Programmer to Software Architect [11] – Software Architecture: Visual Lecture Notes [14]

Several of these textbooks provide supporting resources for instructors including PowerPoint slides, discussion questions, and classroom exercises; some also provide access to online pedagogical materials. And many of these resources have been written by practitioners, which lends an air of authority to their recommendations of best practices. We feel that these example textbooks are of high quality and broad scope, addressing both the technical and non-technical topics mentioned above. While they do not solve our problem on their own, they can support the delivery of a welldesigned course in software architecture. But textbooks alone do not solve all of the challenges of software architecture education. To truly grasp a complex domain, the student needs a significant amount of experiential learning. They need to grapple with complex, non-obvious problems.

104

R. Kazman et al.

They need to grapple with uncertainty, with tradeoffs, with risk, and with the demands of a wide group of stakeholders. These kinds of experiences are difficult, but not impossible, to inject into the classroom experience.

Industry Talent Needs and Architecture Realities There is already a significant lack of software engineering talent to fulfill industry’s existing human resource needs. There is an even thinner pipeline of experienced and well-trained software architects to lead corporate digital transformation projects. The global market intelligence company IDC reports that the worldwide shortage of full-time software developers will increase from 1.4 million in 2021 to 4.0 million in 2025 [7]. This talent shortage problem of software developers is expected to grow worse, as industries continue to rely on experienced developers for delivering solutions that not only digitize business processes and operations but also enable innovative customer service and business models. The talent shortage of skilled software architects is even more dire. The pipeline of experienced developers who can transition to software architect roles is thin to begin with, and the road to becoming an effective software architect requires significant large-scale system portfolio experience over many years. While software developer resources can often be successfully obtained as needed through an interorganization or even international sourcing model, the role of software architect typically requires deep organizational and cultural insights to be effective and successful in the long term. That is, software architects cannot simply be “magicked up” on demand. To further compound this talent pipeline issue, the plethora of digital transformation projects is rapidly growing across all industries. Organizations rely on the unique technical and interpersonal leadership of strong software architects to design, plan, navigate, communicate, and manage these complex projects, to guide the enterprise through their digital transformation journeys. Industrial software projects are rarely greenfield projects. New CS/SE graduates expecting to be designing large-scale software systems from scratch may, unfortunately, have unrealistic expectations. This is not their fault; the problem lies mainly in how they have been educated, which focuses more on software creation than integration, maintenance, and evolution. An industrial software architect’s dayto-day job can include some unglamorous tasks: ad hoc maintenance of brittle, ill-documented systems designed by employees who left the organizations long ago; software evolution planning, including strategies to retire legacy systems; and evaluation and acquisition of new capabilities to incorporate into the existing spaghetti-like enterprise architecture. This reality is at some distance from the often idealized and waterfall-like lifecycle descriptions of large-scale software system examples in some software engineering textbooks and curricula. This situation can lead to disappointment and sometimes even attrition of new computer science graduate talents landing in the deep end of enterprise architectural realities.

A Better Way to Teach Software Architecture

105

Duties, Skills, and Knowledge An effective software architect requires a large skill set—both technical and nontechnical—and a lot of experience. An architect needs to know a lot—about technologies, about organizations, about business, and about people. And an architect has many duties. Much of this can be taught, and that is the good news and the reason for hope in this chapter. Even the shyest, most introverted person can learn to become a better communicator, a better negotiator, and a better leader. It is beyond the scope of this chapter, and of the expertise of its authors, to speculate how much of an architect’s duties, skills, and knowledge can be learned, but we take it as a given that everyone can get better in these three areas. Let us look at them in more detail.

Knowledge Knowledge is perhaps the first area that comes to mind when envisioning what software architecture education should require, and it is likely the easiest area to teach. This aspect of being a good software architect is more similar to the rest of software engineering education. Some of the most important knowledge areas include: – Technology knowledge: keeping abreast of state-of-the-art technologies, prototyping and experimentation, making appropriate tools, and framework selection decisions – Evaluation methods knowledge: architecture analysis, performance testing, reliability evaluation, security and assurance methods, empirical methods, and evaluation criteria definition – Technology-independent knowledge: deep understanding of patterns, design principles, architectural styles, large-scale distributed systems issues, and foundational design knowledge – Fundamental knowledge: understanding what problems are inherent in the domain and which ones are transient and hence more amenable to nonarchitectural solutions – Meta-knowledge: learning how to learn and be a lifelong learner. Awareness of one’s own limits. Ability to collect domain-specific knowledge A good software architecture education program should not just teach these knowledge areas but also put them to use in practical contexts [9]. The more abstract the knowledge, the more that its practical application is critical to understanding. For example, one study [4] showed that even after taking a semester-long course in software design and architecture, a majority of students still made fundamental design errors and were unaware that these violated the design principles that they had been taught. A project-based learning approach may help mitigate these issues [17].

106

R. Kazman et al.

Skills Skills may be the most difficult area to teach, as they are often more challenging and less familiar to software educators. Consider a skill in another domain, say playing the piano. No one would expect that a student could learn to be an effective pianist from observing a lecture or two and doing an assignment. It takes consistent effort and practice to hone a skill. A software architecture course can at least be a starting point for this journey. Here are some of the most important skills for an architect to acquire and master: – Communication: architects must be able to communicate effectively, to sell their architectural vision both to management and to developers. – Negotiation: architects often need to negotiate resources, requirements, priorities, and constraints. These can be the crux of key tradeoff decisions [5]. – Leadership: an architect must be able to rally people around the architectural vision. This is, perhaps, the most difficult skill to teach. – Workload management: architects are often extremely busy, given the depth and breadth of their work roles. – Flexibility/adaptability: architects must be able to react to changes. These could be changes in requirements, in strategic direction, in the external world (e.g., the technical ecosystem in which we all live and operate), in the workforce, in funding, in deadlines, and in many others as well. Any architect who is inflexible will fail sooner or later. This is, like leadership, a very challenging skill to teach and to acquire.

Duties Architects have a large array of duties, which may vary according to the specific working context. In a small project, the architect may perform all of the duties listed below. In a large project, the architect may have direct responsibility for only a few key duties, but should have knowledge of and insight into the others. – Creating, analyzing, and documenting an architecture: these are the most obvious duties of an architect. Who is an architect, if not the one doing highest-level design? But analysis duties may be done by outsiders (outside the team, outside the organization, outside the company), and documentation may not be done at all or may be done by everyone. An architect must be aware of and at least involved with all of these duties. – Tool and technology selection: one of the most important decisions an architect can make in design is the choice of tools and technologies. This selection process is often complex, requiring the architect to be aware of many variables (cost, complexity, familiarity of the team, quality attributes, likely survivability of the technology, licensing, etc.).

A Better Way to Teach Software Architecture

107

– Leading or participating in design review and conformance exercises: if you are creating an architecture for a long-lived project, then it will almost certainly evolve. And if it evolves, there is a strong likelihood that it will diverge from its original intent and original documentation. This can often undermine the quality of the architecture. To guard against this, an architect should participate in code and design reviews, with an eye to spotting any decisions that may undermine or evolve the architecture. – Requirements elicitation and management: while an architect may have little say in the functional requirements for a product, she should have a strong say in the elicitation, baselining, and negotiation of quality attribute requirements. These are among the strongest design drivers, so it is in the architect’s self-interest and in the best interest of the project to manage these effectively. – Managing and designing for quality assurance: when we think about testing, we typically focus on testing the system-level user-facing functionality of the system. But an architect is also responsible for the quality attributes of the system: its maintainability, security, performance, availability, and so forth. To ensure that these are met when the product is created and as it evolves, such non-functional requirements must be explicitly tested for. Thus, one of the architect’s duties is to ensure that the architecture and its environment make such testing possible. As we can see from the above, an architect’s duties, skills, and knowledge form a comprehensive list. How can we hope to teach all of this to students? Perhaps we cannot. But we can prepare the ground so that our students understand the many dimensions of architecture and being an architect, and we can prepare them to be lifelong learners.

Challenges and Opportunities An ample supply of well-trained software architects who are skilled at delivering robust software solutions for the long term is an established need from industry. The existence and maturity of a large number of open-source projects, such as Android, Chrome, and Hadoop, and the research advances that can be used to dissect the architecture of these systems provide a number of opportunities to better educate the much needed cohort of software architects for the future. Below are some opportunities that we see as being most critical and most promising. 1. Conveying real-world settings: Using longer-term projects opens up the opportunity to study the impact of changing requirements and experience maintenance and architecture refactoring [1]. The revision history and the code repository of these open-source projects provide ample examples of realistic settings, change requests, and bug fixing requests. Having students immerse themselves in these projects and asking them to recover the architecture so that they can claim a bug to fix or implement a new feature will put them into real industrial settings with direct impact.

108

R. Kazman et al.

2. Dealing with teamwork: Working in pairs; working in larger groups; and adding in new project members or swapping them along the way (e.g., [18]). Most of these prominent open-source projects are supported by companies like Google, Microsoft, Intel, Red Hat, Meta, Amazon Web Services, SAP, IBM, and many others. Having students work on these projects provides an opportunity for them to work in large groups and communicate with both open-source and industrial practitioners. Still, using open-source projects for undergraduate student education should be done limiting the cost imposed on the project maintainers and ensuring a win-win outcome benefiting both students and the projects to which they contribute to. 3. Design thinking: Architectural decision-making, creating and pruning alternatives, and exploring quality attribute tradeoffs. Numerous design decisions and tradeoffs are weaved into these open-source projects. Decisions and tactics used to ensure security, performance, and availability are indispensable for all these projects. We could leverage recent research tools to extract these decisions from source code and using these extracted source code as teaching examples. 4. New content organization: For each quality attribute and corresponding tactics, we could collect their design and implementations in these open-source projects and organize these materials based on quality attribute, viewpoint, and software development lifecycle. As a result, we could use these realistic examples to teach important architecture concepts. 5. Better use of notations: There are many modeling notations, such as architectural description languages or UML, that are seldom used in practice. It is useful to teach students to record and model their design decisions using architectural decision records. This can bring more precise thinking and analysis to the architecture design and analysis process as well as clarity, precision, and accuracy in the way it is represented. But instead of trying to model every aspect of a software architecture—which is very costly and almost never done in industry—for each project, we could extract a partial high-level model that is relevant to only one or a few key decisions or quality attributes. For example, we could extract and document a high-level component diagram related to security only. In this way, we can get the pedagogical benefits of models while avoiding the complexity and overhead of most models that prevent them from being widely adopted. 6. Grading/evaluation: It remains challenging to assess student learning on software architecture beyond superficial recall of definitions and toy design problems to be solved in an exam setting. It is very unlikely that real-world architects would need to go from zero knowledge about a domain to a complete specification of an architectural model in a few hours’ time. To lower the teaching load, to remove subjectivity, and to provide regular feedback to students, we can and should automate design analysis [4]. Using recently published research tools, it becomes possible to automatically determine if students’ homework assignments have followed proper design principles or whether they are well-modularized, so that their homework can be graded not only based on functionality but also based on design quality.

A Better Way to Teach Software Architecture

109

7. New teaching strategies: We can employ games (e.g., [6]), simulations, architectural katas [13], and live-through exercises, to try to give students the flavor of real-world architecting duties while keeping the overhead manageable for both students and educators.

Next Steps This chapter serves as a call to action. If we are to improve the state of the art in teaching software architecture, then we need to get away from our reliance on “heroic” individual efforts, which is neither sustainable nor scalable. Our community of researchers and educators needs to come together, so that we can better educate the next generation of architects. That means we need better example curricula, teaching materials, and tooling so that people can teach architecture as they would teach other courses, with more rigor and with more repeatable outcomes. We need to create and share resources such as curriculum guidelines, checklists, and—perhaps most importantly—large-scale teaching examples with curated datasets that connect business goals, requirements, architecture, documentation, and code. And these datasets should ideally include some history, so that we can reason about and teach how architectural artifacts and decisions evolve over time.

References 1. Bærbak Christensen, H.: Teaching microservice architecture using devops—an experience report. In: European Conference on Software Architecture, pp. 117–130. Springer, Berlin (2022) 2. Bass, L., Clements, P., Kazman, R.: Software Architecture in Practice, 4 edn. Addison-Wesley, Reading (2021) 3. Buschmann, F., Meunier, R., Rohnert, H., Sommerlad, P., Stal, M.: Pattern-Oriented Software Architecture Volume 1: A System of Patterns. Wiley, New York (1996) 4. Cai, Y., Kazman, R., Jaspan, C., Aldrich, J.: Introducing tool-supported architecture review into software design education. In: Proceedings of the Conference on Software Engineering Education and Training (CSEE&T) (2013) 5. Capilla, R., Zimmermann, O., Carrillo, C., Astudillo, H.: Teaching students software architecture decision making. In: European Conference on Software Architecture, pp. 231–246. Springer, Berlin (2020) 6. Cervantes, H., Haziyev, S., Hrytsay, O., Kazman, R.: Smart decisions: an architectural design game. In: Proceedings of the International Conference on Software Engineering (ICSE) (2016) 7. Dayaratna, A.: Quantifying the worldwide shortage of full-time developers. In: IDC Market perspective (2021) 8. Fairbanks, G.: Just Enough Software Architecture: A Risk Driven Approach. Marshall and Brainerd, New York (2010)

110

R. Kazman et al.

9. Galster, M., Angelov, S.: What makes teaching software architecture difficult? In: 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C), pp. 356–359 (2016) 10. Hazzan, O.: The reflective practitioner perspective in software engineering education. J. Syst. Softw. 63(3), 161–171 (2002) 11. Keeling, M.: Design It! From Programmer to Software Architect. Pragmatic Bookshelf, New York (2017) 12. Lago, P., Van Vliet, H.: Teaching a course on software architecture. In: Proceedings of the 18th Conference on Software Engineering Education and Training (CSEET’05), pp. 35–42. IEEE, New York (2005) 13. Neward, T.: Architectural katas (2012). https://www.architecturalkatas.com 14. Pautasso, C.: Software Architecture: Visual Lecture Notes. LeanPub, New York (2020) 15. Taylor, R.N., Nenad Medvidovic, E.D.: Software Architecture: Foundations, Theory, and Practice. Wiley, New York (2009) 16. Rozanski, N., Woods, E.: Software systems architecture: working with stakeholders using viewpoints and perspectives. Addison-Wesley, Reading (2011) 17. Rupakheti, C.R., Chenoweth, S.V.: Teaching software architecture to undergraduate students: an experience report. In: 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, vol. 2, pp. 445–454. IEEE, New York (2015) 18. Van Deursen, A., Aniche, M., Aué, J., Slag, R., De Jong, M., Nederlof, A., Bouwers, E.: A collaborative approach to teaching software architecture. In: Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education, pp. 591–596 (2017)