Transformation Of Knowledge, Information And Data: Theory And Applications [illustrated edition] 9781591401261, 1-59140-126-7

Citation preview

Transformation of Knowledge, Information and Data: Theory and Applications Patrick van Bommel University of Nijmegen, The Netherlands

Information Science Publishing Hershey • London • Melbourne • Singapore

Acquisition Editor: Senior Managing Editor: Managing Editor: Development Editor: Copy Editor: Typesetter: Cover Design: Printed at:

Mehdi Khosrow-Pour Jan Travers Amanda Appicello Michele Rossi Alana Bubnis Jennifer Wetzel Mindy Grubb Yurchak Printing Inc.

Published in the United States of America by Information Science Publishing (an imprint of Idea Group Inc.) 701 E. Chocolate Avenue, Suite 200 Hershey PA 17033 Tel: 717-533-8845 Fax: 717-533-8661 E-mail: [email protected] Web site: http://www.idea-group.com and in the United Kingdom by Information Science Publishing (an imprint of Idea Group Inc.) 3 Henrietta Street Covent Garden London WC2E 8LU Tel: 44 20 7240 0856 Fax: 44 20 7379 3313 Web site: http://www.eurospan.co.uk Copyright © 2005 by Idea Group Inc. All rights reserved. No part of this book may be reproduced in any form or by any means, electronic or mechanical, including photocopying, without written permission from the publisher. Library of Congress Cataloging-in-Publication Data Transformation of knowledge, information and data : theory and applications / Patrick van Bommel, editor. p. cm. Includes bibliographical references and index. ISBN 1-59140-527-0 (h/c) — ISBN 1-59140-528-9 (s/c) — ISBN 1-59140-529-7 (eisbn) 1. Database management. 2. Transformations (Mathematics) I. Bommel, Patrick van, 1964QA76.9.D3T693 2004 005.74—dc22

2004017926

British Cataloguing in Publication Data A Cataloguing in Publication record for this book is available from the British Library. All work contributed to this book is new, previously-unpublished material. The views expressed in this book are those of the authors, but not necessarily of the publisher.

Transformation of Knowledge, Information and Data: Theory and Applications

Table of Contents

Preface ............................................................................................................. vi

Section I: Fundamentals of Transformations Chapter I Transformation-Based Database Engineering ........................................... 1 Jean-Luc Hainaut, University of Namur, Belgium Chapter II Rule-Based Transformation of Graphs and the Product Type .............. 29 Renate Klempien-Hinrichs, University of Bremen, Germany Hans-Jörg Kreowski, University of Bremen, Germany Sabine Kuske, University of Bremen, Germany Chapter III From Conceptual Database Schemas to Logical Database Tuning ...... 52 Jean-Marc Petit, Université Clermont-Ferrand 2, France Mohand-Saïd Hacid, Université Lyon 1, France

Chapter IV Transformation Based XML Query Optimization ................................... 75 Dunren Che, Southern Illinois University, USA Chapter V Specifying Coherent Refactoring of Software Artefacts with Distributed Graph Transformations ........................................................... 95 Paolo Bottoni, University of Rome “La Sapienza”, Italy Francesco Parisi-Presicce, University of Rome “La Sapienza”, Italy and George Mason University, USA Gabriele Taentzer, Technical University of Berlin, Germany

Section II: Elaboration of Transformation Approaches Chapter VI Declarative Transformation for Object-Oriented Models .................. 127 Keith Duddy, CRC for Enterprise Distributed Systems Technology (DSTC), Queensland, Australia Anna Gerber, CRC for Enterprise Distributed Systems Technology (DSTC), Queensland, Australia Michael Lawley, CRC for Enterprise Distributed Systems Technology (DSTC), Queensland, Australia Kerry Raymond, CRC for Enterprise Distributed Systems Technology (DSTC), Queensland, Australia Jim Steel, CRC for Enterprise Distributed Systems Technology (DSTC), Queensland, Australia Chapter VII From Conceptual Models to Data Models ............................................ 148 Antonio Badia, University of Louisville, USA Chapter VIII An Algorithm for Transforming XML Documents Schema into Relational Database Schema .................................................................... 171 Abad Shah, University of Engineering & Technology (UET), Pakistan Jacob Adeniyi, King Saud University, Saudi Arabia Tariq Al Tuwairqi, King Saud University, Saudi Arabia

Chapter IX Imprecise and Uncertain Engineering Information Modeling in Databases: Models and Formal Transformations ................................ 190 Z. M. Ma, Université de Sherbrooke, Canada

Section III: Additional Topics Chapter X Analysing Transformations in Performance Management .................. 217 Bernd Wondergem, LogicaCMG Consulting, The Netherlands Norbert Vincent, LogicaCMG Consulting, The Netherlands Chapter XI Multimedia Conversion with the Focus on Continuous Media ......... 235 Maciej Suchomski, Friedrich-Alexander University of Erlangen-Nuremberg, Germany Andreas Märcz, Dresden, Germany Klaus Meyer-Wegener, Friedrich-Alexander University of Erlangen-Nuremberg, Germany Chapter XII Coherence in Data Schema Transformations: The Notion of Semantic Change Patterns ........................................................................................ 257 Lex Wedemeijer, ABP Pensioenen, The Netherlands Chapter XIII Model Transformations in Designing the ASSO Methodology ......... 283 Elvira Locuratolo, ISTI, Italy About the Authors ..................................................................................... 303 Index ............................................................................................................ 311

vi

Preface

Background Data today is in motion, going from one location to another. It is more and more moving between systems, system components, persons, departments, and organizations. This is essential, as it indicates that data is actually used, rather than just stored. In order to emphasize the actual use of data, we may also speak of information or knowledge. When data is in motion, there is not only a change of place or position. Other aspects are changing as well. Consider the following examples:

•

The data format may change when it is transferred between systems. This includes changes in data structure, data model, data schema, data types, etc.

•

Also, the interpretation of data may vary when it is passed on from one person to another. Changes in interpretation are part of data semantics rather than data structure.

•

The level of detail may change in the exchange of data between departments or organizations, e.g., going from co-workers to managers or from local authorities to the central government. In this context, we often see changes in level of detail by the application of abstraction, aggregation, generalization, and specialization.

•

Moreover, the systems development phase of data models may vary. This is particularly the case when implementation-independent data models are mapped to implementation-oriented models (e.g., semantic data models are mapped to operational database specifications).

These examples illustrate just a few possibilities of changes in data. Numerous other applications exist and everybody uses them all the time. Most applications are of vital importance for the intelligent functioning of systems, persons, departments, and organizations.

vii

In this book, the fundamental treatment of moving knowledge, information, or data, with changing format, interpretation, level of detail, development phase, etc., is based on the concept of transformation. The generally accepted terms conversion, mutation, modification, evolution, or revision may be used in specific contexts, but the central concept is transformation. Note that this definition covers well-known topics such as rewriting and versioning, and that it is relevant for collaborative information systems and data warehouses. Although data transformation is typically applied in a networked context (e.g., Internet or intranet), it is applied in other contexts as well.

Framework Transformation techniques received a lot of attention in academic as well as in industrial settings. Most of these techniques have one or more of the following problems:

•

Loss of data: the result of the transformation does not adequately describe the original data.

• •

Incomprehensibility: the effect of the transformation is not clear.

•

Focus on types: data types are transformed, without incorporation of data instances.

•

Correctness: the transformation has no provable correctness.

Focus on instances: data instances are transformed, without incorporation of data types.

We therefore aim at generic approaches for the treatment of data transformations. Some of the questions we deal with are the following: What is an adequate data transformation technique? What are the requirements for the input and output of those techniques? What are the problems in existing approaches? What are the possibilities of a generic approach in important areas such as the semantic web, supply chain management, the global information community, and information security? The theory and applications in this book are rooted in database schema transformation, as well as in database contents transformation. This allows for other transformations, including transformation of document type definitions (DTDs) and of concrete documents. It is obvious that graph transformations are relevant here. Note that we do not particularly focus on specific kinds of data or documents (e.g., RDBMS, HTML or XML), although the models under consideration do not exclude such a focus.

viii

From Source to Target Here we discuss general aspects of the move from source to target. They deal with the basic assumptions underlying all transformation processes.

•

Source. This is the structure to be transformed, or in other words, it is the input to the transformation process. An important distinction is made between formal and informal sources. If the source is informal, the transformation process cannot be fully automated. We usually then have a partly automated transformation aiming at support, with sufficient possibilities for interaction. As an example, a modeling process often is the mapping of an informal view to a formal model. In this book, the input and output of most transformations are assumed to be available in some formal language.

•

Target. This is the resulting structure, so it is the output of the transformation process. A main question here is how the relation between the target and the source is defined. Even when the transformation process has been completed, it is important that the relation of the target with the source remains clear. One way of establishing such a clear relation, is to have the target defined in terms of the source. This is also helpful in providing correctness proofs.

•

Applicability. In some cases, transformations are not really general in the sense that the possible source and target are rather restricted. If, for example, a theoretical model of transformations only allows for exotic targets, not being used in practical situations, the theoretical model suffers from applicability problems.

•

Structure vs. access operations. Besides the transformation of structures, we must provide mechanisms for the transformation of access operations. These operations may be modification operations as well as retrieval operations. Consequently, we have a source structure with corresponding access operations, and a target structure with equivalent operations. This situation is shown in Figure 1. The transformation kernel contains all metadata relevant for the transformation.

Correctness Evidently, the correctness of transformations is of vital importance. What purpose would transformations have, if the nature of the result is uncertain? A general setup for guaranteeing transformation correctness consists of three steps.

ix

Figure 1. Framework for transformation of structures and operations source structure

structure transformation

target structure

transformation kernel

target operations

source operations operation transformation

•

Wellformedness conditions. First, we describe the required properties of the target explicitly. We prefer to have basic (independent) wellformedness conditions here, as this facilitates the systematic treatment in the next steps.

•

Transformation algorithm. Next, we describe the construction of the target on the basis of the source at hand. This construction process is defined in the transformation algorithm, which may be enhanced using guidance parameters. Guidance is interpreted as the development towards target structures having certain desirable qualities.

•

Correctness proof. Finally, we prove that the result of the algorithm satisfies the wellformedness conditions. As a consequence, the resulting structure is correct in the sense that all wellformedness conditions are satisfied. Moreover, when specific guidance parameters are used, we have to prove that the resulting structure not only satisfies all wellformedness conditions, but has the desirable qualities (indicated by guidance parameters) as well.

Sequences of Transformations Transformations may be composed or applied in sequences. Such sequences sometimes consist of a relatively small number of steps. In more complex problem areas, however, this is no longer possible. Then, transformation sequences will be longer and due to the various options in each transformation step, the outcome of the overall sequence is not a priori known. This is particularly the case when non-deterministic (e.g., random or probabilistic) transformation processes are considered.

x

Although the outcome is not a priori known, it is often desirable to predict the nature of the result. One way of predicting the behavior of probabilistic transformation processes, is through the use of Markov theory. Here the probabilities of a single transformation step are summarized in a transition matrix, such that transformation sequences can be considered by matrix multiplication. We will illustrate the definition of a single-step matrix for two basic cases. In the first case, consider a transformation in a solution space S where each input x∈S has as possible output some y∈N(x), where N(x)⊆S and x∉N(x). So each neighbor y∈N(x) can be produced from x by the application of some transformation rule. Then the probability P(x,y) for the transformation of x into some y∈N(x) has the following property: P(x,y) × |N(x)| = 1

(1)

Evidently for y∉N(x) we have P(x,y)=0. With this property it is guaranteed that P(x,y) is a stochastic matrix, since 0 ≤ P(x,y) ≤ 1 and Σy∈S P(x,y) = 1. Note that in the above transformation the production of all results is equally likely. In the second case, we consider situations where the production of all results is not equally likely. Consider a transformation in a solution space S where each input x∈S has as possible output some y∈B(x), where B(x)⊆N(x) contains all better neighbors of x. Then the probability P(x,y) for the transformation of x into some y∈B(x) is given by the above mentioned formula (1). However, as a result of accepting only improving transformations, this formula now does not guarantee P(x,y) to be a stochastic matrix. The consequence of rejecting all neighbours in N(x)-B(x) is, that a transformation may fail. So now we have to consider P(x,x). This probability has the following property: P(x,x) × |N(x)| = |N(x)| - |B(x)|

(2)

In this case we have P(x,y)=0 for y ∉ {x}∪B(x). Now we have described a hill climbing transformation sequence. Note that the matrix underlying hill climbing transformations is a stochastic matrix indeed. We will now give an overview of the book. It consists of three parts: fundamentals of transformations, elaboration of transformation approaches, and additional topics. These three sections contain 13 chapters. It is possible to start in a later chapter (e.g., in Section II or III), without reading all earlier chapters (e.g., more theoretical chapters in Section I).

xi

Fundamentals of Transformations Section I is about fundamentals and consists of five chapters. The focus of Chapter I is databases: Transformation-Based Database Engineering. Here we consider the basic theory of the transformation of data schemata, where reversibility of transformations is also considered. We describe the use of basic transformations in the construction of more complex (higher-level) transformations. Several possibilities are recognized here, including compound transformations, and predicate-driven and model-driven transformations. Basic transformations and their higher-level derivations are embedded within database (forward) design processes as well as within database reverse design processes. Most models to be transformed are defined in terms of graphs. In Chapter II we will therefore focus on graph transformations: Rule-Based Transformation of Graphs and the Product Type. Graph transformations are based on rules. These rules yield new graphs, produced from a given graph. In this approach, conditions are used to have more control over the transformation process. This allows us to indicate the order of rule application. Moreover, the result (product) of the transformation is given special attention. In particular, the type of the product is important. This sets the context for defining the precise relation between two or more graph transformations. Having embedded our transformations within the graph transformation context, Chapter III proceeds with graphs for concrete cases: From Conceptual Database Schemas to Logical Database Tuning. Here we present several algorithms, aiming at the production of directed graphs. In databases we have several aims in transformations, including efficiency and freedom from null values. Note that wellformedness of the input (i.e., a conceptual model) as well as wellformedness of the output (i.e., the database) is addressed. It is evident that graphs have to be transformed, but what about operations on graphs? In systems design this corresponds with query transformation and optimization. We apply this to markup languages in Chapter IV: Transformation Based XML Query Optimization. After representing document type definitions in terms of a graph, we consider paths in the graph and an algebra for text search. Equivalent algebraic expressions set the context for optimization, as we know from database theory. Here we combine the concepts from previous chapters, using rule-based transformations. However, the aim of the transformation process now is optimization. In Chapter V, the final chapter of Section I, we consider a highly specialized fundament in the theory behind applications: Specifying Coherent Refactoring of Software Artefacts with Distributed Graph Transformations. Modifications in the structure of systems are recorded in terms of so-called “refactoring”. This means that a coordinated evolution of system components becomes pos-

xii

sible. Again, this graph transformation is rule-based. We use this approach to reason about the behavior of the system under consideration.

Elaboration of Transformation Approaches In Section II, we consider elaborated approaches to transformation. The focus of Chapter VI is object-oriented transformation: Declarative Transformation for Object-Oriented Models. This is relevant not only for object-oriented data models, but for object-oriented programming languages as well. The transformations under consideration are organized according to three styles of transformation: source-driven, target-driven, and aspect-driven transformations. Although source and target will be clear, the term “aspect” needs some clarification. In aspect-driven transformations, we use semantic concepts for setting up the transformation rule. A concrete SQL-like syntax is used, based on rule — forall — where — make — linking statements. This also allows for the definition of patterns. It is generally recognized that in systems analysis we should use conceptual models, rather than implementation models. This creates the context for transformations of conceptual models. In Chapter VII we deal with this: From Conceptual Models to Data Models. Conceptual models are often expressed in terms of the Entity-Relationship approach, whereas implementation models are often expressed in terms of the relational model. Classical conceptual model transformations thus describe the mapping from ER to relational models. Having UML in the conceptual area and XML in the implementation area, we now also focus on UML to XML transformations. We proceed with this in the next chapter: An Algorithm for Transforming XML Documents Schema into Relational Database Schema. A typical approach to the generation of a relational schema from a document definition, starts with preprocessing the document definition and finding the root node of the document. After generating trees and a corresponding relational schema, we should determine functional dependencies and other integrity constraints. During postprocessing, the resulting schema may be normalized in case this is desirable. Note that the performance (efficiency) of such algorithms is a critical factor. The proposed approach is illustrated in a case study based on library documents. Transformations are often quite complex. If data is inaccurate, we have a further complication. In Chapter IX we deal with this: Imprecise and Uncertain Engineering Information Modeling in Databases: Models and Formal Transformations. Uncertainty in information modeling is usually based on fuzzy

xiii

sets and probability theory. Here we focus on transformations in the context of fuzzy Entity-Relationship models and fuzzy nested relations. In the models used in this transformation, the known graphical representation is extended with fuzzy elements, such as fuzzy type symbols.

Additional Topics In Section III, we consider additional topics. The focus of Chapter X is the application of transformations in a new area: Analysing Transformations in Performance Management. The context of these transformations is an organizational model, along with a goal model. This results in a view of organizational management based on cycles of transformations. Typically, we have transformations of organizational models and goal models, as well as transformations of the relationship between these models. Basic transformations are the addition of items and detailing of components. Next we proceed with the discussion of different media: Multimedia Conversion with the Focus on Continuous Media. It is evident that the major challenge in multimedia research is the systematic treatment of continuous media. When focusing on transformations, we enter the area of streams and converters. As in previous chapters, we again base ourselves on graphs here, for instance chains of converters, yielding a graph of converters. Several qualities are relevant here, such as quality of service, quality of data, and quality of experience. This chapter introduces specific transformations for media-type changers, format changers, and content changers. The focus of Chapter XII is patterns in schema changes: Coherence in Data Schema Transformations: The Notion of Semantic Change Patterns. Here we consider updates of data schemata during system usage (operational schema). When the schema is transformed into a new schema, we try to find coherence. A catalogue of semantic changes is presented, consisting of a number of basic transformations. Several important distinctions are made, for example, between appending an entity and superimposing an entity. Also, we have the redirection of a reference to an owner entity, along with extension and restriction of entity intent. The basic transformations were found during empirical studies in real-life cases. In Chapter XIII, we conclude with the advanced approach: Model Transformations in Designing the ASSO Methodology. The context of this methodology is ease of specifying schemata and schema evolution during system usage. The transformations considered here particularly deal with subtyping (also called is-a relationships). This is covered by the transformation of class hierarchies or more general class graphs. It is evident that schema consistency is one of the properties required. This is based on consistency of class definitions, with in-

xiv

ductive approaches by: (a) requiring that initialization adheres to application constraints, and (b) all operations preserve all constraints.

Conclusions This book contains theory and applications of transformations in the context of information systems development. As data today is frequently moving between systems, system components, persons, departments, and organizations, the need for such transformations is evident. When data is in motion, there is not only a change of place or position. Other aspects are changing as well. The data format may change when it is transferred between systems, while the interpretation of data may vary when it is passed on from one person to another. Moreover, the level of detail may change in the exchange of data between departments or organizations, and the systems development phase of data models may vary, e.g., when implementation-independent data models are mapped to implementation-oriented models. The theory presented in this book will help in the development of new innovative applications. Existing applications presented in this book prove the power of current transformation approaches. We are confident that this book contributes to the understanding, the systematic treatment and refinement, and the education of new and existing transformations.

Further Reading Kovacs, Gy. & van Bommel, P. (1997). From conceptual model to OO database via intermediate specification. Acta Cybernetica, (13), 103-140. Kovacs, Gy. & van Bommel, P. (1998). Conceptual modelling based design of object-oriented databases. Information and Software Technology, 40(1), 1-14. van Bommel, P. (1993, May). A randomised schema mutator for evolutionary database optimisation. The Australian Computer Journal, 25(2), 61-69. van Bommel, P. (1994). Experiences with EDO: An evolutionary database optimizer. Data & Knowledge Engineering, 13, 243-263. van Bommel, P. (1995, July). Database design by computer aided schema transformations. Software Engineering Journal, 10(4), 125-132. van Bommel, P., Kovacs, Gy. & Micsik, A. (1994). Transformation of database populations and operations from the conceptual to the Internal level. Information Systems, 19(2), 175-191.

xv

van Bommel, P., Lucasius, C.B. & Weide, Th.P. van der (1994). Genetic algorithms for optimal logical database design. Information and Software Technology, 36(12), 725-732. van Bommel, P. & Weide, Th.P. van der (1992). Reducing the search space for conceptual schema transformation. Data & Knowledge Engineering, 8, 269-292.

Acknowledgments The editor gratefully acknowledges the help of all involved in the production of this book. Without their support, this project could not have been satisfactorily completed. A further special note of thanks goes also to all the staff at Idea Group Publishing, whose contributions throughout the whole process from inception of the initial idea to final publication have been invaluable. Deep appreciation and gratitude is due to Theo van der Weide and other members of the Department of Information Systems at the University of Nijmegen, The Netherlands, for the discussions about transformations of information models. Most of the authors of chapters included in this book also served as reviewers for chapters written by other authors. Thanks go to all those who provided constructive and comprehensive reviews. Special thanks also go to the publishing team at Idea Group Publishing, in particular to Michele Rossi, Carrie Skovrinskie, Jan Travers, and Mehdi Khosrow-Pour. In closing, I wish to thank all of the authors for their insights and excellent contributions to this book. Patrick van Bommel, PhD Nijmegen, The Netherlands February 2004 [email protected] http://www.cs.kun.nl/~pvb

Section I Fundamentals of Transformations

Transformation-Based Database Engineering

1

Chapter I

Transformation-Based Database Engineering Jean-Luc Hainaut, University of Namur, Belgium

Abstract In this chapter, we develop a transformational framework in which many database engineering processes can be modeled in a precise way, and in which properties such as semantics preservation and propagation can be studied rigorously. Indeed, the transformational paradigm is particularly suited to database schema manipulation and translation, that are the basis of such processes as schema normalization and optimization, model translation, reverse engineering, database integration and federation or database migration. The presentation first develops a theoretical framework based on a rich, wide spectrum specification model. Then, it describes how more complex transformations can be built through predicate-based filtering and composition. Finally, it analyzes two major engineering activities, namely database design and reverse engineering, modeled as goal-oriented schema transformations.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

2 Hainaut

Motivation and Introduction Modeling software design as the systematic transformation of formal specifications into efficient programs, and building CASE1 tools that support it, has long been considered one of the ultimate goals of software engineering. For instance, Balzer (1981) and Fikas (1985) consider that the process of developing a program [can be] formalized as a set of correctness-preserving transformations [...] aimed to compilable and efficient program production. In this context, according to Partsch (1983), “a transformation is a relation between two program schemes P and P’ (a program scheme is the [parameterized] representation of a class of related programs; a program of this class is obtained by instantiating the scheme parameters). It is said to be correct if a certain semantic relation holds between P and P’.” These definitions still hold for database schemas, which are special kinds of abstract program schemes. The concept of transformation is particularly attractive in this realm, though it has not often been made explicit (for instance, as a user tool) in current CASE tools. A (schema) transformation is most generally considered to be an operator by which a data structure S1 (possibly empty) is replaced by another structure S2 (possibly empty) which may have some sort of equivalence with S1. Some transformations change the information contents of the source schema, particularly in schema building (adding an entity type or an attribute) and in schema evolution (removing a constraint or extending a relationship type). Others preserve it and will be called semantics-preserving or reversible. Among them, we will find those which just change the nature of a schema object, such as transforming an entity type into a relationship type or extracting a set of attributes as an independent entity type. Transformations that are proved to preserve the correctness of the original specifications have been proposed in practically all the activities related to schema engineering: schema normalization (Rauh, 1995), DBMS2 schema translation (Hainaut, 1993b; Rosenthal, 1988), schema integration (Batini, 1992; McBrien, 2003), schema equivalence (D’Atri, 1984; Jajodia, 1983; Kobayashi, 1986; Lien, 1982), data conversion (Navathe, 1980; Estiévenart, 2003), reverse engineering (Bolois, 1994; Casanova, 1984; Hainaut, 1993, 1993b), schema optimization (Hainaut, 1993b; Halpin, 1995) database interoperability (McBrien, 2003; Thiran, 2001) and others. The reader will find in Hainaut (1995) an illustration of numerous application domains of schema transformations. The goal of this chapter is to develop and illustrate a general framework for database transformations in which all the processes mentioned above can be

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Transformation-Based Database Engineering

3

formalized and analyzed in a uniform way. We present a wide spectrum formalism in which all the information/data models currently used can be specified, and on which a set of basic transformational operators is defined. We also study the important property of semantics-preservation of these operators. Next, we explain how higher-level transformations can be built through three mechanisms, from mere composition to complex model-driven transformation. The database design process is revisited and given a transformational interpretation. The same exercise is carried out in the next section for database reverse engineering then we conclude the chapter.

Schema Transformation Basics This section describes a general transformational theory that will be used as the basis for modeling database engineering processes. First, we discuss some preliminary issues concerning the way such theories can be developed. Then, we define a wide-spectrum model from which operational models (i.e., those which are of interest for practitioners) can be derived. The next sections are dedicated to the concept of transformation, to its semantics-preservation property, and to the means to prove it. Finally, some important basic transformations are described.

•

Warning. In the database world, a general formalism in which database specifications can be built is called a model. The specification of a database expressed in such a model is called a schema.

Developing Transformational Theories Developing a general purpose transformational theory requires deciding on the specification formalism, i.e., the model, in which the schemas are expressed and on the set of transformational operators. A schema can be defined as a set of constructs (entity types, attributes, keys, indexes, etc.) borrowed from a definite model whose role is to state which constructs can be used, according to which assembly rules, in order to build valid schemas. For simplicity, the concept of entity type is called a construct of the ERA3 model, while entity type CUSTOMER is a construct of a specific schema. They are given the same name, though the latter is an instance of the former. Though some dedicated theories rely on a couple of models, such as those which are intended to produce relational schemas from ERA schemas, the most interesting theories are based on a single formalism. Such a formalism defines

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

4 Hainaut

the reference model on which the operators are built. According to its generality and its abstraction level, this model defines the scope of the theory, that can address a more or less wide spectrum of processes. For instance, building a theory on the relational model will allow us to describe, and to reason on, the transformation of relational schemas into other relational schemas. The 1NF4 normalization theory is a popular example. Another example would be a transformational theory based on the ORM (Object-Role model) that would provide techniques for transforming (normalizing, optimizing) conceptual schemas into other schemas of the same abstraction level (de Troyer, 1993; Proper, 1998). The hard challenge is to choose a unique model that can address not only intramodel transformations, but inter-model operators, such as ORM-to-relational conversion. To identify such models, let us consider a set of models Γ that includes, among others, all the operational formalisms that are of interest for a community of practitioners, whatever the underlying paradigm, the age and the abstraction level of these formalisms. For instance, in a large company whose information system relies on many databases (be they based on legacy or modern technologies) that have been designed and maintained by several teams, this set is likely to include several variants of the ERA model, UML class diagrams, several relational models (e.g., Oracle 5 to 10 and DB2 UDB), the object-relational model, the IDMS and IMS models and of course the standard file structure model on which many legacy applications have been developed. Let us also consider the transitive inclusion relation “≤” such that M ≤ M’, where M≠M’ and M,M’ ∈ Γ, means that all the constructs of M also appear in M’.5 For instance, if M denotes the standard relational model and M’ the objectrelational model, then M ≤ M’ holds, since each schema expressed in M is a valid schema according to model M’. Now, we consider a model M* in Γ, such that: ∀M∈Γ, M≠M*: M ≤ M*, and a model M0 in Γ, for which the following property holds: ∀M∈Γ, M≠M0: M0 ≤ M. (ΓxΓ, ≤) forms a lattice of models, in which M0 denotes the bottom node and M* the upper node. M0, admittedly non-empty, is made up of a very small set of elementary abstract constructs, typically nodes, edges and labels. An ERA schema S comprising an entity type E with two attributes A1 and A2 would be represented in M0 by the Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Transformation-Based Database Engineering

5

nodes n1, n2, n3 which are given the labels “E”, “A1” and “A2”, and by the edges (n1,n2) and (n1,n3). On the contrary, M* will include a greater variety of constructs, each of them being a natural abstraction of one or several constructs of lower-level models. This model should include, among others, the concepts of object type, attribute and inter-object association, so that the contents of schema S will be represented in M* by an object type with name “E” comprising two attributes with names “A1” and “A2”. Due to their high level of abstraction, models M0 and M* are good candidates to develop a transformational theory relying on a single model. Considering the context-dependent definition of Γ, M0 and M*, we cannot assert that these concepts are unique. Therefore, there is no guarantee that a universal theory can be built. Approaches based on M0 generally define data structures as semantics-free binary graphs on which a small set of rewriting operators are defined. The representation of an operational model M such as ERA, relational or XML, in M0 requires some additional features such as typed nodes (object, attribute, association and roles for instance) and edges, as well as ad hoc assembly rules that define patterns. A transformation specific to M is also defined by a pattern, a sort of macro-transformation, defined by a chain of M0 transformations. McBrien (1998) is a typical example of such theories. We can call this approach constructive or bottom-up, since we build operational models and transformations by assembling elementary building blocks. The approaches based on M* naturally require a larger set of rewriting rules. An operational model M is defined by specializing M*, that is, by selecting a subset of concepts and by defining restrictive assembly rules. For instance, a relational schema can be defined as a set of object types (tables), a set of attributes (column), each associated with an object type (at least one attribute per object type) and a set of uniqueness (keys) and inclusion (foreign keys) constraints. This model does not include the concept of association. The transformations of M are those of M* which remain meaningful. This approach can be qualified by specialization or top-down, since an operational model and its transformational operators are defined by specializing (i.e., selecting, renaming, restricting) M* constructs and operators. DB-MAIN (Hainaut, 1996b) is an example of this approach. In the next section, we describe the main aspects of its model, named GER.6

Data Structure Specification Model Database engineering is concerned with building, converting and transforming database schemas at different levels of abstraction, and according to various

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

6 Hainaut

paradigms. Some processes, such as normalization, integration and optimization operate in a single model, and will require intra-model transformations. Other processes, such as logical design, use two models, namely the source and target models. Finally, some processes, among others, reverse engineering and federated database development, can operate on an arbitrary number of models (or on a hybrid model made up of the union of these models) as we will see later on. The GER model is a wide-spectrum formalism that has been designed to:

•

express conceptual, logical and physical schemas, as well as their manipulation,

• •

support all the data-centered engineering processes, and support all DMS7 models and the production and manipulation of their schemas.

The GER is an extended entity-relationship model that includes, among others, the concepts of schema, entity type, entity collection, domain, attribute, relationship type, keys, as well as various constraints. In this model, a schema is a description of data structures. It is made up of specification constructs which can be, for convenience, classified into the usual three abstraction levels, namely conceptual, logical and physical. We will enumerate some of the main constructs that can appear at each level:

•

A conceptual schema comprises entity types (with/without attributes; with/without identifiers), super/subtype hierarchies (single/multiple, total and disjoint properties), relationship types (binary/N-ary; cyclic/acyclic; with/without attributes; with/without identifiers), roles of relationship type (with min-max cardinalities; with/without explicit name; single/multi-entitytype), attributes (of entity or relationship types; multi/single-valued; atomic/ compound; with cardinality), identifiers (of entity type, relationship type, multivalued attribute; comprising attributes and/or roles), constraints (inclusion, exclusion, coexistence, at-least-one, etc.)

•

A logical schema comprises record types, fields, arrays, foreign keys, redundancy, etc.

•

A physical schema comprises files, record types, fields, access keys (a generic term for index, calc key, etc.), physical data types, bag and list multivalued attributes, and other implementation details.

It is important to note that these levels are not part of the model. The schema of Figure 1 illustrates some major concepts borrowed to these three levels. Such a hybrid schema could appear in reverse engineering.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Transformation-Based Database Engineering

7

Figure 1. Typical hybrid schema made up of conceptual constructs (e.g., entity types PERSON, CUSTOMER, EMPLOYEE and ACCOUNT, relationship type of, identifiers Customer ID of CUSTOMER), logical constructs (e.g., record type ORDER, with various kinds of fields including an array, foreign keys ORIGIN and DETAIL.REFERENCE) and physical objects (e.g., table PRODUCT with primary key PRO_CODE and indexes PRO_CODE and CATEGORY, table space PRODUCT.DAT) (Note that the identifier of ACCOUNT, stating that the accounts of a customer have distinct Account numbers, makes it a dependent or weak entity type.) PERSON Name Address

T PRODUCT PRO_CODE CATEGORY DESCRIPTION UNIT_PRICE id: PRO_CODE acc acc: CATEGORY

PRODUCT.DAT PRODUCT

ORDER ORD-ID DATE_RECEIVED ORIGIN DETAIL[1-5] array REFERENCE QTY-ORD id: ORD-ID ref: ORIGIN ref: DETAIL[*].REFERENCE

CUSTOMER Customer ID id: Customer ID

EMPLOYEE Employe Nbr Date Hired id: Employe Nbr

0-N of 1-1 ACCOUNT Account NBR Amount id: of.CUSTOMER Account NBR

One remarkable characteristic of wide spectrum models is that all the transformations, including inter-model ones, appear as intra-model operators. This has highly interesting consequences. First, a transformation Σ designed for manipulating schemas in an operational model M1 can be used in a model M2 as well, provided that M2 includes the constructs on which Σ operates. For instance, most transformations dedicated to COBOL data structure reverse engineering appear to be valid for relational schemas as well. This strongly reduces the number of operators. Secondly, any new model can profit from the techniques and reasoning that have been developed for current models. For instance, designing methods for translating conceptual schemas into object-relational structures or into XML schemas (Estiévenart, 2003), or reverse engineering OO-databases (Hainaut, 1997) have proved particularly easy since these new methods can be, to a large extent, derived from standard ones. The GER model has been given a formal semantics in terms of an extended NF2 model (Hainaut, 1989, 1996). This semantics will allow us to analyze the properties of transformations, and particularly to precisely describe how, and under which conditions, they propagate and preserve the information contents of schemas.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

8 Hainaut

Let us note that we have discarded the UML class model as a candidate for M* due to its intrinsic weaknesses, including its lack of agreed-upon semantics, its non-regularity and the absence of essential concepts. On the contrary, a carefully defined subset of the UML model could be be a realistic basis for constructive approaches.

Specifying Operational Models with the GER In this section, we illustrate the specialization mechanism by describing a popular operational formalism, namely the standard 1NF relational model. All the other models, be they conceptual, logical or physical can be specified similarly. A relational schema mainly includes tables, domains, columns, primary keys, unique constraints, not null constraints and foreign keys. The relational model can therefore be defined as in Figure 2. A GER schema made up of constructs from the first columns only, that satisfy the assembly rules, can be called relational. As a consequence, a relational schema cannot comprise is-a relations, relationship types, multivalued attributes or compound attributes. The physical aspects of the relational data structures can be addressed as well. Figure 3 gives additional specifications through which physical schemas for a specific RDBMS can be specified. These rules generally include limitations such as no more than 64 columns per index, or the total length of the components of any index cannot exceed 255 characters.

Figure 2. Defining standard relational model as a subset of the GER model GER constructs schema

relational constructs

assembly rules

database schema

entity type

table

simple domain

domain

single-valued and atomic attribute with cardinality [0-1]

nullable column

single-valued and atomic attribute with cardinality [1-1]

not null column

primary identifier

primary key

an entity type includes at least one attribute

a primary identifier comprises attributes with cardinality [1-1]

secondary identifier

unique constraint

reference group

foreign key

the composition of the reference group must be the same as that of the target identifier

GER names

SQL names

the GER names must follow the SQL syntax

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Transformation-Based Database Engineering

9

Figure 3. Defining the main technical constructs of relational data structures as they are implemented in a specific RDBMS GER constructs

relational constructs

assembly rules for a specific DBMS

access key

index

comprises from 1 to 64 attributes of the parent entity type

collection

table space

a collection includes 1 to 255 entity types; an entity type belongs to at most 1 collection

Transformation: Definition The definitions that will be stated here are model-independent. In particular, they are valid for the GER model, so that the examples will be given in the latter. Let us denote by M the model in which the source and target schemas are expressed by S the schema on which the transformation is to be applied and by S’ the schema resulting from this application. Let us also consider sch(M), a function that returns the set of all the valid schemas that can be expressed in model M, and inst(S), a function that returns the set of all the instances that comply with schema S. A transformation Σ consists of two mappings T and t (Figure 4):

•

T is the structural mapping from sch(M) onto itself, that replaces source construct C in schema S with construct C’. C’ is the target of C through T, and is noted C’ = T(C). In fact, C and C’ are classes of constructs that can be defined by structural predicates. T is therefore defined by the minimal precondition P that any construct C must satisfy in order to be transformed by T, and the maximal postcondition Q that T(C) satisfies. T specifies the rewriting rule of Σ.

•

t is the instance mapping from inst(S) onto inst(S’), that states how to produce the T(C) instance that corresponds to any instance of C. If c is an instance of C, then c’ = t(c) is the corresponding instance of T(C). t can be specified through any algebraic, logical or procedural expression.

Figure 4. Two mappings of schema transformation Σ ≡ (The inst_of arrow from x to X indicates that x is an instance of X.) C

T

inst_of

c

C' = T(C)

inst_of

t

c' = t(c)

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

10 Hainaut

According to the context, Σ will be noted either or . Each transformation Σ is associated with an inverse transformation Σ' which can undo the result of the former under certain conditions that will be detailed in the next section.

Reversibility of a Transformation The extent to which a transformation preserves the information contents of a schema is an essential issue. Some transformations appear to augment the semantics of the source schema (e.g., adding an attribute), some remove semantics (e.g., removing an entity type), while others leave the semantics unchanged (e.g., replacing a relationship type with an equivalent entity type). The latter are called reversible or semantics-preserving. If a transformation is reversible, then the source and the target schemas have the same descriptive power, and describe the same universe of discourse, although with a different presentation.

•

A transformation Σ1 = = is reversible, iff there exists a transformation Σ2 = = such that, for any construct C, and any instance c of C: P1(C) ⇒ ([T2(T1(C))=C] and [ t2(t1(c)=c]). Σ2 is the inverse of Σ1, but the converse is not true8. For instance, an arbitrary instance c’ of T(C) may not satisfy the property c’=t1(t2(c’)).

•

If Σ2 is reversible as well, then Σ1 and Σ2 are called symmetrically reversible. In this case, Σ2 = . Σ1 and Σ2 are called SRtransformations for short.

Similarly, in the pure software engineering domain, Balzer (1981) introduces the concept of correctness-preserving transformation aimed at compilable and efficient program production. We have discussed the concept of reversibility in a context in which some kind of instance equivalence is preserved. However, the notion of inverse transformation is more general. Any transformation, be it semantics-preserving or not, can be given an inverse. For instance, del-ET(et_name), which removes entity type with name et_name from its schema, clearly is not a semantics-preserving operation, since its mapping t has no inverse. However, it has an inverse transformation, namely create-ET(CUSTOMER). Since only the T part is defined, this partial inverse is called a structural inverse transformation.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Transformation-Based Database Engineering

11

Proving the Reversibility of a Transformation Thanks to the formal semantics of the GER, a proof system has been developed to evaluate the reversibility of a transformation. More precisely, this system relies on a limited set of NF2 transformational operators whose reversibility has been proven, and that can generate a large number of GER transformations. Basically, the system includes five families of transformations, that can be combined to form more complex operators:

•

denotation, through which a new object set is defined by a derivation rule based on existing structures,

• •

project-join which is a variant of the decomposition theorem,

• •

nest-unnest, the typical 1NF ↔ N1NF operators, and

composition which replaces two relations by one of them and their composition, container, that states the equivalence between non-set containers (e.g., bags, lists, arrays) and sets .

Thanks to a complete set of mapping rules between the GER model and the NF2 model in which these basic transformations have been built, the latter can be applied to operational schemas. Figure 5 shows how we have defined a decomposition operator for normalizing relationship types from the basic projectjoin transformation. It is based on a three-step process: 1.

Source schema (Figure 5, top-left) is expressed in the NF2 formalism (bottom-left): {entities:A,B,C; R(A,B,C); A → B}

2.

Basic project-join transformation is applied and yields a normalized relational schema (bottom-right): {entities:A,B,C; R1(A,B); R2(A,C); R1[A]=R2[A]}

3.

NF2 schema is expressed in the GER, leading to the target schema (Figure 5, top-right).

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

12 Hainaut

Figure 5. Proving the SR property of the decomposition of a relationship type according to a multivalued dependency (here an FD) source schema A

target schema B

0-N

0-N

GER

A

T1

R

⇒ ⇐

0-N

T1'

coex: R2.C[*] R1.B

R1

0-N

C

0-N

B

0-N R2

C

0-1

R: A → B T11 ⇓⇑ T11'

NF2

entities: A,B,C R(A,B,C) A

→B

T13 ⇑ T12

⇒ ⇐

T12'

⇓ T13'

entities:A,B,C R1(A,B) R2(A,C) R1[A]=R2[A]

Since the the GER ↔ NF2 mappings are symmetrically reversible and the project-join is an SR-transformation, the ERA transformation is symmetrically reversible as well. It can be defined as follows: T1 = T11οT12 οT13 T1' = T11'οT12'οT13' We note the important constraint R1[A]=R2[A] that gives the project-join transformation the SR property, while Fagin’s theorem merely defines a reversible operator. We observe how this constraint translates into a coexistence constraint in the GER model that states that if an A entity is connected to a B entity, it must be connected to at least one C entity as well, and conversely. The reader interested in a more detailed description of this proof system is refered to Hainaut (1996).

Six Mutation Transformations A mutation is an SR-transformation that changes the nature of an object. Considering the three main natures of object, namely entity type, relationship type and attribute, six mutation transformations can be defined. In Figure 6, the couples of operators Σ1 to Σ3, show them applied on typical schema fragments. The transformations Σ4 are not primitive since they can be defined by combining other mutations. However, they have been added due to their usefulness. More

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Transformation-Based Database Engineering

13

Figure 6. Six mutation transformations Σ1 to Σ3 (Though not primitive, compound transformations Σ4 are shown as well.) source schema T1

Σ1

Σ2

A

A A1 id: A1

0-N

0-N

r

0-5

r

0-5

B

B

⇒ ⇐

comment

R

Transforming relationship type r into entity type R (T1) and conversely (T1'). Note that R entities are identified by any couple (a,b) ∈ AxB through relationship types rA and rB (id:ra.A,rB.B).

A

B

0-N

0-5

T1'

rA

T2

A A1 id: A1

⇒ ⇐

target schema

1-1

id: rA.A rB.B

1-1

rB

B A1[0-5] ref: A1[*]

Transforming relationship type r into reference attribute B.A1 (T2) and conversely (T2').

T2'

Σ3

A A1 A2[0-5] A3

T3

⇒ ⇐

A A1 A3

0-5

ra2

1-N

EA2 A2 id: A2

Transforming attribute A2 into entity type EA2 (T3) and conversely (T3').

T3'

Σ4

A A1 A2[0-5] A3

T4

⇒ ⇐

A A1 A3

0-5

T4'

ra2

1-1

EA2 A2 id: ra2.A A2

Not a primitive operator. T4 can be defined by application of T3 to EA2.A2, then of T1' to EA2 in the schema above. Note that the EA2 entities depending on the same A entity have distinct A2 values (id:ra2.A,A2).

sophisticated mutation operators can be defined as illustrated in Hainaut (1991) in the range of entity-generating transformations.

Other Basic Transformations The mutation transformations can solve many database engineering problems, but other operators are needed to model special situations. The CASE tool associated with the DB-MAIN methodologies includes a kit of about 30 basic operators that have proven sufficient for most engineering activities. When necessary, user-defined operators can be developed through the meta functions of the tool (Hainaut, 1996b). We will describe some of the basic operators. Expressing supertype/subtype hierarchies in DMS that do not support them explicitly is a recurrent problem. The technique of Figure 7 is one of the most commonly used (Hainaut, 1996c). It consists in representing each source entity type by an independent entity type, then to link each subtype to its supertype through a one-to-one relationship type. The latter can, if needed, be further transformed into foreign keys by application of Σ2-direct (T2).

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

14 Hainaut

Figure 7. Transforming an is-a hierarchy into one-to-one relationship types and conversely source schema

target schema

A A1 A2

A

Σ5

T5

⇒ ⇐

D C C1 C2

B B1 B2

T5'

r

0-1

1-1

A1 A2 excl: s.C r.B

0-1

B B1 B2

comment

s 1-1 C C1 C2

An is-a hierarchy is replaced by one-to-one relationship types. The exclusion constraint (excl:s.C,r.B) states that an A entity cannot be simultaneously linked to a B entity and a C entity. It derives from the disjoint property (D) of the subtypes.

Figure 8. Converting an array into a set-multivalued attribute and conversely source schema

Σ6

A A1 A2[0-5] array A3

target schema T6

⇒ ⇐ T6'

A A1 A2[0-5] Index Value A3 id(A2): Index

comment Array A2 (left) is transformed into a multivalued compound attribute A2 (right), whose values are distinct wrt component Index (id(A2):Index). The latter indicates the position of the value (Value). The domain of Index is the range [1..5].

Figure 9. Transforming serial attributes into a multivalued attribute and conversely source schema A

Σ7

A1 A2X A2Y A2Z A3

target schema T7

A

T7'

A1 A2[3-3] Dimension Value A3 id(A2): Dimension

⇒ ⇐

dom(A2.Dimension) = {'X','Y','Z'}

comment The serial attributes {A2X, A2Y, A2Z} are transformed into the multivalued compound attribute A2 where the values (Value) are indexed with the distinctive suffix of the source attributes, interpreted as a dimension (sub-attribute Dimension, whose domain is the set of prefixes).

Transformations Σ3 and Σ4 show how to process standard multivalued attributes. When the collection of values is no longer a set but a bag, a list or an array, operators to transform them into pure set-oriented constructs are most useful. Transformations Σ6 in Figure 8 are dedicated to arrays. Similar operators have been defined for the other types of containers. Attributes defined on the same domain and the name of which suggests a spatial or temporal dimension (e.g., departments, countries, years or pure numbers) are called serial attributes. In many situations, they can be interpreted as the

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Transformation-Based Database Engineering

15

representation of an indexed multivalued attributes (Figure 9). The identification of these attributes must be confirmed by the analyst.

Higher-Level Transformations The transformations described in the section, Schema Transformation Basics, are intrinsically atomic: one elementary operator is applied to one object instance, and (Σ4 excluded) none can be defined by a combination of others (orthogonality). This section develops three ways through which more powerful transformations can be developed.

Compound Transformations A compound transformation is made up of a chain of more elementary operators in which each transformation applies on the result of the previous one. The transformation Σ8 in Figure 10, illustrated by a concrete example, transforms a complex relationship type R into a sort of bridge entity type comprising as many foreign keys as there are roles in R. It is defined by the composition of Σ1-direct and Σ2-direct. This operator is of frequent use in relational database design. The transformation Σ9 is more complex (Figure 11). It is composed of a chain of four elementary operators. The first one transforms the serial attributes Expense-2000, ..., Expense-2004 into multivalued attribute Expense comprising sub-attributes Year (the dimension) and Amount (transformation Σ7-direct). The second one extracts this attribute into entity type EXPENSE, with attributes Year and Amount (transformation Σ4-direct). Then, the same operator is applied to Figure 10. Transformation of a complex relationship type into relational structures source schema COMPANY Cy_Name id: Cy_Name

COUNTRY Ctry_Name id: Ctry_Name

0-N

Σ8

0-N

export Volume 0-N PRODUCT Prod_ID id: Prod_ID

target schema

T8

⇒ ⇐ T8'

EXPORT Prod_ID Ctry_Name Cy_Name Volume id: Ctry_Name Prod_ID Cy_Name ref: Cy_Name ref: Prod_ID ref: Ctry_Name

COMPANY Cy_Name id: Cy_Name

PRODUCT Prod_ID id: Prod_ID

comment The relationship type export is first transformed into an entity type + three many-to-one relationship types. Then, the latter are converted into foreign keys.

COUNTRY Ctry_Name id: Ctry_Name

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

16 Hainaut

Figure 11. Extracting a temporal dimension from serial attributes source schema

Σ9

Project Dep# InitialBudget Expense-2000 Expense-2001 Expense-2002 Expense-2003 Expense-2004

target schema T9

⇒ ⇐ T9'

Project Dep# InitialBudget 5-5 expense Amount 1-N YEAR Year id: Year

comment The serial attributes are first transformed into a multivalued attribute, which in turn is extracted as external entity type EXPENSE. The dimension attribute (Year) is also extracted as entity type YEAR. Finally, EXPENSE is mutated into relationship type expense.

dom(Year) = [2000..2004]

Figure 12.

Three examples of predicate-driven transformation

predicate-driven transformation RT_into_ET(ROLE_per_RT(3 N))

RT_into_REF(ROLE_per_RT(2 2) and ONE_ROLE_per_RT(1 2)) INSTANTIATE(MAX_CARD_of_ATT(2 4))

interpretation transform each relationship type R into an entity type (RT_into_ET), if the number of roles of R (ROLE_per_RT) is in the range [3 N]; in short, convert all N-ary relationship types into entity types. transform each relationship type R into reference attributes (RT_into_ET), if the number of roles of R is 2 and if R has from 1 to 2 one role(s), i.e., R has at least one role with max cardinality 1; in short, convert all one-to-many relationship types into foreign keys. transform each attribute A into a sequence of single-value instances, if the max cardinality of A is between 2 and 4; in short, convert multivalued attributes with no more than 4 values into serial attributes.

attribute Year, yielding entity type YEAR, with attribute Year. Finally, the entity type EXPENSE is transformed into relationship type expense (Σ1-inverse).

Predicate-Driven Transformations A predicate-driven transformation Σp applies an operator Σ to all the schema objects that meet a definite predicate p. It will be specified by Σ(p). p is a structural predicate that states the properties through which a class of patterns can be identified. Interestingly, a predicate-based transformation can be interpreted as a user-defined elementary operator. Indeed, considering the standard definition Σ = , we can rewrite Σp as Σ*(true) where Σ* = . In general, the inverse of Σp cannot be derived from the expression of Σ and p. Indeed, there is no means to derive the predicate p’ that identifies the constructs resulting from the application of Σp, and only them. We give in Figure 12 some useful transformations that are expressed in the specific language of the DB-MAIN tool, which follows the Σ(p) notation. Most predicates are parametric; for instance, the predicate ROLE_per_RT( ), where and are integers such that ≤ states that the number

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Transformation-Based Database Engineering

17

of roles of the relationship type falls in the range [..]. The symbol “N” stands for infinity.

Model-Driven Transformations A model-driven transformation is a goal-oriented compound transformation made up of predicate-driven operators. It is designed to transform any schema expressed in model M into an equivalent schema in model M’. As illustrated in the discussion of the relational model expressed as a specialization of the GER (Figure 2), identifying the components of a model also leads to identifying the constructs that do not belong to it. Except when M ≤ M’, an arbitrary schema S expressed in M may include constructs which violate M’. Each construct that can appear in a schema can be specified by a structural predicate. Let PM denote the set of predicates that defines model M and P M’ that of model M’. In the same way, each potentially invalid construct can be specified by a structural predicate. Let PM/M’ denote the set of the predicates that identify the constructs of M that are not valid in M’. In the DB-MAIN language used in Figure 12, ROLE_per_RT(3 N) is a predicate that identifies N-ary relationship types that are invalid in DBTG CODASYL databases, while MAX_CARD_of_ATT(2 N) defines the family of multivalued attributes that is invalid in the SQL2 database model. Finally, we observe that each such set as PM can be perceived as a single predicate formed by anding its components. Let us now consider predicate p ∈ P M/M’ , and let us choose a transformation Σ = such that: (p ⇒ P) ∧ (PM’ ⇒ Q) Clearly, the predicate-driven transformation Σp solves the problem of invalid constructs defined by p. Proceeding in the same way for each component of PM/M’ provides us with a series of operators that can transform any schema in model M into schemas in model M’. We call such a series a transformation plan, which is the practical form of any model-driven transformation. In real situations, a plan can be more complex than a mere sequence of operations, and may comprise loops to process recursive constructs for instance. In addition, transformations such as those specified above may themselves be compound, so that the set of required transformations can be quite large. In such cases, it can be better to choose a transformation that produces constructs that are not fully compliant with M’, but that can be followed by other operators which complete the job. For instance, transforming a multivalued attribute can be

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

18 Hainaut

obtained by an ad hoc compound transformation. However, it can be thought more convenient to first transform the attribute into an entity type + a one-tomany relationship type (Σ4-direct), which can then be transformed into a foreign key (Σ2-direct). This approach produces transformation plans which are more detailed and therefore less readable, but that rely on a smaller and more stable set of elementary operators. The transformation toolset of DB-MAIN includes about 30 operators that have proven sufficient to process schemas in a dozen operational models. If all the transformations used to build the plan have the SR-property, then the modeldriven transformation that the plan implements is symmetrically reversible. When applied to any source schema, it produces a target schema semantically equivalent to the former. This property is particularly important for conceptual→logical transformations. Figure 13 sketches, in the form of a script, a simple transformation plan intended to produce SQL2 logical schemas from ERA conceptual schemas. Actual plans are more complex, but follow the approach developed in this section. It must be noted that this mechanism is independent of the process we are modeling, and that similar transformation plans can be built for processes such as conceptual normalization or reverse engineering. Though model-driven transformations provide an elegant and powerful means of specification of many aspects of most database engineering processes, some other aspects still require human expertise that cannot be translated into formal rules.

Figure 13. Simple transformation plan to derive a relational schema from any ERA conceptual schema (To make them more readable, the transformations have been expressed in natural language instead of in the DB-MAIN language. The term rel-type stands for relationship type.) step

predicate-based transformation

1

transform IS-A relations into one-to-one rel-types

operator Σ5-direct;

2

transform complex rel-types into entity types

operator Σ1-direct; complex means N-ary or binary many-to-many or with attributes;

3

disaggregate level-1 compound attributes

4

transform level-1 multivalued attributes into entity types

5

repeat steps 3 to 4 until the schema does not include complex attributes any more

6

transform relationship types into reference groups at this point, only one-to-many and one-to-one rel-types subsist; they are transformed into foreign keys;

7

if the schema still includes rel-types, add a step 6 fails in case of missing identifier; a technical technical identifier to the relevant entity types and attribute is associated with the entity type that will be apply step 6 referenced by the future foreign key;

comment

each compound attribute directly depending on an entity type is replaced by its components; operator Σ4-direct; each multivalued attribute directly depending on an entity type is replaced by an entity type; to cope with multi-level attribute structures;

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Transformation-Based Database Engineering

Figure 14.

19

Standard strategy for database design

Database design

Users requirements

Conceptual design

Conceptual schema

Logical design

Logical schema

Physical design

Physical schema

Coding

View design

Users views

Operational code

Transformation-Based Database Design Most textbooks on database design of the eighties and early nineties propose a five-step approach that is sketched in Figure 14. Through the Conceptual Design phase, users’ requirements are translated into a conceptual schema, which is the formal and abstract expression of these requirements. The Logical Design phase transforms the conceptual schema into data structures (the logical schema) that comply with the data model of a family of DMS such as relational, OO or standard file data structures. Through the Physical Design phase, the logical schema is refined and augmented with technical specifications that make it implementable into the target DMS and that gives it acceptable performance. From the logical schema, users’ views are derived that meet the requirements of classes of users (View Design). Finally, the physical schema and the users views are coded into the DDL9 of the DMS (Coding).

Database Design as a Transformation Process Ignoring the view design process for simplification, database design can be modeled by (the structural part of) transformation DB-design: code = DB-design(UR) where code denotes the operational code and UR the users requirements.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

20 Hainaut

Denoting the conceptual, logical and physical schemas respectively by CS, LS and PS and the conceptual design, logical design, physical design and coding phases by C-design, L-design, P-design and Coding, we can refine the previous expression as follows: CS = C-design(UR) LS = L-design(CS) PS = P-design(LS) code = Coding(PS) Clearly, these processes are model-driven transformations and can then be described by transformation plans. The level of formality of these processes depends on the methodology, on the existence of CASE support and of nonfunctional requirements such as performance and robustness, that generally require human expertise. For instance, conceptual design (C-design) is a highly informal process based on human interpretation of complex information sources, while logical design can be an automated process completely described by a transformation plan. Anyway, these processes can be decomposed into subprocesses that, in turn, can be modeled by transformations and described by transformation plans, and so forth, until the latter reduce to elementary operators. Three of these processes are worth being examined a bit further.

Conceptual Design This process includes, among others, two major sub-processes, namely Basic Analysis, through which informal or semi-formal information sources are analyzed and their semantic contents are translated into conceptual structures, and (Conceptual) Normalization, through which these raw structures are given such additional qualities as readability, normality, minimality, extensibility, compliance with representation standards, etc. (Batini, 1992; Blaha, 1998). This second process is more formal than the former, and is a good candidate for transformational modeling. The plan of Figure 15, though simplistic, can improve the quality of many raw conceptual schemas.

Logical Design As shown in the preceding sections, this process can be specified by a modelbased transformation. In fact, we have to distinguish two different approaches,

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Transformation-Based Database Engineering

Figure 15. schemas step

21

Simple transformation plan to normalize ERA conceptual

predicate-based transformation

comment

1

transform attribute entity types into attributes

an attribute entity type has key attributes only, and is linked to one other entity type only through a mandatory rel-type; operator Σ3-inverse and Σ4-inverse;

2

transform relationship entity types into rel-types

a relationship entity type has few attributes, is linked to at least 2 entity types through mandatory many-to-one rel-types and is identified by these entity types; operator Σ1-inverse;

3

decompose N-ary rel-types where a non full-key FD holds

transformation of Fig. 5;

4

remove transitive IS-A relations

if A is-a B and B is-a C, then A is-a C cannot be explicitly asserted;

5

transform complex multivalued attributes into entity types

a complex attribute is compound, multivalued and has a local identifier; it is assumed to express an autonomous entity type; operator Σ4-inverse;

namely ideal and empirical. The ideal design produces a logical schema that meets two requirements only: it complies with the target logical model M and it is semantically equivalent to the conceptual schema. According to the transformational paradigm, the logical design process is a M-driven transformation comprising SR-operators only. The plan of Figure 13 illustrates this principles for relational databases. Similar plans have been designed for CODASYL DBTG, Object-relational and XML (Estievenart, 2003) databases, among others. Empirical design is closer to the semi-formal way developers actually work, relying on experience and intuition, rather than on standardized procedures. Other requirements such as space and time optimization often are implicitly taken into account, making formal modeling more difficult, if not impossible. Though no comprehensive model-driven transformations can describe such approaches, essential fragments of empirical design based on systematic and reproducible rules can be described by compound or predicate-driven transformations.

Coding Quite often overlooked, this process can be less straightforward and more complex than generally described in the literature or carried out by CASE tools. Indeed, any DMS can cope with a limited range of structures and integrity constraints for which its DDL provides an explicit syntax. For instance, plain SQL2 DBMSs know about constraints such as machine value domains, unique keys, foreign keys and mandatory columns only. If such constructs appear in a physical schema, they can be explicitly declared in the SQL2 script. On the other hand, all the other constraints must be either ignored or expressed in any other

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

22 Hainaut

way, at best through check predicates or triggers, but more frequently through procedural sections scattered throughout the application programs. Distinguishing the DDL code from the external code, the operational code can be split into two distinct parts: code = codeddl ∪ codeext Despite this variety of translation means, the COD process typically is a twomodel transformation (in our framework, GER to DMS-DDL) that can be automated.

Transformation-Based Database Reverse Engineering Database reverse engineering is the process through which one attempts to recover or to rebuild the technical and functional documentation of a legacy database. Intensive research in the past decade have shown that reverse engineering generally is much more complex than initially thought. We can put forward two major sources of difficulties. First, empirical design has been, and still is, more popular than systematic design. Second, only the codeddl part of the code provides a reliable description of the database physical constructs. Empirical design itself accounts for two understanding problems. First, it often relies on non-standard, unpublished, translation rules that may be difficult to interpret. Second, actual logical schemas often are strongly optimized, so that extracting a conceptual schema from the logical schema involves understanding not only how the latter has been translated in the target model, but also how, and according to which criteria, it has been optimized. The codeddl component expresses a part of the physical schema only. Therefore, the codeext part must be retrieved and interpreted, which leads to two independent problems. The first one requires parsing a huge volume of program code to identify code sections that cope with implicit, i.e., undeclared, constructs such as decomposed (flattened) fields or referential constraints. The second problem concerns the correct interpretation of these code fragments, that translates into constructs to be added to the physical schema. The whole process is described in Figure 16. It shows that database reverse engineering is decomposed into two main sub-processes, namely Extraction and Conceptualization. The objective of the Extraction process is to recover the

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Transformation-Based Database Engineering

Figure 16.

23

Main processes of database reverse engineering Logical schema

Conceptual schema

Logical schema

Cleaning

Extraction

Database reverse engineering

Conceptualization

Physical schema

Parsing

Refinement

codeddl

codeext

complete logical schema of the legacy database. It includes three activities: Parsing the DDL code to extract the raw physical schema, schema Refinement through which implicit and hidden constructs are elicited from external code (as well as from other sources, such as the data themselves, but we will ignore them in this discussion) and Cleaning, in which the technical constructs of the physical schema are removed. The second main sub-process, Conceptualization, is intended to derive a plausible conceptual schema from the logical schema. It consists in identifying the trace of the translation of conceptual constructs, then in replacing them with their source. For instance, a foreign key is interpreted as (i.e., replaced by) a many-to-one relationship type. The transformational interpretation of the reverse engineering process is straighforward: CS = DBRE(code) where code denotes operational code and CS the conceptual schema. DBRE can be developed as follows: LS = Extraction(code) CS = Conceptualization(LS)

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

24 Hainaut

Extraction itself includes three processes: PS = Parsing(codeddl) PS = Refinement(PS,codeext) LS = Cleaning(PS) By comparing the transformational expression of the database design and database reverse engineering processes, we can state the following equivalence rules, in which, as usual, Σ’ denotes the inverse of transformation Σ: Parsing + Refinement = Coding’ Cleaning = P-design’ Conceptualization = L-design’ Through this analysis, we have shown that, if database design and reverse engineering can be modeled by transformations, then database reverse engineering is, to a large extent, the inverse of database design. This induces important consequences. In particular:

•

database reverse engineering requires a deep understanding of empirical database design methodologies, and

•

the Conceptualization process can be analyzed and specified by identifying the strategies and the transformations that are most popular in empirical logical design, and by considering their inverse.

Among the operators that have been described, the transformations Σ1-inverse, Σ2-inverse, Σ3-direct and Σ3-inverse, Σ4-direct and Σ4-inverse, Σ5-inverse, Σ6direct, Σ7-direct, form a sound (but unfortunately not complete) basis for conceptualizing logical schemas. This process can be supported by predicatedriven and model-driven transformations, but, even more than for forward engineering, reverse engineering heavily relies on human expertise. An in-depth description of a wide-scope reverse engineering methodology can be found in Hainaut (2002).

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Transformation-Based Database Engineering

25

Conclusions and Perspectives In this chapter, we have shown that schema transformation can be used as a major paradigm in database engineering. In particular, being formally defined, it can be used to precisely model complex processes and to reason on their properties such as semantics preservation. It has also been used to derive new processes from former ones, as illustrated by the formalization of database reverse engineering as the inverse of database design. Due to their formality, transformations can be implemented in CASE tools, either as implicit operators, or as tools that are explicitly made available to the developer. Two implementations are worth being mentioned, namely Rosenthal (1994) and Hainaut (1996b). The latter reference describes the DB-MAIN CASE environment which includes a transformation toolbox as well as special engines for user-defined, predicate-driven and model-driven transformations. Further information can be found at http://www.info.fundp.ac.be/libd. Transformations also have a great potential in other domains, such as database interoperability, in which mediation between existing databases (McBrien, 2003) and data wrapping (Thiran, 2001) can be formalized and automated thanks to the use of transformational operators. In this domain, data instance transformations are modeled by the t part of the transformations. Specifying how the source schema is transformed into the target schema automatically provides a chain of instance transformations that are used to generate the data conversion code that is at the core of data migrators (ETL processors), wrappers and mediators.

References Balzer, R. (1981). Transformational implementation: An example. IEEE TSE, SE-7(1). Batini, C., Ceri, S. & Navathe, S. B. (1992). Conceptual Database Design. Benjamin/Cummings. Blaha, M. & Premerlani, W. (1998). Object-oriented Modeling and Design for Database Applications. Prentice Hall. Bolois, G. & Robillard, P. (1994). Transformations in reengineering techniques. Proceedings of the Fourth Reengineering Forum ‘Reengineering in Practice’. Victoria, Canada. Casanova, M. & Amaral De Sa, A. (1984). Mapping uninterpreted schemes into entity-relationship diagrams: Two applications to conceptual schema design. IBM Journal of Research & Development, 28(1). Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

26 Hainaut

D’Atri, A. & Sacca, D. (1984). Equivalence and mapping of database schemes. Proceedings of the 10th VLDB Conference. Singapore. De Troyer, O. (1993). On Data Schema Transformation. PhD thesis, University of Tilburg, Tilburg, The Netherlands. Estiévenart, F., François, A., Henrard, J. & Hainaut, J.L. (2003). Web site engineering. Proceedings of the Fifth International Workshop on Web Site Evolution, Amsterdam, September 2003. IEEE CS Press. Fagin, R. (1977). Multivalued dependencies and a new normal form for relational databases. ACM TODS, 2(3). Fikas, S. F. (1985). Automating the transformational development of software. IEEE TSE, SE-11. Hainaut, J.L. (1989). A generic entity-relationship model. Proceedings of the IFIP WG 8.1 Conference on Information System Concepts: An Indepth Analysis, North-Holland. Hainaut, J.L. (1991). Entity-generating schema transformations for entityrelationship models. Proceedings of the 10th Entity-Relationship Approach, San Mateo (CA), North-Holland. Hainaut, J.L. (1995). Transformation-based database engineering. Tutorial notes, VLDB’95, Zürich, Switzerland, September 1995. Retrieved from the WWW: http://www.info.fundp.ac.be/libd Hainaut, J.L. (1996). Specification preservation in schema transformations Application to semantics and statistics. Data & Knowledge Engineering, 11(1). Hainaut, J.L. (2002). Introduction to Database Reverse Engineering, LIBD lecture notes. Retrieved from the WWW: http://www.info.fundp.ac.be/ ~dbm/publication/2002/DBRE-2002.pdf Hainaut, J.L., Chandelon, M., Tonneau, C. & Joris, M. (1993). Contribution to a theory of database reverse engineering. Proceedings of the IEEE Working Conference on Reverse Engineering, Baltimore, May 1993. IEEE Computer Society Press. Hainaut, J.L., Chandelon, M., Tonneau, C. & Joris, M. (1993b). Transformational techniques for database reverse engineering. Proceedings of the 12th International Conference on ER Approach, Arlington-Dallas, ER Institute. Hainaut, J.L., Englebert, V., Henrard, J., Hick, J.M. & Roland, D. (1996b). Database reverse engineering: From requirements to CASE tools. Journal of Automated Software Engineering, 3(1). Hainaut, J.L., Henrard, J., Hick, J.M., Roland, D. & Englebert, V. (1997). Contribution to the reverse engineering of OO applications - Methodology

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Transformation-Based Database Engineering

27

and case study. Proceedings of the IFIP 2.6 WC on Database Semantics (DS-7), Leysin (CH), October 1997. Chapman-Hall. Hainaut, J.L., Hick, J.M., Englebert, V., Henrard, J. & Roland, D. (1996c). Understanding implementations of IS-A relations. Proceedings of the Conference on the ER Approach, Cottbus, October 1996, LNCS. Springer-Verlag. Halpin, T. A. & Proper, H. A. (1995). Database schema transformation and optimization. Proceedings of the 14th International Conference on ER/ OO Modelling (ERA). Jajodia, S., Ng, P. A. & Springsteel, F. N. (1983). The problem of equivalence for entity-relationship diagrams. IEEE Trans. on Software Engineering, SE, 9(5). Kobayashi, I. (1986). Losslessness and semantic correctness of database schema transformation: Another look of schema equivalence. Information Systems, 11(1), 41-59. Lien, Y. E. (1982). On the equivalence of database models. JACM, 29(2). McBrien, P. & Poulovassilis, A. (1998). A general formal framework for schema transformation. Data & Knowledge Engineering, 28(1), 47-71. McBrien, P. & Poulovassilis, A. (2003). Data integration by bi-directional schema transformation rules. Proceedings of 19th International Conference on Data Engineering (ICDE’03). IEEE Computer Society Press. Navathe, S. B. (1980, June). Schema analysis for database restructuring. ACM TODS, 5(2). Partsch, H. & Steinbrüggen, R. (1983). Program transformation systems. Computing Surveys, 15(3). Proper, H.A. & Halpin, T.A. (1998). Database schema transformation & optimization. Proceedings of the 14th International Conference on Conceptual Modeling, LNCS, 1021, 191-203. Springer. Rauh, O. & Stickel, E. (1995). Standard transformations for the normalization of ER schemata. Proceedings of the CAiSE’95 Conference, Jyväskylä, Finland, LNCS. Springer-Verlag. Rosenthal, A. & Reiner, D. (1988). Theoretically sound transformations for practical database design. Proceedings of Entity-Relationship Approach. Rosenthal, A. & Reiner, D. (1994). Tools and transformations - Rigourous and otherwise - for practical database design. ACM TODS, 19(2). Thiran, Ph. & Hainaut, J.L. (2001). Wrapper development for legacy data reuse. Proceedings of WCRE’01. IEEE Computer Society Press.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

28 Hainaut

Endnotes 1

Computer-aided Software Engineering.

2

Database Management System.

3

Entity-relationship-attribute model. The UML class model is a variant of the ERA model.

4

1NF, or First Normal Form designates the class of relations defined on simple domains (which are neither relations nor powersets). By contrast, a non 1NF relation is said to be in N1NF, or NF2 for short.

5

Defining more formally what the assertion this construct of M also belongs to M’ exactly means would require a development which would be useless in this paper. Therefore, we will rely on an intuitive meaning of this relation only. For example, the concepts of field and of column will be considered the same though some slight differences exist between them. The same can be said for entity type (ERA), object class (UML), segment type (IMS), record type (standard files, CODASYL) and table (SQL2).

6

For Generic Entity-Relationship model.

7

For Data Management System, a term that encompasses file managers and DBMSs.

8

The so-called decomposition theorem of the 1NF relational theory (Fagin, 1977) is an example of reversible transformation. Grossly sketched, it states that the schema {R(A,B,C); A→→BC} can be losslessly replaced by {R1(A,B); R2(A,C)}, since, for any instance r of R, the relation r = r[A,B]*r[A,C] holds. However, there is no reason for any arbitrary instances r1 of R1 and r2 of R2 to enjoy the inverse property r1 = (r1*r2)[A,B]. Therefore, this transformation is not symmetrically reversible. This example and some of its variants are developed in (Hainaut, 1996).

9

Data Definition Language: that part of a database language intended to declare the data structures of the database.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Rule-Based Transformation of Graphs and the Product Type 29

Chapter II

Rule-Based Transformation of Graphs and the Product Type Renate Klempien-Hinrichs, University of Bremen, Germany Hans-Jörg Kreowski, University of Bremen, Germany Sabine Kuske, University of Bremen, Germany

Abstract This chapter presents rule-based graph transformation as a framework for modeling data-processing systems. It recalls the structuring concept of graph transformation units which allows for transforming graphs in a rulebased, modularized, and controlled way. In order to get a flexible typing mechanism and a high degree of parallelism, this structuring concept is extended to the product of transformation units. Moreover, it is demonstrated how the product type can be used to transform graph transformation units.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

30

Klempien-Hinrichs, Kreowski, and Kuske

The authors advocate rule-based graph transformation for all applications where data, knowledge, and information can be modeled as graphs and their transformation can be specified by means of rules in an adequate way.

Introduction The area of graph transformation brings together the concepts of rules and graphs with various methods from the theory of formal languages and from the theory of concurrency, and with a spectrum of applications (Figure 1). Graphs are important structures in computer science and beyond to represent complex system states, networks, and all kinds of diagrams. The application of rules provides graphs with a dynamic dimension yielding a rich methodology of rule-based graph transformation. The three volumes of the Handbook of Graph Grammars and Computing by Graph Transformation give a good overview of the state of the art in theory and practice of graph transformation (Rozenberg, 1997; Ehrig, Engels, Kreowski & Rozenberg, 1999; Ehrig, Kreowski, Montanari & Rozenberg, 1999). Although one encounters quite a large number of different approaches to graph transformation in the literature, nearly all of them contain five basic features.

•

Graphs to represent complex relations among items in an intuitive but mathematically well-understood way.

• •

Rules to describe possible changes and updates of graphs in a concise way. Rule applications to perform the possible changes and updates on graphs explicitly as they are embodied in the rules.

•

Graph class expressions to specify special classes of graphs to be used as initial as well as terminal graphs.

•

Control conditions to regulate the applications of rules such that the inherent non-determinism of rule application can be cut down.

Figure 1. Main ingredients of graph transformation

rules

theory

graphs

application

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Rule-Based Transformation of Graphs and the Product Type 31

A particular choice of these five features establishes a graph transformation approach. A selection of rules, initial and terminal graphs, and a control condition is often called a graph transformation system or a graph grammar if there is a single initial graph as axiom (or a finite set of initial graphs likewise). Following Kreowski and Kuske (1999a), we use the term graph transformation unit for such a selection where we also allow importing other graph transformation units for structuring purposes. In this chapter, we recall the elementary features of graph transformation and – based on it – discuss some new concepts that enhance the usefulness of graph transformation. As graphs are derived from graphs by applying rules, the obvious semantics is a binary relation on graphs or a binary relation between initial and terminal graphs if one provides a subtyping mechanism. To overcome this quite restricted kind of typing, we introduce product types. The basic notion is a product of graph transformation units that comprises tuples of graphs to be processed component-wise, but where the transformations of the components run in parallel. The product, together with typical operations on products like embedding and projection, provides a very flexible kind of typing because one can declare a sequence of input components and a sequence of output components independently. To transform input graphs into output graphs, all components are combined into a proper product of graph transformation units. If one controls the parallel transformations of the components suitably, one can get the desired interrelations between input and output graphs. We then demonstrate that the product type is also quite useful if one wants to transform graph transformation units.

Graph Transformation In this section we introduce main concepts of graph transformation like graphs, rules, and transformation units. The concepts are illustrated with a simple example from the area of graph theory. In the literature one can find many more applications of graph transformation which underline the usefulness from a practical point of view. These are, for example, applications from the area of functional languages (Sleep, Plasmeijer & van Eekelen, 1993), visual languages (Bardohl, Minas, Schürr & Taentzer, 1999), software engineering (Nagl, 1996), and UML (e.g., Bottoni, Koch, Parisi-Presicce & Taentzer, 2000; Engels, Hausmann, Heckel & Sauer, 2000; Fischer, Niere, Torunski & Zündorf, 2000; Petriu & Sun, 2000; Engels, Heckel & Küster, 2001; Kuske, 2001; Kuske, Gogolla, Kollmann & Kreowski, 2002).

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

32

Klempien-Hinrichs, Kreowski, and Kuske

Graph transformation comprises devices for the rule-based manipulation of graphs. Given a set of graph transformation rules and a set of graphs, one gets a graph transformation system in its simplest form. Such a system transforms a start graph by applying its graph transformation rules. The semantics can be defined as a binary relation on graphs where the first component of every pair is a start graph G and the second component is a graph derived from G by applying a sequence of graph transformation rules. In general, the application of a graph transformation rule to a graph transforms it locally, i.e., it replaces a part of the graph with another graph part. Often one wishes to start a derivation only from certain initial graphs, and accepts as results only those derived graphs that are terminal. Moreover, in some cases the derivation process is regulated in a certain way to cut down the nondeterminism of rule applications. For example, one may employ a parallel mode of transformation as in L systems, or one may restrict the order in which rules are applied. Altogether the basic elements of a graph transformation approach are graphs, rules, their application, graph class expressions, and control conditions.

Graphs First of all, there is a class of graphs G, that may be directed or undirected, typed or untyped, labelled or unlabeled, simple or multiple. Examples for graph classes are labeled directed graphs, hypergraphs, trees, forests, finite automata, Petri nets, etc. The choice of graphs depends on the kind of applications one has in mind and is a matter of taste. In this chapter, we consider directed, edge-labeled graphs with individual, multiple edges. A graph is a construct G = (V, E, s, t, l) where V is a set of vertices, E is a set of edges, s, t: E → V are two mappings assigning each edge e ∈ E a source s(e) and a target t(e), and l: E → C is a mapping labeling each edge in a given label alphabet C. A graph may be represented in a graphical way with circles as nodes and arrows as edges that connect source and target, with the arrowhead pointing to the target. The labels are placed next to the arrows. In the case of a loop, i.e., an edge with the same node as source and target, we may draw a flag that is posted on its node with the label inside the box. To cover unlabeled graphs as a special case, we assume a particular label * that is invisible in the drawings. This means a graph G is unlabeled if l(e) = * for all e ∈ E. For instance the graph in Figure 2 consists of six nodes, one of them with a beginflag, another with an end-flag, and a third one with an unlabeled flag. Moreover, it consists of seven directed edges where some of them are labeled with p. The p-edges form a simple path (i.e., a path without cycles) from the begin-flagged node to the end-flagged node. If one takes the subgraph induced by the edges of the simple path and the begin- and end-flag and removes all occurrences of

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Rule-Based Transformation of Graphs and the Product Type 33

Figure 2.

Figure 3.

the label p, one gets the string graph (i.e., a graph that is a simple path from a begin-flagged node to an end-flagged node) that is shown in Figure 3. String graphs can be used to represent natural numbers. The string graph in Figure 3 represents the number four because it has four unlabeled edges between its begin-flagged and its end-flagged node. Whenever a string graph represents a natural number k in this way, we say that it is the k-string graph.

Rules and Rule Applications To be able to transform graphs, rules are applied to graphs yielding graphs. Given some class R of graph transformation rules, each rule r ∈ R defines a binary relation ⇒r ⊆ G×G on graphs. If G ⇒r H, one says that G directly derives H by applying r. There are many possibilities to choose rules and their applications. Rule classes may vary from the more restrictive ones, like edge replacement (Drewes, Kreowski & Habel, 1997) or node replacement (Engelfriet & Rozenberg, 1997), to the more general ones, like double-pushout rules (Corradini et al., 1997), single-pushout rules (Ehrig et al., 1997), or PROGRES rules (Schürr, 1997). In this chapter, we use rules of the form r = (L →K R) where L and R are graphs (the left- and right-hand side of r, respectively) and K is a set of nodes shared by L and R. In a graphical representation of r, L and R are drawn as usual, with numbers uniquely identifying the nodes in K. Its application means to replace an

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

34

Klempien-Hinrichs, Kreowski, and Kuske

occurrence of L with R such that the common part K is kept. In particular, we will use rules that add or delete flags, label edges, and add or delete a node together with an edge. The rule r = (L →K R) can be applied to some graph G directly deriving the graph H if it can be constructed up to isomorphism (i.e., up to renaming of nodes and edges) in the following way. 1.

Find an isomorphic copy of L in G, i.e., a subgraph that coincides with L up to the naming of nodes and edges.

2.

Remove all nodes and edges of this copy except the nodes corresponding to K, provided that the remainder is a graph (which holds if the removal of a node is accompanied by the removal of all its incident edges).

3.

Add R by merging K with its corresponding copy.

For example (Figure 4), the rule move has as its left-hand side a graph consisting of an end-flagged node 1, a node 2 with unlabeled flag, and an unlabeled edge from node 1 to node 2. The right-hand side consists of the same two nodes where node 1 has no flag and node 2 has an end-flag. Moreover, there is a p-labeled edge from node 1 to node 2. The common part of the rule move consists of the nodes 1 and 2. The application of move labels an unlabeled edge with p if the edge connects an end-flagged node and a node with an unlabeled flag, moves the end-flag from the source of the edge to its target, and removes the unlabeled flag. For example, the application of move to the graph above yields the graph shown in Figure 5.

Figure 4.

Figure 5.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Rule-Based Transformation of Graphs and the Product Type 35

Note that this rule cannot be applied to the former graph in any other way; for instance, its left-hand side requires the presence of an unlabeled flag.

Graph Class Expressions The aim of graph class expressions is to restrict the class of graphs to which certain rules may be applied or to filter out a subclass of all the graphs that can be derived by a set of rules. Typically, a graph class expression may be some logic formula describing a graph property like connectivity, or acyclicity, or the occurrence or absence of certain labels. In this sense, every graph class expression e specifies a set SEM(e) of graphs in G. For instance, all refers to all directed, edge-labeled graphs, whereas empty and bool designate a class of exactly one graph each (the empty graph EMPTY for empty, and the graph TRUE consisting of one true-flagged node for bool). Moreover, graph specifies all unlabeled graphs each node of which carries a unique flag (which is unlabeled, too). Also, a particular form of the graphs may be requested; e.g., the expression nat defines all k-string graphs.

Control Conditions A control condition is an expression that determines, for example, the order in which rules may be applied. Semantically, it relates start graphs with graphs that result from an admitted transformation process. In this sense, every control condition c specifies a binary relation SEM(c) on G. As control condition, we use in particular the expression true that allows all transformations (i.e., all pairs of graphs). Moreover, we use regular expressions as control conditions. They describe in which order and how often the rules and imported units are to be applied. In particular, the Kleene star states that an arbitrary number of iterations may be executed. The precise meaning of a regular expression is explained where it is used. More about control conditions can be found in Kuske (2000). All together, a class of graphs, a class of rules, a rule application operator, a class of graph class expressions, and a class of control conditions, form a graph transformation approach based on which graph transformation units as a unifying formalization of graph grammars and graph transformation systems can be defined. To transform graphs, a unit has local rules, but may also import other graph transformation units. Therefore, the semantic relation of a unit is given by the interleaving of rule applications and calls of imported units. Transformation units were presented in Andries et al. (1999) and Kreowski and Kuske (1999b) as a modularization concept for graph transformation systems (cf. also Heckel, Engels, Ehrig & Taentzer, 1999). In the literature there are Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

36

Klempien-Hinrichs, Kreowski, and Kuske

case studies where transformation units are employed to model the semantics of functional programming languages (Andries et al., 1999), UML state machines (Kuske, 2001), and logistic processes (Klempien-Hinrichs, Knirsch & Kuske, 2002).

Transformation Units In general, a graph transformation system may consist of a huge set of rules that by its size alone is difficult to manage. Transformation units provide a means to structure the transformation process. The main structuring principle of transformation units relies on the import of other transformation units or — on the semantic level — on binary relations on graphs. The input and the output of a transformation unit each consists of a class of graphs that is specified by a graph class expression. The input graphs are called initial graphs and the output graphs are called terminal graphs. A transformation unit transforms initial graphs to terminal graphs by applying graph transformation rules and imported transformation units in a successive and sequential way. Since rule application is nondeterministic in general, a transformation unit contains a control condition that may regulate the graph transformation process. A graph transformation unit is a system tu = (I, U, R, C, T) where I and T are graph class expressions, U is a (possibly empty) set of imported graph transformation units, R is a set of rules, and C is a control condition. To simplify technicalities, we assume that the import structure is acyclic (for a study of cyclic imports see Kreowski, Kuske and Schürr (1997)). Initially, one builds units of level 0 with empty import. Then units of level 1 are those that import only units of level 0, and units of level n+1 import only units of level 0 to level n, but at least one from level n. In graphical representations of transformation units we omit the import component if it is empty, the initial or terminal component if it is set to all, and the control condition if it is equal to true. In the following, we present some examples of transformation units. We start with very simple specifications of natural numbers and truth values because they are auxiliary data types to be used later to model the more interesting examples of simple paths, long simple paths, and Hamiltonian paths. The first transformation unit nat0 (Figure 6, left side) constructs all string graphs that represent natural numbers by starting from its initial graph, which represents 0, and transforming the n-string graph into the n+1-string graph by applying the rule succ.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Rule-Based Transformation of Graphs and the Product Type 37

Figure 6.

The second transformation unit nat1 (Figure 6, right side) is a variant of nat0, but now with all n-string graphs as initial graphs. Consequently, it describes arbitrary additions to arbitrary n-string graphs by sequentially increasing the represented numbers by 1. The third transformation unit nat2 (Figure 7) also transforms string graphs into string graphs. It has two rules pred and is-zero. The application of the rule pred to the n-string graph (with n ≥ 1 since otherwise the rule cannot be applied) converts it into the n–1-string graph. The second rule is-zero can be applied only to the 0-string graph but does not transform it, which means that this rule can be used as a test for 0. Moreover, the transformation unit nat2 imports nat1 so that arbitrary additions can be performed, too. The rules of nat2 and the imported unit nat1 can be applied in arbitrary order and arbitrarily often. Hence nat2 converts n-string graphs into m-string graphs for natural numbers m, n. Therefore nat2 can be considered as a data type representing natural numbers with a simple set of operations. Figure 7. nat2 initial: nat uses: nat1 rules: end

end pred:

1

1 begin is-zero:

end 1

* begin

end 1

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

38

Klempien-Hinrichs, Kreowski, and Kuske

Figure 8. bool0 initial:empty

rules:

true

set-true:

empty

true terminal:

The forth transformation unit, bool0 = (empty, ∅, set-true, true, bool), is shown in Figure 8. It has a single initial graph, the empty graph EMPTY. It does not import other transformation units and it has one rule set-true which turns EMPTY to the graph TRUE. The control condition allows all transformations, meaning that TRUE may be added arbitrarily often to EMPTY. However, the terminal graph class expression specifies the set consisting of TRUE, which ensures that the rule set-true is applied exactly once to the initial graph. One can consider bool0 as a unit that describes the type Boolean in its most simple form. At first sight, this may look a bit strange. But it is quite useful if one wants to specify predicates on graphs by nondeterministic graph transformation: If one succeeds to transform an input graph into the graph TRUE, the predicate holds, otherwise it fails. In other words, if the predicate does not hold for the input graph, none of its transformations yield TRUE. The transformation unit simple-path given in Figure 9 constitutes an example of another kind. As an initial graph, it admits all unlabeled graphs with exactly one flag on every node. It chooses an arbitrary simple path in an initial graph by labeling the edges of the path with p and adding a begin-flag and an end-flag to the beginning and the end of the path, respectively. This is done with the help of two rules start and move. The rule start turns an unlabeled flag of an arbitrary node into two flags, respectively labeled with begin and end, and the rule move is the same as above, i.e., it labels with p an edge from an end-flagged node to a node with an unlabeled flag, moves the end-flag to the other node, and removes the unlabeled flag. The control condition is a regular expression which is satisfied if first the rule start is applied, followed by move applied arbitrarily often. The terminal graph class expression admits all graphs, which is why it is not explicitly shown. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Rule-Based Transformation of Graphs and the Product Type 39

Figure 9. simple-path initial: graph rules: begin

end

start: 1

1 end

end p

move: 1

2

1

2

cond: ; move * cond:start start ; *move

Interleaving Semantics of Transformation Units Transformation units transform initial graphs to terminal graphs by applying graph transformation rules and imported transformation units so that the control condition is satisfied. Hence, the semantics of a transformation unit can be defined as a binary relation between initial and terminal graphs. For example, the interleaving semantics of the transformation unit simple-path consists of all pairs (G,G’) such that G is an unlabeled graph with exactly one flag on every node and G’ is obtained from G by labeling the edges of a simple path with p, setting a begin-flag at the source of the path and an end-flag at the target of the path, and removing the flags from the intermediate nodes on the path. In general, for a transformation unit tu without import, the semantics of tu consists of all pairs (G,G’) of graphs such that: 1.

G is an initial graph and G’ is a terminal graph;

2.

G’ is obtained from G via a sequence of rule applications, i.e., (G,G’) is in the reflexive and transitive closure of the binary relation obtained from the union of all relations ⇒r where r is some rule of tu; and

3.

the pair (G,G’) is allowed by the control condition.

If the transformation unit tu has a non-empty import, the interleaving semantics of tu consists of all pairs (G,G’) of graphs which satisfy the preceding items 1 and 3, and where, in addition to rules, imported transformation units can be applied in the transformation process of tu, i.e., the second item above is extended to: Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

40

2’.

Klempien-Hinrichs, Kreowski, and Kuske

G’ is obtained from G via a sequence of rule applications and applications of imported units. This means that (G,G’) is in the reflexive and transitive closure of the binary relation obtained from the union of all relations ⇒r and SEM(u) where r is some rule of tu and u is some imported transformation unit of tu.

More formally, the interleaving semantics of tu is defined as follows: Let tu = (I, U, R, C, T) be a transformation unit. Then the interleaving semantics SEM(tu) is recursively defined as: SEM(tu) = SEM(I) × SEM(T) ∩ ( ∪r ∈ R ⇒r ∪ ∪u ∈ U SEM(u))* ∩ SEM(C). If the transformation unit tu is of level 0, the semantic relation is well-defined because the union over U is the empty set. If tu is of level n+1, we can inductively assume that SEM(u) of each imported unit u is already well-defined, so that SEM(tu) is also well-defined as a union and intersection of defined relations.

Product Type As the iterated application of rules transforms graphs into graphs yielding an input-output relation, the natural type declaration of a graph transformation unit tu = (I, U, R, C, T) is tu: I→T where moreover the initial and terminal graphs are subtypes of the type of graphs that are transformed by the unit. But in many applications one would like to have a typing that allows one to consider several inputs and maybe even several outputs, or at least an output of a type different from all inputs. For instance, a test whether a given graph has a simple path of a certain length would be suitably declared by long-simple-path: graph × nat → bool (or something like this) asking for a graph and a non-negative integer as inputs and a truth value as output. Such an extra flexibility in the typing of graph transformations can be provided by products of graph transformation units, together with some concepts based on the products. In more detail, we introduce the following new features: 1.

The product of graph transformation units providing tuples of graphs to be processed and particularly tuples of initial and terminal graphs as well as tuples of rules and calls of imported units, called action tuples, that can be executed on graph tuples in parallel.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Rule-Based Transformation of Graphs and the Product Type 41

2.

The embedding and projection of a product into resp. onto another product that allow one to choose some components of a product as inputs or outputs and to copy some components into others.

3.

The semantics of a product of graph transformation units is the product of the component semantics such that — intuitively seen — all components run independently from each other. If one wants to impose some iteration and interrelation between the components, one can use control conditions for action tuples like for rules and imported units.

The product type generalizes the notion of pair grammars and triple grammars as introduced by Pratt (1971) and Schürr (1994), respectively.

Product of Graph Transformation Units Let tu 1, …, tu m for m ≥ 1 be a sequence of graph transformation units with tu j = (Ij , Uj , Rj , Cj , Tj) for j = 1, ..., m. Then the product:

prod = tu1 × … × tum =

m

∏ tu i =1

i

transforms m-tuples of graphs (G1, …, Gm) by means of component-wise transformation. The global semantic relation of the product is just the product of the semantic relations of the components, i.e., ((G 1, …, Gm), (H1, …, Hm)) ∈ SEM(prod) if and only if (G i,Hi) ∈ SEM(tui) for i = 1, ..., m. But there is also a notion of a single computation step that transforms graph tuples by applying action tuples. An action tuple (a1, …, am) consists of rules, imported units and an extra void action, i.e., ai ∈ Ri or ai ∈ Ui or ai = – for i = 1, ..., m. It transforms a graph tuple (G 1, …, Gm) into a graph tuple (G 1’, …, Gm’) if Gi →ai Gi’ for ai ∈ Ri and (G i,Gi’) ∈ SEM(ai) for ai ∈ Ui and Gi = Gi’ for ai = – . In other words, a computation step simultaneously applies rules to some components and performs calls of import units in other components while the remaining components of the graph tuple are kept unchanged. Let a single computation step be denoted by (G 1, …, G m) → (G1’, …, Gm’), and let →∗ be the reflexive and transitive closure of →. Then one can say that (G1, …, Gm) →∗ (H1, …, Hm) satisfies the control condition tuple (C1, …, Cm) if (G i,Hi) ∈ SEM(Ci) for i = 1, …, m. Similarly, (G 1, …, G m) is an initial graph Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

42

Klempien-Hinrichs, Kreowski, and Kuske

tuple if Gi ∈ SEM(Ii), and (H1, …, Hm) is a terminal graph tuple if H i ∈ SEM(T i), for i = 1, …, m. If all this holds, the pair of tuples belongs to the step semantics of the product, which is denoted by STEPSEM(prod). It is easy to see that the global semantics and the step semantics coincide, i.e., SEM(prod) = STEPSEM(prod). For example, consider the product simple-path × nat2 of the transformation units simple-path and nat2. Its semantics consists of all pairs ((G1,G 2), (H1,H2)) where (G 1,G2) is in the semantics of simple-path and (H 1,H2) is in the semantics of nat2. This product combines two units in a free way like the well-known Cartesian product. In order to model interrelation between the components, e.g., to test if a path in a graph is of a certain length, we would like to have control conditions for the computation steps and a boolean value as output. This can be achieved with the concepts introduced in the following two sections.

Embedding and Projection If not all initial graph class expressions of a product are meant as inputs, but some of them are just of an auxiliary nature for intermediate computations or to be used as outputs, one may choose the input types and embed their product into the actual product that provides the graph tuples to be transformed. This is possible whenever the auxiliary components have unique initial graphs and if every chosen input type is a subtype of the corresponding initial graphs. Let prod = tu 1 × … × tum be a product of transformation units and let X be a set of graph class expressions that is associated with the product components by an injective mapping ass: X→{1,…,m} such that SEM(x) ⊆ SEM(Iass(x)) for all x ∈ X. Assume, moreover, for all j ∈ {1,…,m}\ ass(X) that either SEM(Ij) = {Gj} for some graph Gj or SEM(x) ⊆ SEM(Ij) for some chosen x ∈ X, which will be denoted by copy: x→j. Then we get an embedding of the product of the graphs in SEM(x) for x ∈ X into the product of initial graphs of the product prod:

embed:

∏ x∈X

SEM ( x) →

m

∏ SEM ( I ) j

j =1

defined by embed((Gx)x∈X) = (G1, …, Gm) with Gi = G x for ass(x) = i and copy: x → i, and G i ∈ SEM(Ii) = {Gi} otherwise. This means that each input component is embedded into its corresponding component of the product of units with respect to ass and into all other

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Rule-Based Transformation of Graphs and the Product Type 43

components given by the copy relation. All remaining components of the product of units are completed by the single initial graphs of these components. As a simple example, let prod = simple-path × nat2 × bool0 and let X = {graph,nat}. Consider the initial graph class expressions graph, nat and empty of the transformation units simple-path, nat2, and bool0, respectively. Every pair (G1,G 2) ∈ SEM(graph) × SEM(nat) can be embedded into SEM(graph) × SEM(nat) × SEM(empty) by choosing ass(graph) = 1 and ass(nat) = 2, i.e., we get embed((G 1,G2))=(G 1,G 2,EMPTY) for every pair (G1,G2)∈SEM(graph) × SEM(nat). Conversely, if one wants to get rid of some component graphs, the well-known projection may be employed. The same mechanism can be used to multiply components, which allows one, in particular, to copy a component graph into another component. Let Y be a set which is associated with the product prod by ass: Y → {1, …, m}. Then one can consider the product of the terminal graphs in SEM(Tass(y)) for all y ∈ Y as the semantics of the association ass, i.e.:

SEM(ass) =

∏ SEM (T

ass ( y )

).

y∈Y

The product of terminal graphs of the product prod can be projected to SEM(ass), i.e.:

m

proj:

∏ SEM (T ) → SEM (ass ) i

i =1

defined by proj(H 1, …, Hm) = (Hass(y))y ∈ Y. For example, consider the terminal graph class expressions all and bool of the transformation units simple-path, nat2, and bool0. Let Y = {3} and let ass(3) = 3. The semantics of ass is equal to the terminal graph TRUE of bool0 and every triple (H1, H 2, H3) ∈ SEM(all) × SEM(all) × SEM(bool) is projected to H 3, i.e., to TRUE. In general, there are two cases of interest. Firstly, if Y ⊆ {1, …, m} and ass is the corresponding inclusion, then proj is the ordinary projection of a product to some of its components. (This is the case in the described example.) Secondly, if several elements of Y are mapped to the same index i, this results in the multiplication of the i-th component. Embedding and projection may be used to realize transformations on graphs with type declarations of the form trans: in 1 × … × in k → out 1 × … × outl where the Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

44

Klempien-Hinrichs, Kreowski, and Kuske

in i and the outj are graph class expressions. The intention is that trans relates the product of inputs SEM(in 1) × … × SEM(in k) with the product of outputs SEM(out1) × … × SEM(outl). This is obtained by using a product prod of graph transformation units tu1,…,tu k+l such that SEM(ini) ⊆ SEM(Ii) for i = 1,…,k and SEM(Tj) ⊆ SEM(outj) for j = k+1,…,k+l. The first k inclusions allow one to embed the inputs into the initial graph tuples of the product prod if, for j = k+1,…,k+l, we can choose some i with copy: i → j or SEM(Ij) = {Gj} for some graph G j. The last l inclusions allow one to project the terminal graph tuples of prod onto outputs. Therefore, the semantic relation of trans has the proper form, but the output tuples are totally independent of the input tuples due to the product semantics. To overcome this problem, we generalize the notion of control conditions in such a way that it applies not only to the control of rule applications and calls of imported units, but also to action tuples.

Control Conditions for Action Tuples A control condition regulates the use of rules and imported units formally by intersecting the interleaving semantics with the semantic relation given by the control condition. This is easily generalized to action tuples if one replaces the interleaving semantics by the step semantics of the product of graph transformation units. In concrete cases, the control condition may refer to action tuples, just as it can refer to rules and imported units. To make this more convenient, action tuples may get identifiers. As an example how the features based on the product may be used, we specify the test long-simple-path that transforms graphs and non-negative integers as inputs into truth values as output. long-simple-path: graph × nat → bool prod: simple-path × nat2 × bool0 actions: a0 = (start, –, – ) a1 = (move, pred, – ) a2 = (–, is-zero, set-true) cond: a 0 ; a1* ; a2 It is modeled on top of the product of the units simple-path, nat2 and bool0. The typing is appropriate as graph and nat specify the initial graphs of simple-path and nat2 respectively, and bool refers to the terminal graph of bool0.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Rule-Based Transformation of Graphs and the Product Type 45

Hence, a computation in long-simple-path starts with an unlabeled graph and a non-negative integer completed to a triple by the initial graph of bool0. Then the control condition requires to perform a0 that chooses a start node in the graph without changing the other two components. This is followed by the iteration of a1 which in each step synchronously prolongs a simple path in the first component by one edge and decreases the integer in the second component by 1. Hence we get a graph with a path of the input length if the second component becomes zero. This is tested by the second component of a2. In the positive case, a2 is performed yielding TRUE as output in the third component. In other words, long-simplepath computes TRUE if and only if the input graph G has a simple path of the input length n.

Transformation of Graph Transformations Two graph transformations may be related with each other in various significant ways: 1.

They may be semantically equivalent, meaning that their semantic relations coincide or, seen from another perspective, that a semantic relation is modeled in two different ways.

2.

One graph transformation may be the refinement of the other one, meaning that each computational step of the one can be accomplished by an interleaving sequence of the other.

3.

One graph transformation may be reduced to the other, meaning that the semantic relation of the one can be translated into the semantic relation of the other.

Such situations are nicely modeled by transformations of graph transformations. In the case of two graph transformation units tu i = (Ii, U i, Ri, Ci, Ti) with SEM(tui) ⊆ SEM(Ii) × SEM(Ti) for i = 1,2, a transformation of the translational type (or a translation for short) of tu1 into tu 2 is defined by two graph transformation units tu1-2-tu 2(I) = (I1, UI, RI, C I, I2) and tu 1-2-tu2(T) = (T1, U T, RT, C T, T2) where the former transforms initial graphs of tu 1 into initial graphs of tu2 and the latter does the same with respect to terminal graphs. How a translation relates graph transformations units is depicted in Figure 10. Clearly, such a translation is only meaningful if it preserves the semantics, which is covered by the notion of correctness. A translation of tu1 into tu2 is correct if the diagram in Figure 10 commutes, i.e., if the sequential compositions of the

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

46

Klempien-Hinrichs, Kreowski, and Kuske

Figure 10.

Translation of graph transformation units

semantic relations of SEM(tu1) with SEM(tu1-2-tu 2(T)) on one hand and of SEM(tu1-2-tu2(I)) with SEM(tu2) on the other hand coincide. Correct translations can be very helpful because they carry over certain properties from the source unit to the target unit and the other way round. For example, if some question is undecidable for the source unit, the corresponding question must be undecidable for the target unit, provided that the translating units have computable semantic relations (which holds in all reasonable cases). To demonstrate the usefulness of translations more explicitly, we restrict the notion of translations to the notion of reductions as used in the study of the complexity class NP of all decision problems that can be computed nondeterministically in polynomial time. A reduction of tu1 to tu2 is a correct translation of tu1 into tu2 subject to the following further conditions: 1.

tu1 and tu2 model predicates, i.e., their output domain is bool,

2.

tu1-2-tu 2(T) is the identity on bool, and

3.

tu1-2-tu 2(I) has no import and runs in polynomial time, i.e., each derivation starting in an initial graph of tu1 has a length polynomial in the size of its start graph and can be prolonged such that it derives an initial graph of tu 2.

If tu 2 models an NP-problem, i.e., it has no import and each derivation starting in an initial graph has a length that is polynomial in the size of the start graph, then the composition of the reduction and the semantic relation of tu2 is in NP, too. While the reduction yields an output for every input in a polynomial number of steps, the following computation in tu2 runs also in polynomial time, but it is nondeterministic because it may compute TRUE for some of its inputs while other computations for the same input end in deadlocks. Hence, the sequential composition, which is the semantic relation of tu 1 due to the correctness of the translation, is nondeterministic, too, with polynomial runtime. By a similar reasoning, it turns out that tu 2 models an NP-complete problem if tu1 does, i.e.,

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Rule-Based Transformation of Graphs and the Product Type 47

if each NP-problem can be reduced to the semantic relation of tu1. So the graphtransformational variants of reductions may be used to investigate the class NP in the same way as ordinary reductions are useful. But, as many interesting problems in NP are graph problems, graph-transformational reductions may be quite suitable. As an illustrating example, we specify a reduction from the Hamiltonian-path problem HP into the unit long-simple-path. We assume that HP is a predicate with the typing HP: graph → bool that yields TRUE for an input graph G if and only if G has a simple path that visits all nodes. An explicit specification by graph transformation is not needed, but it would look similar to simple-path, only making sure that all nodes are involved. Due to the typing, the reduction must consist of a graph transformation unit of the type HP-2-lsp: graph → graph × nat that copies the input graph as output graph and computes the number of nodes minus one of the input graph as second output. For this purpose, the product of units mark-all-nodes, graph and nat0 will be used. The unit graph = (graph, ∅, ∅, true, graph) takes unlabeled graphs as initial and terminal graphs and is without import and rules such that its semantics is the identity relation on SEM(graph), i.e., the input graph becomes the output graph. The unit mark-allnodes consists of unlabeled graphs as initial graphs, of one rule mark that replaces the unlabeled flag by another flag (ok-labeled for example), and of graphs without unlabeled flags as terminal graphs. This is an auxiliary unit the meaning of which is that each derivation from an initial to a terminal graph has the number of nodes as length. Hence, an action tuple that applies the rule mark in the first component allows one to count the number of nodes. Summarizing, we get the following specification: HP-2-lsp: graph → graph × nat0 prod: mark-all-nodes × graph × nat0 copy: 1 → 2 actions: b 0 = (mark, –, –) b1 = (mark ,–, succ) cond: b 0 ; b1* Note that the length of all computations is bounded by the number of nodes of the input graph and that each computation can be prolonged until all nodes are marked. As one always marks the first node without increasing the initial integer 0 and as all other nodes are marked while the integer is increased by one in each step, one ends up with the number of nodes minus one as integer output. And the runtime of HP-2-lsp is linear. If one composes the semantic relation of

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

48

Klempien-Hinrichs, Kreowski, and Kuske

HP-2-lsp with that of long-simple-path, it returns TRUE if and only if the original input graph has a simple path of a length that is the number of nodes minus one such that it visits all nodes. In other words, the translation is correct. As the Hamilton-path problem is NP-complete, our reduction shows that long-simplepath is also NP-complete (which is already well known in this case).

Conclusions In this chapter, we have given an introductory survey of graph transformation with graphs, rules, rule application, graph class expressions, and control conditions as basic features. As all of the concepts are handled in a generic, parametric way, this covers nearly all of the graph transformation approaches one encounters in the literature (see, e.g., Rozenberg, 1997, for an overview). Readers who are interested in seeing the full spectrum of applications of graph transformation and its relation to the theory of concurrency are referred to the Handbook of Graph Grammars and Computing by Graph Transformation, Vol. 2 and 3 (Ehrig, Engels, Kreowski & Rozenberg, 1999; Ehrig, Kreowski, Montanari & Rozenberg, 1999). In addition, we have proposed the new concept of product types that allows one to transform a tuple of graphs by the synchronous transformation of the components. This is quite helpful to specify transformations with a flexible typing, i.e., with an arbitrary sequence of input graphs and an arbitrary sequence of output graphs. Moreover, the types of the input and output graphs need not be subtypes of the same type of graphs anymore. As a consequence, the product type is particularly useful if one wants to transform graph transformations into each other. Further investigation of the product type may concern the following aspects: As we used graph-transformational versions of the truth values and the natural numbers in our illustrating examples, one may like to combine graph types with arbitrary abstract data types. In the presented definition, we consider the product of graph transformation units. But one may like to import products in units and to use components that are again products. Whether such a composite use of products works must be investigated. The transformation of graph transformation units is only tentatively sketched. It must be worked out how it helps to study refinement and semantic equivalence and other interesting relationships between graph transformation systems.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Rule-Based Transformation of Graphs and the Product Type 49

Acknowledgments The research presented here was partially supported by the EC Research Training Network SegraVis (Syntactic and Semantic Integration of Visual Modelling Techniques) and the Collaborative Research Centre 637 (Autonomous Cooperating Logistic Processes: A Paradigm Shift and Its Limitations) funded by the German Research Foundation (DFG).

References Andries et al. (1999). Graph transformation for specification and programming. Science of Computer Programming, 34(1), 1-54. Bardohl, R., Minas, M., Schürr, A. & Taentzer, G. (1999). Application of Graph Transformation to Visual Languages. In H. Ehrig, G. Engels, H.-J. Kreowski & G. Rozenberg (Eds.), Handbook of Graph Grammars and Computing by Graph Transformation, Vol. 2: Applications, Languages and Tools (pp. 105-180). Singapore: World Scientific. Bottoni, P., Koch, M., Parisi-Presicce, F., & Taentzer, G. (2000). Consistency Checking and Visualization of OCL Constraints. In A. Evans, S. Kent & B. Selic (Eds.), Proceedings of UML 2000 – The Unified Modeling Language. Advancing the Standard, Lecture Notes in Computer Science (Vol. 1939, pp. 294-308). Springer. Corradini, A., Montanari, U., Rossi, F., Ehrig, H., Heckel, R. & Löwe, M. (1997). Algebraic Approaches to Graph Transformation – Part I : Basic Concepts and Double Pushout Approach. In G. Rozenberg (Ed.), Handbook of Graph Grammars and Computing by Graph Transformation, Vol. 1: Foundations (pp. 163-245). Singapore: World Scientific. Drewes, F., Kreowski, H.-J. & Habel, A. (1997). Hyperedge Replacement Graph Grammars. In G. Rozenberg (Ed.), Handbook of Graph Grammars and Computing by Graph Transformation, Vol. 1: Foundations (pp. 95-162). Singapore: World Scientific. Ehrig et al. (1997). Algebraic Approaches to Graph Transformation – Part II: Single Pushout Approach and Comparison with Double Pushout Approach. In G. Rozenberg (Ed.), Handbook of Graph Grammars and Computing by Graph Transformation, Vol. 1: Foundations (pp. 247-312). Singapore: World Scientific. Ehrig, H., Engels, G., Kreowski, H.-J. & Rozenberg, G. (Eds.) (1999). Handbook of Graph Grammars and Computing by Graph Transformation, Vol. 2: Applications, Languages and Tools. Singapore: World Scientific. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

50

Klempien-Hinrichs, Kreowski, and Kuske

Ehrig, H., Kreowski, H.-J., Montanari, U. & Rozenberg, G. (Eds.) (1999). Handbook of Graph Grammars and Computing by Graph Transformation, Vol. 3: Concurrency, Parallelism, and Distribution. Singapore: World Scientific. Engelfriet, J. & Rozenberg, G. (1997). Node Replacement Graph Grammars. In G. Rozenberg (Ed.), Handbook of Graph Grammars and Computing by Graph Transformation, Vol. 1: Foundations (pp. 1-94). Singapore: World Scientific. Engels, G., Hausmann, J.H., Heckel, R. & Sauer, S. (2000). Dynamic Meta Modeling: A Graphical Approach to the Operational Semantics of Behavioral Diagrams in UML. In A. Evans, S. Kent & B. Selic (Eds.), Proceedings of UML 2000 – The Unified Modeling Language. Advancing the Standard, Lecture Notes in Computer Science (Vol. 1939, pp. 323337). Springer. Engels, G., Heckel, R. & Küster, J.M. (2001). Rule-Based Specification of Behavioral Consistency Based on the UML Meta-model. In M. Gogolla & C. Kobryn (Eds.), UML 2001 – The Unified Modeling Language. Modeling Languages, Concepts, and Tools, Lecture Notes in Computer Science, Vol. 2185 (pp. 272-286). Springer. Fischer, T., Niere, J., Torunski, L. & Zündorf, A. (2000). Story Diagrams: A new Graph Transformation Language based on UML and Java. In H. Ehrig, G. Engels, H.-J. Kreowski & G. Rozenberg (Eds.), Proc. Theory and Application of Graph Transformations, Lecture Notes in Computer Science, Vol. 1764 (pp. 296-309). Springer. Heckel, R., Engels, G., Ehrig, H. & Taentzer, G. (1999). Classification and Comparison of Module Concepts for Graph Transformation Systems. In H. Ehrig, G. Engels, H.-J. Kreowski & G. Rozenberg (Eds.), Handbook of Graph Grammars and Computing by Graph Transformation, Vol. 2: Applications, Languages and Tools (pp. 669-689). Singapore: World Scientific. Klempien-Hinrichs, R., Knirsch, P. & Kuske, S. (2002). Modeling the Pickupand-Delivery Problem with Structured Graph Transformation. In H.-J. Kreowski & P. Knirsch (Eds.), Proc. APPLIGRAPH Workshop on Applied Graph Transformation, Satellite Event of ETAPS 2002 (pp. 119-130). Kreowski, H.-J. & Kuske, S. (1999a). Graph Transformation Units and Modules. In H. Ehrig, G. Engels, H.-J. Kreowski & G. Rozenberg (Eds.), Handbook of Graph Grammars and Computing by Graph Transformation, Vol. 2: Applications, Languages and Tools (pp. 607-638). Singapore: World Scientific. Kreowski, H.-J. & Kuske, S. (1999b). Graph transformation units with interleaving semantics. Formal Aspects of Computing, 11(6), 690-723. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Rule-Based Transformation of Graphs and the Product Type 51

Kreowski, H.-J., Kuske, S. & Schürr, A. (1997). Nested graph transformation units. International Journal on Software Engineering and Knowledge Engineering, 7, 479-502. Kuske, S. (2000). More about control conditions for transformation units. In H. Ehrig, G. Engels, H.-J. Kreowski & G. Rozenberg (Eds.), Proc. Theory and Application of Graph Transformations, Lecture Notes in Computer Science, Vol. 1764 (pp. 323-337). Springer. Kuske, S. (2001). A Formal Semantics of UML State Machines Based on Structured Graph Transformation. In M. Gogolla & C. Kobryn (Eds.), UML 2001 – The Unified Modeling Language. Modeling Languages, Concepts, and Tools, Lecture Notes in Computer Science, Vol. 2185 (pp. 241-256). Springer. Kuske, S., Gogolla, M., Kollmann, R. & Kreowski, H.-J. (2002). An Integrated Semantics for UML Class, Object, and State Diagrams based on Graph Transformation. In M. Butler & K. Sere (Eds.), Third Int. Conference Integrated Formal Methods (IFM’02), Lecture Notes in Computer Science, Vol. 2335 (pp. 11-28). Springer. Nagl, M. (Ed.) (1996). Building Tightly Integrated Software Development Environments: The IPSEN Approach. Lecture Notes in Computer Science, Vol. 1170. Springer. Petriu, D.C. & Sun, Y. (2000). Consistent Behaviour Representation in Activity and Sequence Diagrams. In A. Evans, S. Kent & B. Selic (Eds.), Proc. UML 2000 – The Unified Modeling Language. Advancing the Standard, Lecture Notes in Computer Science, Vol. 1939 (pp. 359-368). Springer. Pratt, T.W. (1971). Pair grammars, graph languages, and string-to-graph translations. Journal of Computer and System Sciences, 5(6), 560-595. Rozenberg, G. (ed.) (1997). Handbook of Graph Grammars and Computing by Graph Transformation, Vol. 1: Foundations. Singapore: World Scientific. Schürr, A. (1994). Specification of Graph Translators with Triple Graph Grammars. In G. Tinhofer (Ed.), Proceedings of WG’94 20th Int. Workshop on Graph-Theoretic Concepts in Computer Science, Lecture Notes in Computer Science, Vol. 903 (pp. 151-163). Springer. Schürr, A. (1997). Programmed Graph Replacement Systems. In G. Rozenberg (Ed.), Handbook of Graph Grammars and Computing by Graph Transformation, Vol. 1: Foundations (pp. 479-546). Singapore: World Scientific. Sleep, R., Plasmeijer R. & van Eekelen, M. (eds.) (1993). Term Graph Rewriting: Theory and Practice. New York: John Wiley.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

52 Petit and Hacid

Chapter III

From Conceptual Database Schemas to Logical Database Tuning Jean-Marc Petit, Université Clermont-Ferrand 2, France Mohand-Saïd Hacid, Université Lyon 1, France

Abstract This chapter revisits conceptual database design and focuses on the socalled “logical database tuning”. We first recall fundamental differences between constructor-oriented models (like extended Entity-Relationship models) and attribute-oriented models (like the relational model). Then, we introduce an integrated algorithm for translating ER-like conceptual database schemas to relational database schemas. To consider the tuning of such logical databases, we highlight two extreme cases: null-free databases and efficient — though non redundant — databases. Finally, we point out how SQL workloads could be used a posteriori as a help for the designers and/or the database administrators to reach a compromise Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

From Conceptual Database Schemas to Logical Database Tuning

53

between these extreme cases. While a lot of papers and books have been devoted for many years to database design, we hope that this chapter will clarify the understanding of database designers when implementing their databases and database administrators when maintaining their databases.

Introduction Semantic data modeling is the activity of specifying the structure and the semantics of the data to be managed within an application. Since the 1970s, semantic data modeling has been the subject of a large body of work in several areas, including databases, information systems, software engineering and knowledge representation. For database design, approaches to data modeling advocate the use of abstract formalisms, such as the popular Entity Relationship model (Chen, 1976), for describing data, mostly based on the notion of class or entities. Two main families of semantic data models are addressed in the literature:

•

Attribute-oriented models: Data structure is captured through the notion of attributes, i.e., objects and relationships between objects are modeled thanks to attributes. Most of data semantics is expressed by means of additional constraints. The relational data model or object-oriented data models fall into this family.

•

Constructor-oriented models: Data semantics is captured through various constructors, including attributes but also a constructor for objects and another one for relationships between objects. A key feature of such models is that most of data semantics is already expressed by the constructors. Entity-Relationship models (Chen, 1976) fall into this family.

Conceptual data models like ER models provide a high-level abstraction for information concerning an application domain. To do so, they rely on many constructors, such as relationship-type which is very powerful to capture data semantics at a high level abstraction. Moreover, such models are widely used in practice, as advocated by methods of information systems design like MERISE (Moulin, 1976; Tardieu, 1979) or R.U.P. (Rational Unified Process) based on UML (Jacobson, 1999). Most of them have conceptual data models derived from ER models except that the interpretation of relationship cardinalities substantially differs. As an example, a database schema created with one interpretation of relationship cardinalities is quite different from the same database schema created with the other interpretation. This point is often left to

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

54 Petit and Hacid

the reader of many textbooks devoted to databases in general, and database design in particular. However, these differences are often misunderstood by database designers and may lead to practical errors at the implementation stage. Their use requires a thorough knowledge of the theory behind them, though such knowledge is not required of the average designer. For tasks like database design, some mappings are necessary to translate conceptual schemas into physical schemas, i.e., schemas which can be processed by some database management system (DBMS). Mapping conceptual data models — like ER models (Chen, 1976) — to logical data models — like relational data model — has been widely investigated (e.g., Theorey, 1986; Markowitz, 1992; Mannila, 1994; Fahrner, 1995; Rosenthal, 1994). Despite the maturity of this field, we believe these approaches have a common weakness, for example the lack of an integrated algorithm to perform the translation process whatever the convention used to represent the conceptual data model, mainly around the interpretation of cardinality constraints on relationship-types. However, practitioners may use automated tools to carry out the mappings. Nevertheless, choices made by the tools could be far from the objectives of the designer and do not provide any insight into the key issues of such mappings. Furthermore, approaches are either too formal to be used by practitioners (Markowitz, 1992) or very informal so that a great deal of work is left to the practitioners (Rosenthal, 1994; Fahrner, 1995). In this chapter, we try to fill in the gap between these two approaches. We make a step toward a unified approach of translating conceptual data schemas to relational logical data schemas in which different kinds of interpretations can be dealt with without any overhead. Our work complements those done by Markowitz (1992) and Mannila (1994). Due to space limitation, we do not attempt to define the whole process. Rather, we have tried to reach a compromise between a formal treatment, e.g., algorithms are given, and a clear understanding of the issues of such a mapping, e.g., through a running example. We also study some classical choices database designers are faced with when they want to implement their databases from a logical database schema. This stage is a matter of compromises to be reached between getting a null-free database and getting an efficient database. To end up, we argue that logical database tuning could also be defined a posteriori with respect to SQL workloads representative of the database activity during some period of time. It can be thought of as a help to guide the designer when implementing (if she/he can predict some SQL workloads) or maintaining her/his database. The chapter is organized as follows: Notations and background materials are given first. Then the chapter gives a mapping from conceptual data models to

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

From Conceptual Database Schemas to Logical Database Tuning

55

logical data models. Next, the chapter discusses strategies for logical database tuning. The chapter then concludes.

Background This section is intended to bring to the fore the abstracting power of constructororiented data models with respect to attribute-oriented data models as done previously by others (e.g., Hull, 1987; Navathe, 1992). Numerous investigations have been conducted in the data modeling area in order to seek more appropriate formalisms for accurately representing real-world applications. These investigations have resulted in a class of data models called semantic data models (Peckham, 1988). Semantic data models provide constructors for explicitly representing the semantics of the application. The constructs implement information modeling tools called data abstractions. These abstractions enable a complex world to be examined in terms of a simplified world that incorporates the most significant points. Most importantly, data abstractions provide the basis for a step-wise design methodology for databases. Each data model has its own structuring mechanism from which to build application schemas. In semantic data models this mechanism is in terms of semantic structures expressed in some textual language or in graph-theoretic terms. In such data models, most of the data semantics has to be captured by so-called constructors, which are high-level abstraction mechanisms. Two main families of semantic data models were addressed:

•

With attribute-oriented data models, the main constructor is related to the notion of attributes. Basically, an attribute associates a meaningful name in the application context with a type, the permitted values. They are used to describe the characteristics (or properties) of objects of the real-world, but also to describe the relationships between objects.

•

With constructor-oriented data models, many constructors are available to produce a conceptual schema. Among the main constructors, we find again the notion of attributes, but also an explicit constructor intended to capture the relationships between objects.

The analysis of the above cited models makes it clear that, although they address the same issues, attribute-oriented models seem to be less expressive, or at least less simple, than the constructor-oriented models. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

56 Petit and Hacid

Therefore, when schemas based on constructor-oriented models have to be translated into schemas based on attribute-oriented models, these later rely on explicit constraints to accurately capture data semantics conveyed by the former. For instance, the relational model requires constraints like functional dependencies and inclusion dependencies to equivalently represent a relationship-type of an Entity-Relationship model. So, implicit constraints of constructor-oriented models should be encoded, if any, by explicit constraints of attribute-oriented models. Now, we introduce an instance of attribute-oriented models (namely the relational model) and an instance of constructor-oriented models based on an extension of the Entity-Relationship model (Chen, 1976).

Relational Data Model Simplified notations of relational databases used in this chapter are given in this section (for more details, see, e.g., Mannila, 1994; Levene, 1999; Ramakrishnan, 2003). An attribute A is associated with a domain, the set of its possible values. A relation schema R is associated with a set of attributes, noted schema(R). A relation r is defined over a relation schema R and corresponds to a set of tuples, an element of the cartesian product of attribute domains of R. A database schema R is a set of relation schemas. A relational database d (or database) over a database schema R corresponds to a set of relations, a relation of d being in one-to-one correspondence with a relation schema of R. In this chapter, we are interested in two main types of constraints of the relational model: functional dependencies and inclusion dependencies. The former allows us to specify constraints within a relation schema and the latter allows us to specify constraints between two relation schemas (though possible within one relation schema). The set of functional dependencies associated with a database schema is denoted by F, and the set of inclusion dependencies associated with a database schema is denoted by I. In the sequel, a relational database schema will be denoted by a triple (R, F, I). A set of attributes X of a relation schema R is a (minimal) key of R with respect to F if (1) F logically implies Xàschema(R) and (2) no subset of X satisfies this property. Let X and Y be two sets. X+Y (respectively, X-Y) stands for X∪Y (respectively, X\Y) and we omit the brackets for sets reduced to singletons, i.e., the set {A} is denoted by A.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

From Conceptual Database Schemas to Logical Database Tuning

57

A Conceptual Data Model An extension of the Entity-Relationship is quickly given in this section. For more details, see e.g., Chen (1976) and Ramakrishnan (2003). Recall that our objective is to be able to deal with different interpretations of relationship cardinalities. However, in order to be able to run an example throughout the chapter, we have to select an interpretation. For that, we have chosen UML-like interpretation since it seems to be widely accepted at present as a standard. The model used in this chapter has the following basic and extended modeling constructs1:

•

Attributes: they are single-valued and are defined with respect to a type, omitted when clear from the context.

•

Entity-type: it represents a set of entities (or objects) sharing common properties, described by means of attributes. Keys can be specified, i.e., they are subsets of attributes identifying uniquely every entity.

•

Relationship-type: it allows to associate together other entity-types. We also allow relationship-types to participate in other relationship-types, sometimes called generalized aggregation. In that case, an arrow is set to know “who associates who”. In the sequel, an object-type will denote either an entity-type or a relationship-type. Restrictions may be set to specify the number of relationships that an object (or a group of objects) can participate in. Those restrictions are known as cardinality constraints.

•

Generalization/specialization relationships: They define inheritance hierarchy (or is-a-hierarchy) over entity-types. We will say that A is-a B, A being an inherited entity-type. Note that these relationships are a special kind of binary relationship-types with a one-to-one constraint.

•

We also allow is-a-hierarchy for relationship-types, which does not share any element with is-a-hierarchy of entity-types. In the sequel, we shall use inherited object-types to denote either inherited entity-types or inherited relationship-types.

•

Identifying relationships: They allow to define weak entity-types, i.e., entity-types whose key is made up of both a local key and the key inherited from the identifying relationship. Such relationships may form a hierarchy, called id-hierarchy. We will say that A id B, A being a weak-entity type. Note that this is a special kind of binary relationship-types with a one-tomany constraint.

As a consequence of our notation, an entity-type can be weak, inherited or just an entity-type. In the sequel, we denote by strong entity-types an entity-type

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

58 Petit and Hacid

which is neither weak nor inherited. We require the existence of a (minimal) key for each strong entity-type (It should be noted that we use this requirement here to simplify the rest of this chapter. In practice, if such a key does not exist, a surrogate key can be defined at the logical level.)

Different Interpretations of Cardinalities The difference between UML- and Merise-like relationships can be explained as follows: Let A be a relationship-type defined on E1, …, En and let us consider that the domain of each Ei is a set of objects. Thus, A ⊆ E 1 x … x E n. Let mi be the cardinality of the role between A and E i . Let us consider two main cases: m i =1 and m i = N.

•

mi = 1

• UML-like cardinality: It means that the functional dependency E1…Ei-1 Ei+1 … En à Ei must hold in every instance of A.

• Merise-like cardinality: It means that Ei à E1 … Ei-1 Ei+1…En must hold in every instance of A.

•

mi = N: Neither UML- nor Merise-like interpretations can be used to define constraints on E1 x … x En.

Graphical notations used to express a schema are not detailed here — they are just given in our running example. Example: For the sake of comparison, we borrow our running example from a widely disseminated textbook with only minor changes (Ramakrishnan, 2003). The application deals with the management of projects by employees in a company organized in several departments. Figure 1 shows the diagram for the application example. More details can be found in Ramakrishnan (2003). In this diagram, we have three strong entity-types (People, Projects, Departments), one inherited entity-type (Employees), one weak entity-type (Dependents) and two relationship-types (monitors, sponsors). Keys are underlined in the diagram. The one-to-many relationship-type sponsors between Projects and Departments has to be read as follows: A department sponsors one project only, whereas a project could be sponsored by several departments. Note that the relationship-type monitors associates an entity-type (Employees) with a relationship-type (sponsors). Dependents is a weak entity-type: its identification is defined by its local key pname and by the key of Employees, which is itself derivable from the key associated with People, i.e., ssn.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

From Conceptual Database Schemas to Logical Database Tuning

59

Figure 1. A conceptual schema expressed with our extended ER model

Formal approaches exist for describing and reasoning on an extended EntityRelationship model. The interested reader may refer to Markowitz (1992), Hacid (2001) and Franconi (2000).

Well Formed Conceptual Data Schemas The intuitive meaning of diagrams used to represent conceptual data schemas is both a strength and a weakness of such a kind of modeling. Indeed, it should not hide the underlying semantics of each constructor, the risk being to produce schemas which could lead to inconsistencies at the implementation stage. We identify two main restrictions on conceptual schemas:

•

The hierarchy formed by the merging of is-a and id hierarchies is acyclic. This condition requires that each hierarchy (is-a-hierarchy on entity-types, is-a-hierarchy on relationship-types and id-hierarchy) has to be acyclic, and once merged, the result is acyclic.

•

At most one leaving id relationship is allowed from any weak entitytype. Indeed, many problems may arise when their identity can be derived by at least two different paths with two different keys.

Note that our conditions differ from those given in Mannila (1994), where for example a weak entity-type may be involved in different paths in the id-

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

60 Petit and Hacid

hierarchy. Some contributions (e.g., Mannila, 1994; Levene, 1999) consider also inherited entity-types as weak entity-types.

From Conceptual Data Schemas to Logical Data Schemas We rigorously define a mapping which produces highly normalized schemas (stronger than the Boyce-Codd normal form (BCNF)), without null values but maximizing the length of join paths. Normal forms have been defined to capture such kind of database schemas, such as entity-relationship normal form (ERNF) or inclusion dependency normal form (IDNF). Due to space limitation, we will not develop such issues and we refer to Markowitz (1992), Mannila (1994) and Levene (1999) for more details.

A Classical Framework From a formal point of view, the translation process involves four steps: 1.

Verifying whether the conceptual schema is well formed.

2.

Translating the conceptual schema into an oriented graph: doing so, some classical misconceptions can be rectified and thus a formal treatment can be carried out.

3.

Dealing with attribute naming problems: indeed, assigning names to the attributes in a relational database schema turns out to be a challenging task.

4.

Translating the conceptual schema to a logical database schema: often, the whole process is reduced to this step according to a set of rules.

The first step is certainly the most easy to perform. Note that some tools supporting both syntactic and semantic checking of a given conceptual schema exist (e.g., Franconi, 2000). The second step is described in the next section and could be coupled with the first step: if the graph is acyclic, then the first condition of a well-formed conceptual schema is reached. The third step is trickier to achieve. In fact, the intuition is that at the conceptual level, attribute names are local, i.e., their scope is delimited by the entity or the relationship-type in which they are defined. This is not true anymore with the relational model at least from a theoretical point of view: an attribute name has a global scope within a relational database schema and its meaning is related to

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

From Conceptual Database Schemas to Logical Database Tuning

61

the underlying assumption, such that for instance URSA for Universal Relation Schema Assumption (many others do exist). In this setting, this apparently simple problem turns out to be technically difficult. In this chapter, we avoid such problems by considering that the scope of an attribute is local, i.e., valid only within its relation schema. More details on this issue can be found in Rosenthal (1994) and Mannila (1994). In the following, we will focus on steps two and four only since we believe they are crucial for such a mapping.

Using a Graph-Based Representation From a conceptual schema S, we produce a directed graph G = (V, E) as follows:

• •

Each entity-type or relationship-type of S becomes a node of V. Let A, B be nodes of V. An edge is created in E from A to B, noted (A→B), whenever:

• there exists A is-a B in S, • there exists A id B in S, • A is a relationship-type and B participates in A. Formally speaking, we obtain a multi-graph since more than one edge can be defined between two nodes. Nevertheless, we do not take advantage of the multi-graph structure. Therefore, we will speak about graph in the sequel. Example: From our previous example, we derive the oriented graph depicted in Figure 2. We can easily verify that this oriented graph is acyclic. With such graphs at hand, we shall use in the next subsection the notion of successor of a node x in V, denoted by successor(x), and defined as follows: successor(x) = {y∈V∃x1, ..., x n with x1 = x, xn = y and (x i -> xi+1)∈E, ∀ i ∈ {1, n-1}}

Integrated Algorithm The main algorithm takes advantage of the underlying existing pre-order between object-types of the conceptual schema. We can therefore define a priority for performing the mapping: we first consider the elements of G without

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

62 Petit and Hacid

Figure 2. Directed graph of the conceptual schema of Figure 1

successor (except themselves) — they can be processed alone — then we remove those nodes and update the graph accordingly, until no more nodes remain in G. This is the main loop of the Algorithm 1 (line 3).

Algorithm 1. Main algorithm Algorithm MapERtoR Input: S, a well formed conceptual schema N, the notation to be used, i.e., UML or Merise Output: (R, F, I) a relational database schema Begin 1. (R, F, I) = ({},{},{}) Build the directed graph G = (V,E) from S while (V not empty) do 3. X := empty for all x ∈ V such that successor(x)=x do 5. X := X + x case x do : 7. x is an strong entity-type : (R, F, I) += Map_Entity (x) 9. x is an relationship-type : (R, F, I) += Map_Relationship (x) x is an inherited entity-type : (R, F, I) += Map_Inherited(x) 11. x is an weak entity-type : (R, F, I) += Map_Weak (x) end case end for all 13. V := V – X 15. E = E - {(x→y) ∈ Ε | x ∈ X or y ∈ X} end while return (R, F, I) End

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

From Conceptual Database Schemas to Logical Database Tuning

63

The complexity of this algorithm is clearly polynomial in the size of the input. The terminating condition is ensured whenever the graph G is acyclic, a condition which is met if the conceptual schema S is well-formed. Example: Continuing our example, three iterations are necessary: The first iteration gives three candidates {People, Projects, Departments}. Then, these nodes are removed and we get {Employees, sponsors} during the second iteration. Finally, it retains only one node {Dependents, monitors}. At a given iteration, there is no particular order to be taken into account to consider elements. For instance, the three elements of the first iteration can be treated in any order. Then the Algorithm 1 calls three main procedures, one for each type of main constructors: Map_Entity, Map_Relationship, Map_Inherited and Map_Derived. They are described below.

Mapping Strong Entity-Types with Map_Entity The procedure Map_Entity is straightforward since we do not deal with details of attribute names assignment. It is worth noting that Map_Entity is the unique procedure which does not add any new attribute during the translation. Example: Consider the strong entity-type People. A new relation schema People is added to R with schema(People)={ssn,name,tel} and one functional dependency People: ssn → name,tel is added to F, since only one key is defined over People (see Figure 1).

Algorithm 2. Mapping an entity-type to a relation schema Algorithm Map_Entity Input: E, a strong entity-type Output: (R, F, I) a relational database schema Begin Create a new relation schema R from E schema(R) = attributes of E Let K be a key defined on E R = R + {R}; for each key K defined over E do F = F + {R: K → schema(R)} end for Return (R, F, {}) End

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

64 Petit and Hacid

Algorithm 3. Mapping a relationship-type to a relation schema Algorithm Map_Relationship Input: A, a relationship type between O1, …, On Output: (R, F, I) a relational database schema Begin 1. Let R1, …, Rn be the relation schemas created from O1, …, On respectively. Let Ki be a key of Ri, i ∈ {1, n} and K = K1 + … + Kn 3. Create a new relation schema R from A R = R + {R} 5. schema(R) = attributes of A + K F = F + {K → schema(R) }; 7. for each Oi participating in relationship-type A do I = I + {R[Ki] ⊆ Ri[Ki] } if cardinality between Oi and A is equal to 1 then 9. if US notation then 11. F = F + {K- Ki → Ki } else 13. F = F + { Ki → K- Ki } end if 15. end if end for return (R, F, I) 17. End

Mapping Relationship Types with Map_Relationship The procedure Map_Relationship deals with relationship-types and handles both kind of cardinalities interpretation, referred to as, “US notation” and “EU notation” in the sequel. To our knowledge, this is the first contribution dealing with both kinds of constraints in a unified manner. We have seen that attributeoriented models need additional attributes and constraints to overcome their intrinsic limitations with respect to constructor oriented-models: this procedure implements such a kind of mapping, i.e., new attributes and new constraints must be added to get a semantically equivalent database schema. New attributes which are necessary to encode the relationship-type are exactly those denoted by K (line 2), i.e., they are the keys of the participating objecttypes. If several keys exist, one is arbitrarily chosen, for instance, the smallest one in the number of attributes. The new attributes and descriptive attributes of the relationship-type are added to the relation schema (line 5). Implicit constraints of the relationship-type are made explicit (lines 6, 8, 11 and 13): A functional dependency linking new attributes with descriptive attributes of the relationship-type is added (line 6). Then, each participating object-type leads to the creation of an inclusion dependency (line 8). If the cardinality constraint is equal to 1 (line 9) for this participating object-type, let say the ith component of the relationship-type, a functional dependency is added to the relation schema (line 11 or 13). If one complies with U.S. notation, all components except the ith

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

From Conceptual Database Schemas to Logical Database Tuning

65

determine the ith component (line 11). Otherwise, the ith component determines all other components (line 13). Note that in case of a reflexive relationship-type, there exists O i and O j referring to the same entity-type (line 1). In that case, attribute(s) of one of the corresponding key (Ki or Kj) must be renamed (line 2). Example: First recall that the conceptual model used in this chapter complies with U.S. notation. Now, consider the relationship-type sponsors: A new relation schema sponsors is added to R with schema(sponsors)= {did,pid,since}. Attributes did and pid are new attributes. The functional dependency sponsors: did,pid→since is added to F (line 6) and two leaving inclusion dependencies are created from sponsors: one to Departements (sponsors[did] ⊆ Departments[did]) and the other one to Projects (sponsors[pid] ⊆ Projects[pid]) (line 8). The cardinality constraint equals to 1 between Projects and sponsors yields to the creation of a functional dependency sponsors: did→pid (line 11). From the two functional dependencies defined over sponsors, did turns out to be a key and sponsors is trivially in BCNF.

Mapping Inherited Object-Types with Map_Inherited The procedure Map_Inherited deals with inherited object-types. Since multiple inheritance is allowed, we may have several generalizations for a given inherited object-type.

Algorithm 4. Mapping an inherited object-type to a relation schema Algorithm Map_Inherited Input: E, an inherited object-type derived from O1, …, On i.e. E is-a O1, …, E is-a On Output: (R, F, I) a relational database schema Begin 1. Let R1, …, Rn be the relation schemas created from O1, …, On respectively. Let Ki be a key of Ri, i ∈ {1, n} and K = K1 + … + Kn 3. Create a new relation schema R from E R = R + {R} 5. schema(R) = attributes of E + K for each key K defined over E do F = F + {R: K → schema(R)} end for for each E is-a Oi do 7. I = I + {R[Ki] ⊆ Ri[Ki] } F = F + {Ki → schema(R) } 9. end for return (R, F, I) End

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

66 Petit and Hacid

Note that if we have E is-a A, E is-a B, E is-a C, A is-a D and B is-a D, then E has two different keys, derived from D and from C. Example: Consider the inherited entity-type Employees. A new relation schema Employees is added to R with schema(Employees)={ssn,sal,lot}. Only one is-a relationship follows from Employees, so only one inclusion dependency Employees[ssn] ⊆ People[ssn]is created in I. No keys are defined over Employees and only one key is available from People, so we get one functional dependency Employees: ssn → sal,lot in F.

Mapping Weak Entity-Types with Map_Weak The procedure Map_Weak deals with weak entity-types. Basically, weak entity-types are a special case of relationship-types and due to restrictions imposed by well-formed conceptual schemas, a weak entity-type may be derived from only one object-type, which can be an entity-type or a relationship-type. Example: Consider the weak entity-type Dependents. A new relation schema D e p e n d e n t s is added to R with sc h e m a ( D e p e n d e n t s ) = {ssn,pname,age,cost}, one inclusion dependency Dependents[ssn] ⊆ Employees[ssn] and one functional dependency Dependents: ssn,pname → age,cost are added to I and F, respectively. A unique compound key exists, i.e., {ssn,pname}. Finally, applying MapERtoR yields the following relational database schema: R={Employees,Projects, Departments, Dependents, monitors, sponsors}. Details are shown in Table 1. Attributes in bold are exactly the Algorithm 5. Mapping a weak entity-type to a relation schema Algorithm Map_Derived Input: W, a weak entity-type derived from O Output: (R, F, I) a relational database schema Begin 1. Let S be the relation schema created from O Let K be a key of S 3. Create a new relation schema R from W R = R + {R} 5. schema(R) = attributes of W + K Let LK be the local key of W 7. F = F + {LK + K → schema(R)} I = I + {R[Ki] ⊆ S[Ki] } return (R, F, I) 9. End

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

From Conceptual Database Schemas to Logical Database Tuning

67

Table 1. Relational database schema (logical schema) Relation

Attributes

FD specified within the

schema

(schema of)

relation schema ssn → name, tel

People

ssn, name, tel

Departments

did, dname,

did → dname, budget;

budget

key of Departments: did

pid, started,

pid → started, pbudget

Leaving IND /

key of Employees: ssn

Projects

Employees

Sponsors

/

/

pbudget

key of Projects: pid

ssn, sal, lot

ssn → sal, lot

Employees[ssn] ⊆

key of Employees: ssn

People[ssn]

did, pid, since did, pid → since

sponsors[did] ⊆

did → pid

Departments[did],

key of sponsors : did

sponsors[pid] ⊆ Projects[pid]

Dependents

pname, ssn,

ssn, pname → age, cost

Dependents[ssn] ⊆

age, cost

key of Dependents:

Employees[ssn]

{ssn,pname} Monitors

did, ssn, until did, ssn → until did → ssn;

monitors[did] ⊆ sponsors[did] monitors[ssn] ⊆

key of monitors: did

Employees[ssn]

attributes induced by the transformation of a constructor-oriented model to an attribute-oriented model.

Properties of the Translation Some formal properties can be stated for such kinds of mappings (Markowitz, 1992; Rosenthal, 1994; Mannila, 1994). Without going into many details, the database schema obtained after applying the algorithm MapERtoR has the following properties:

• •

Each relation schema is in BCNF. I is made up of key-based inclusion dependencies (their right-hand sides are keys) and the oriented graph of I (a node corresponds to a relation schema and an edge between R and S corresponds to an element R[X] ⊆ S[Y] of I) is acyclic.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

68 Petit and Hacid

•

F and I do not interact, i.e., logical implication with F and I can be restricted to logical implication with F or with I.

Other properties can be given on such a mapping such as those related to information capacity (e.g., Miller, 1994; Hacid, 2001).

Logical Database Tuning Once a logical database schema has been generated, a physical database schema has to be derived using some DBMS-dependent language. The implementation of a physical database schema by itself is rather straightforward from a logical database schema. Physical database tuning such as definition of block size, indexes or clusters is out of the scope of this chapter. Therefore, we focus on de-normalization techniques usually performed at the logical level with respect to three criteria:

• • •

null values, length of join paths, and data redundancies, i.e., BCNF.

Then, we will show how SQL workloads can be used to tune the design of database schemas with respect to SQL accesses performed over the database during some periods of time.

Getting a Null Free Database Occurrence of null values is quite common in real life databases and is known to be one of the major difficulties for database programmers when they have to write SQL queries. In fact, problems raised by null values depend on which kind of attributes they occur:

•

On duplicated attributes, i.e., attributes which enable attribute-oriented models to simulate constructor-oriented models: null values can be a nightmare to compose queries involving joins, specific RDBMS functions, etc.

•

On non-duplicated attributes: null values are not very challenging for designing SQL queries. Most of the time, null values were missing at the insertion time of a tuple, but such values are not used anymore to navigate

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

From Conceptual Database Schemas to Logical Database Tuning

69

through the database schema. These attributes are descriptive only: they are defined within a relation schema and convey part of the information or semantics of this relation schema. To get a “null free” database on a duplicate attribute, the logical database schema obtained from the conceptual schema with the MapERtoR algorithm has to be implemented as it is, i.e., no transformation (or degradation) has to be performed. However, the price to pay is that the length of join paths is maximized. Indeed each functional dependency turns out to be a key or a super-key and each inclusion dependency turns out to be a foreign key, both of them being enforced to be not null by the RDBMS.

Getting an Efficient Database In that case, there are two main options:

•

Reducing the length of join paths without sacrificing high normal form based on functional dependencies such as BCNF or third normal form (3NF).

•

Reducing the length of join paths by introducing data redundancy (2NF, 1NF).

Denormalisation without Data Integrity Problems To be compliant with the first option, a very well known transformation can be done: instead of creating a new relation schema for each one-to-many or oneto-one binary relationship-type, it consists of migrating attributes of the relationship-type (if any) and a foreign key into the entity-type (or relationship-type) that participates with the cardinality constraint equals to one. Example: From our example, if we follow the previous transformation, we get the logical database design given in Table 2. Note that the number of relation schemas and inclusion dependencies decreases at the price of null values on the three duplicate attributes: pid and ssn into Departments and ssn into Dependents. Consider the one-to-many relationship-type sponsors. It has been merged into the relation schema Departments, its own attributes being now part of the schema of Departments. To show an example (not very realistic here) of the impact of such a denormalization on the number of null values, let’s assume we have 10,000 departments of which only 100 participate in the relationship-type

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

70 Petit and Hacid

Table 2. Another relational database schema (logical schema) Relation

Attributes

schema

(schema of)

Employees

Keys

ssn, name,

Leaving IND /

lot, tel, sal ssn Projects

pid, started, pbudget

Departments

/

pid Departments[pid] ⊆ Projects[pid]

did, dname, budget, pid,

did

since, ssn,

Departments[ssn] ⊆ Employees[ssn]

until Dependents

Dependents[ssn] ⊆ Employees[ssn]

pname, ssn, age, cost

{ssn,pname}

sponsors. In that case, 99% of tuples of Departements get null values on the duplicated attribute pid, and less importantly, on attribute since. Such kind of problems never append with database schema produced by the algorithm MapERtoR. To sum up, such kind of logical database schema is often chosen to produce physical database schemas, its main advantage being to minimize the length of join paths, and thus to be rather efficient. The often misunderstood problem of such schemas concerns the number of null values which can be generated once the database is operational. For database designers, it might not be an important issue at database design time, but that could become a nightmare for database programmers who have to devise SQL queries in presence of null values on duplicated attributes.

Denormalisation with Data Integrity Problems From Table 2, we may be tempted to go a step beyond in order to avoid costly joins. In that case, data integrity problems will inevitably occur. Example: Consider the merging of Dependents and Employees into Employees. In that case, we have schema(Employees)={ssn, name, lot, tel, sal, pname, age, cost}, the inclusion dependency Dependents[ssn] ⊆ Employees[ssn] being removed. Two functional dependencies need to be defined over Employees, namely ssn,pname → age,cost and ssn → name,lot,tel,sal.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

From Conceptual Database Schemas to Logical Database Tuning

71

The only key is made up of two attributes, i.e., ssn,pname. As a consequence, this schema is only in first normal form since the left-hand side of the second functional dependency is strictly included in a key.

How to Reach a Compromise? Clearly, a compromise has to be reached between the two opposite goals. In the spirit of (Ramakrishnan, 2003), we argue that a good design cannot be obtained at database design time: too many parameters have to be taken into account at an early stage of the design, specifically those related to application programs accessing the database. Nevertheless, an optimal design could be defined and obtained with respect to the database accesses as given by SQL workloads. We argue that SQL workloads could be used to tune the database design of operational databases since they offer a nice setting in which logical database tuning can be treated objectively — with respect to SQL workloads — instead of subjectively — with respect to the database designer expertise.

Gathering SQL Workloads SQL workloads represent a set of SQL accesses performed over the database during some periods of time. They should be representative of the database activity, either Select From Where SQL queries or update SQL queries (insert/ delete/update). Recently, SQL workloads can be easily gathered from operational databases by means of advanced functions available on top of major RDBMS products: a representative workload can be generated by logging activity on the server and filtering the events we want to monitor (Agrawal, 2001).

Using SQL Statements to Tune the Logical Database Design The key idea is to tune the design with respect to three main goals: minimizing the occurrence of null values, maximizing both the efficiency of cost-sensitive SQL queries performed against the database and data integrity of the database. Example: Assume that SQL workloads reveal that cost-sensitive SQL queries occur in a majority of cases between Departments and Projects. In that

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

72 Petit and Hacid

case, the logical database schema given in Table 1 could be transformed (or denormalized) only with respect to the length of join paths implied by SQL workloads, i.e., the relation schema sponsors could be merged into Departments, the rest of the logical schema remaining unchanged, up to some transformations of the set of inclusion dependencies. The idea of tuning database parameters at the physical or logical level is not new. For instance, they are proven to be useful in many database applications such as physical database tuning, e.g., automatic index definition (Agrawal, 2001), logical database tuning (Lopes, 2002) or materialized view selections in a data warehouse (Agrawal, 2001).

Conclusions Relational database technology and semantic data modeling have been two major areas of database research in recent decades. Relational database technology is based on solid theoretical foundations, and it is understood what constitutes a well-designed relational database schema. Semantic modeling, on the other hand, provides a rich set of data abstraction primitives which can capture additional semantics of the application in the database schema. Until recently, relational database technology and semantic modeling have evolved almost separately. There is a need for establishing and understanding connections between semantic models and the relational model. This chapter is an attempt to investigate this connection. We tackled this problem by restricting the class of data dependencies to functional dependencies and inclusion dependencies. The results of our work are directed toward the understanding of the properties of relational translations of (extended) ER schemas. We clarified two main steps in such a translation: (1) the order of the translation of entity and relationship-types and (2) the translation of cardinalities for relationship-types, whatever the convention chosen to interpret these cardinalities (for example, UML class diagrams or conceptual data schemas of Merise). These considerations are simple though very important in practice. Between the desire to get efficient databases for end-users and the desire to get null-free databases on duplicated attributes for database programmers, we have pointed out how SQL workloads could be used to reach a compromise among contradictory objectives.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

From Conceptual Database Schemas to Logical Database Tuning

73

References Agrawal, S., Chaudhuri, S. & Narasayya, V.R. (2001). Materialized View and Index Selection Tool for Microsoft SQL Server 2000, ACM SIGMOD 2001, California, May 21-24. Chen, P. (1976). The Entity-Relationship Model - Toward a Unified View of Data. ACM TODS, 1(1), 9-36. Fahrner, C. & Vossen, G. (1995). A Survey of Database Design Transformations Based on the Entity-Relationship Model. DKE, 15, 213-250. Franconi, E. & Ng, G. (2000). The i.com tool for Intelligent Conceptual Modeling. Proceedings of the Seventh International Workshop on Knowledge Representation Meets Databases (KRDB 2000), Berlin, Germany, 2000 (pp. 45-53). Hacid, M.S., Petit, J.M. & Toumani, F. (2001). Representing and reasoning on database conceptual schemas. Knowledge and Information Systems, 3(1), 52-80. Hull, R. & King, R. (1987). Semantic Database Modelling: Survey, Applications, and Research Issues. ACM Computing Surveys, 19(3), 201-260. Jacobson, I., Booch, G. & Rumbaugh, J.E. (1999). Excerpt from “The Unified Software Development Process”: The Unified Process. IEEE Software, 16(3), 82-90. Levene, M. & Loizou, G. (1999). A Guided Tour of Relational Databases and Beyond. Springer. Lopes, S., Petit, J.M. & Toumani, F. (2002). Discovering interesting inclusion dependencies: Application to logical database tuning. Information Systems, 27(1), 1-19. Mannila, H. & Räihä, K.J. (1994). The Design of Relational Databases (2nd ed.). Addison-Wesley. Markowitz, V. & Shoshani, A. (1992). Representing Extended Entity-Relationship Structures in Relational Databases: A Modular Approach. ACM TODS, 17(3), 423-464. Miller, R.J., Ioannidis, Y.E. & Ramakrishnan, R. (1994). Schema equivalence in heterogeneous systems: Bridging theory and practice. Information Systems, 19(1), 3-31. Moulin, P., Randon, J., Teboul, M., Savoysky, S., Spaccapietra, S. & Tardieu, H. (1976). Conceptual Model as a Data Base Design Tool. Proceeding of the IFIP Working Conference on Modelling in Data Base Management Systems. In G. M. Nijssen (Ed.), Modelling in Data Base Management

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

74 Petit and Hacid

Systems, Freudenstadt, Germany, January 5-8, 1976 (pp. 221-238). NorthHolland. Navathe, S. (1992). Evolution of data modeling for databases. Communications of the ACM, 35(9), 112-123. Peckham, J. & Maryanski, F. (1988). Semantic data models. ACM Computing Surveys, 20(3), 153-189. Ramakrishnan, R. & Gehrke, J. (2003). Database Management Systems (3rd ed.). McGraw-Hill. Rosenthal, A. & Reiner, D.S. (1994). Tools and Transformations - Rigorous and Otherwise - for Practical Database Design. ACM TODS, 19(2), 167-211. Tardieu, H., Pascot, D., Nanci, D. & Heckroth, H. (1979). A Method, a Formalism and Tools for Database Design - Three Years of Experimental Practice. ER 1979: 353-378. Teorey, T., Dongqing, Y. & Fry, J. (1986). A Logical Design Methodology for Relational Databases Using the Extended Entity-Relationship Model. ACM Computing Surveys, 18(2), 197-222.

Endnote 1

Other extensions could have been integrated into our ER model such as multi-valued attributes or composite attributes. In order to ease the presentation of the mapping, they are not taken into account in this chapter but could be integrated into our framework without any major difficulty.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Transformation Based XML Query Optimization

75

Chapter IV

Transformation Based XML Query Optimization Dunren Che, Southern Illinois University, USA

Abstract This chapter addresses the fundamental issues of XML query optimization – equivalences and transformation rules. Equivalences are typically used as the basis to pursue transformation-based query optimization in traditional database systems. In this chapter, we develop the conception of equivalences in the context of XML database and XML query optimization. We then propose to derive deterministic transformation rules by incorporating general and XML-specific optimization heuristics to achieve high optimization efficiency for XML queries. We hope the work presented in this chapter will set a preliminary foundation or framework for further in-depth investigation for transformation-based XML query optimization.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

76 Che

Introduction With the advent of the Internet and World Wide Web (WWW), the repositories of SGML (Standard Generalized Markup Language) compliant structured documents have been fast mounting up. XML (Extensible Markup Language), as the new proposal of W3C (World Wide Web Consortium) for standard, is hurriedly getting the dominance of representing data in the Web and elsewhere. Therefore, commensurate management technology, including efficient query processing and optimization for XML data is specially needed. It has been commonly recognized that structured documents in general and SGML/XML documents in particular should benefit from the same type of database management functionality as offered to traditional data. This requires the storage of the documents within a database (which we call a structured-document database) and management of these documents by a database management system (DBMS). Within the context of this chapter, structured documents refer to documents according to the SGML/XML/HTML standards (Cover, 2002). Efficient processing and execution of declarative queries over structureddocument databases are an essential issue for structured-document database management systems. This issue, however, has not been adequately studied. Structured-document query optimization is fundamentally different from classical query optimization in two aspects. First, because of the high complexity of the intrinsic data model behind XML data, the search space for query optimization is much larger, which means the efficiency of traditional optimization approaches will degrade unacceptably when applied to XML data. In other words, we have to work out a much more progressive way of pruning the search space to achieve acceptable performance. Second, the structure of XML documents, which can be interrogated in an XML query and is normally implied in the DTD or XML schema of the documents, provides opportunities for efficient semantic query optimization, which shall be effectively exploited to achieve better optimization efficiency for XML queries. This chapter addresses the optimization issue of structured-document queries in a database environment. Considering the dominance that XML has already gained, our discussion is focused on XML-compliant documents, which are more generally referred to as XML data. The query optimization strategy we present here is transformation-based. The optimization of a query is accomplished through a series of equivalent transformations applied to the query. Transformation rules in our system are all derived from the equivalences that we identified in the specific context of XML document queries. The main theme of this chapter is XML-document specific equivalences and the transformation rules derived for XML query optimization.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Transformation Based XML Query Optimization

77

The remainder of this chapter is organized as follows: the chapter first gives a brief review of related work. Next, it provides the preliminaries needed for the subsequent discussion of this chapter. The chapter then addresses selected equivalences that we have identified and used in our test bed. This is followed by a section that discusses the intended application of our equivalences, i.e., deterministic transformations for XML query optimization, followed by an optimization example. The chapter then concludes with a summary of the discussion of this chapter and indicates future directions.

Related Work Since SGML/XML entered the arena of database technology, a lot of work has been done on the various aspects related to XML data management, e.g., structured-document data modeling (Abiteboul, 1997; Özsu, 1997; Navarro, 1997; Lee, 1997; Yan, 1994; Conse, 1994; Chaudhuri, 1996; Morishima, 1997; Gonnet, 1992; Fernadez, 2000; Florescu, 1999; Shanmugasundaram, 1999; Bohannon, 2002; Klettke, 2000), XML document indexing (Chan, 2002; Grust, 2002; Li, 2001; Milo, 1999), and advanced algorithms for fast query processing (Fernandez, 1998; McHugh, 1997; Gottlob, 2002; Li, 2001; Guha, 2002; Chien, 2002; Srivastava, 2002). As structured-documents are essentially semistructured, the work done on semistructured data management (e.g., Deutsch, 1999; McHugh, 1997) actually addresses similar issues as structured-document management. Lore (McHugh, 1997, 1999), a DBMS designed for semistructured data and later migrated to XML, has a fully-implemented cost-based query optimizer that transforms a query into a logical query plan, and then explores the (exponential) space of possible physical plans looking for the one with the least estimated cost. Lore is well known by its DataGuide path index that together with stored statistics describing the “shape” of the database provides the structure knowledge about the data to help Lore’s optimizer prune its search space for a better plan. In this sense, Lore is related to our work, but we capture the structure knowledge of document data mainly from the DTDs and apply this knowledge on conducting exclusively deterministic transformations on query expressions. In the work of Fernandez (1998), a comparable strategy for exploiting a grammar specification for optimizing queries on semistructured data is discussed, where effort is made to make complete use of the available grammar for expanding a given query. Our focus is different. We identify transformations that introduce improvements on query expressions in a very goal-oriented manner.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

78 Che

The work of Consens and Milo (1994) also exploits the document type definition knowledge for query optimization. They replace a query algebra operator with a cheaper one whenever the DTD allows. However, the DTDs considered in their study are simpler than the ones of SGML/XML, and the authors do not consider different grammar constructors. Our work is innovative in systematically addressing the query optimization issue for structured documents through algebraic transformations. Our approach exploits the structure knowledge implied by the DTD and other heuristics to conduct strongly goal-driven, deterministic and thus highly efficient transformations on query expressions for optimization.

Preliminaries In this chapter, we are interested in structured documents that follow the SGML/ XML standards (Cover, 2002). Our optimization techniques perform algebraic transformations on query expressions based on the PAT algebra (Salminen, 1994). The main theme of this work is to exploit the structure knowledge about the documents, which is usually characterized by the DTD or XML schema of the documents. The structure knowledge is used to conduct profitable transformations on query expressions for optimization. In this section we first introduce a few DTD-related notions that are important for the subsequent discussion of this chapter, followed by the PAT algebra, which forms the basis of query transformations in our work.

DTD-Related Notions XML, derived from SGML, is a markup language that identifies components of structured documents by means of “tags”. For a given class of documents, the legal markup tags are defined in the DTD or the XML schema of the documents. XML schema was proposed to be an alternative to DTD for defining document element types and is currently used in parallel with DTD. Subsequent discussion in this chapter is based on the DTD notion, but the techniques we discuss apply likewise to documents using XML schemas. A DTD is the grammar for a class of documents. The figurative representation of this grammar is a DTD graph, which is a useful concept and is formally defined in the following text.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Transformation Based XML Query Optimization

79

Definition 1 (DTD graph). The DTD graph of a specific DTD is a directed graph G = (V,E). Its vertices in V are the names of the element types from the DTD, and each element type name occurs only once. An edge (ETi,ETj) in E indicates that ETj occurs in the content model of ETi. RT ∈ V is the root element type of the DTD. Figure 1 is an illustration of a fragment of the DTD graph for conference proceedings. By means of a DTD graph, we can visualize some important relationships induced by the DTD, such as the containment relationships among document elements, which is defined next. Definition 2 (Directly-contains/Directly-contained-in). Element type ETi directly-contains element type ETj if there exists an edge (ET i,ETj) in the DTD graph G. Conversely, ETj is directly-contained-in ETi. ETi is referred to as an external type, and ETj an internal type with regard to the containment relationship. The transitive closure of directly-contains (or directly-contained-in) reflects the generalized situation, which is usually referred to as contains (or containedin). Definition 3 (Path in DTD graph). A path in a DTD graph G, is a sequence of element types (ETi, ... , ET j) s.t. ETk directly contains ETk+1, i ≤ k < j. Figure 1. Fragment of a sample DTD graph for conference proceedings

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

80 Che

It is obvious that the edge and path concepts in a DTD graph are the graphic counterparts of directly-contains/contained-in and contains/contained-in relationships among element types of the DTD. Literally, the term “a path from ET1 to ET2” is different from “a path between ET1 and ET2”. The latter does not concern the direction of the path. Path and edge are important concepts for the identification of relevant structural properties of documents for semantic query optimization, which we discuss in subsequent sections. Notice that the above notions are defined at the type level of document components, but imply structural relationships at the instance level. As long as DTDs are available, we rely on DTDs to extract the structure knowledge of documents, otherwise we need to obtain this useful structure knowledge for query optimization by means of a document parser. In addition to the above notions, further notions regarding the properties of the structural relationships among document components are defined and used for deriving the core transformation equivalences for query optimization.

PAT Algebra The PAT algebra (Salminen, 1994) was designed as algebra for searching structured documents. We adopted the PAT algebra and extended it according to the features of SGML/XML compliant documents. The PAT algebra is set oriented, in the sense that each PAT algebraic operator and each PAT expression evaluate to a set of elements. Herein, we present a restricted version of it to serve the purpose of this chapter. A PAT expression is generated according to the following grammar: E ::= etn | E1∪E2 | E1∩E2 | E1 – E2 | σr(E) | σ A,r(E) | E1⊂E2 | E1⊃E2 | (E) “E” (as well “E1” and “E2”) generally stands for a PAT expression, etn introduces a document element type’s name, “r” is a regular expression representing a matching condition on the textual content of the document elements, and “A” designates an attribute of the elements. ∪, ∩ and – are the standard set operators, union, intersection and difference. The two operands of a set operator have to be type-compatible, i.e., returning the same type of elements.

σr(E) takes a set of elements and returns those whose content matches the regular expression r, whileσA,r(E) takes a set of elements and returns those whose value of attribute A matches the regular expression r. Operator ⊂ returns

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Transformation Based XML Query Optimization

81

all elements of the first argument that are contained in an element of the second argument, while ⊃ returns all elements of the first argument that contain an element of the second argument. In the subsequent discussion, we use ext(E) and τ(E) as shorthands to denote the element extent determined by the expression E after evaluation and the result type of the elements in ext(E), respectively. Following is a query example formulated according to the PAT algebra with regard to the DTD shown in Figure 1: Query example. Find all the paragraphs containing both “xpath” and “xlink” in any article. ( (σr=“xpath”(Paragraph) ⊂ Article) ∩ (σr=“xlink”(Paragraph) ⊂ Article) )

Structured-Document Indices We consider three types of index structures in the context of structureddocument databases: element content indices, attribute value indices, and structure indices. Content indexing has been extensively exploited in information retrieval (IR) systems. Attribute indexing corresponds to the classical type of indexing pursued in conventional DBMSs. Structure indexing is specific to structured-documents. It is similar to path indexing (Bertino, 1994) studied in object-oriented databases. However, in our case the indexing mechanism needs to consider the containment relationship between the relating elements. When the containment relationship between the elements of two different element types is expected to be frequently taken as a criterion for querying the document database, a structure index between the two element types is favorably built. Structure indices may be defined either from an external type to an internal type or vice versa, corresponding to the “contains” and “containedin” relationships, respectively. To serve the purpose of this chapter, it suffices to indicate that a structure index is simply defined as a map between the elements of the two types related via the containment relationship. We introduce an additional PAT operator, Iτ(E1)(E2), to indicate the application of a structure index which is defined between the two types, τ (E1) and τ (E2), with the subscript “τ(E1)” showing the return type of the index operation. This index operator will be used in relevant equivalences and transformation rules.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

82 Che

Equivalences for Document Query Optimization Our approach to XML query optimization is based on equivalent transformations applied to the queries represented as PAT algebraic expressions. Equivalences are the laws in our approach, based on which we derive the transformation rules to accomplish query optimization. In this section, we elaborate on the equivalence issue in the context of XML queries. There are potentially four types of equivalences:

•

Equivalences derived from general set-theoretic properties of the PAT operators.

•

Equivalences derived from the constraints imposed by the DTD of the documents.

•

Equivalences derived from particular structural properties of the document.

•

Equivalences that explore opportunities of applying potential indices.

Considering that the number of the PAT algebraic operators and the complexity of their potential interactions are rather high, we also apply certain criteria in the selection of the equivalences to restrict their number. Otherwise, we would obtain an unmanageably large set of potential transformations. The criteria we observe are as follows:

• •

Equivalence must have the potential to imply a profitable transformation.

• •

Equivalences must not require complex conditions to be checked.

Equivalence must not imply transformations to further complicate or expand the query expressions. Equivalences must not target at merely generating alternative expressions.

c In our study, equivalences take the form “E1 ← → E2,” meaning E2 is equivalent to E1 under condition c; when the condition c is omitted, the corresponding equivalence holds unconditionally. Furthermore, we introduce the following shorthand notations:

• • •

“ ∪ ∩ ”: stands for the set operator ∪ and/or ∩. “σR”: acts for the selection operator σr and/or σA,r. “ ⊃ ⊂ ”: represents the containment operator ⊃ and/or ⊂.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Transformation Based XML Query Optimization

83

Set-Oriented Equivalences Set-oriented equivalences are based on the set-theoretic properties of the PAT operators. We identified 31 generic equivalences in Che (2003). Due to lack of space, we present only a few of them below: Subset laws (5 stands for any subset of ext(E))

ε1. 5 – E ←→ φ ε2. 5 ⊃ ⊂ E ←→ 5 ‘–’ specific laws

ε3. ((E1 – E2) – E2) ←→ (E1 – E2) ε4. ((E1 – E2)∪E2) ←→ (E1∪E2) Commutativity ε5. E1∪E2 ←→ E2∪E1 ε6. (E1 ⊃ ⊂ E2) ⊃ ⊂ E3 ←→ (E1 ⊃ ⊂ E3) ⊃ ⊂ E2 Distributivity

ε7. E1∩(E2∪E3) ←→ (E1∩E2)∪(E1∩E3) ε8. E1 – (E2∪E3) ←→ (E1 – E2) – E3 Associativity c ε9. (E1 ⊃ ⊂ E2) ⊃ ⊂ E3 ← E1 ⊃ ⊂ (E2 ⊃ ⊂ E3) → if τ (E2) contains/is contained in τ (E3)

The “subset laws” are useful for simplification of query expressions involving a subset relationship. The proof of the above equivalences is straightforward except for the commutativity and associativity rules. In the following, as an example, we give the proof of the commutativity law, ε 6, with regard to only the ⊂ operator. Proof. (the ⊂ commutativity) (E1 ⊂ E2) ⊂ E3 = {e1∈ext(E1 ⊂E2) | ∃e3∈ext(E3) s.t. e1 is contained in e3} = {e1∈ext(E1) | ∃e2∈ext(E2) s.t. e1 is contained in e2 and ∃e3∈ext(E3) s.t. e1 is contained in e3} = {e1 ∈ext(E1⊂E3) | ∃e2∈ext(E2) s.t. e1 is contained in e2} = (E1⊂E3)⊂E2 Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

84 Che

Equivalences Based on DTD Constraints This type of equivalences is based on checking whether a single operator of the PAT algebra is consistent with the given DTD. A single operand can be either atomic, i.e., an element type name, or any of the operators, σA,r, σr(E), ⊂ or ⊃. Following is an example of such equivalences:

ε10. σA,r(E) ←→ φ if ‘A’ is not an attribute of τ(E) Note that more complicated equivalences reducing expressions to an empty set exist, but need to take more than just one operator into account for deducing an empty result, thus are precluded from our consideration according to our rule selection criteria mentioned before.

Equivalences Based on Properties of DTD Structure Although we have ruled out in the previous section the situations where complicated exploitation of the DTD knowledge is needed, there are particularly interesting situations that the DTD structure can be used most profitably to achieve query optimization. We show a few of such special cases and present the corresponding equivalences in this subsection. First, we introduce the notions of exclusivity, obligation, and entrance locations. In a given DTD, some types may be shared among others. For example, the element type Authors shown in Figure 1 is contained in element type Article and in ShortPaper. But the types that are not shared, i.e., exclusively contained-in another type, bear potential for query optimization. Definition 4 (Exclusivity). Element type ETj is exclusively contained in element type ETi if each path (e j,...,ek) with ej being an element of type ETj and ek being the document root contains an element of type ET i. We say an expression, E, is free of restriction on its extent, denoted as free(E), if the expression evaluates to the full extent of type τ(E). One typical case that the free(E) condition holds is when E is a plain element-type name etn. The free PAT expression notion often serves as a necessary condition to make an equivalence hold. The following equivalence comes from the exclusivity and the free of restriction notions.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Transformation Based XML Query Optimization

85

ε11. E1⊂E2 ←→ E1 if τ(E1) is exclusively contained in τ(E2) and free(E2) holds If τ(E1) is exclusively contained in τ(E2), an element e1 of the internal type τ(E1) must be contained in an element e2 of type τ(E2). Furthermore, free(E2) renders E2 to return the full extent of τ(E2). Therefore, the containment relationship in the above equivalence imposes no condition on the elements of E1, and thus the equivalence holds. It is easy to conclude from the operators’ definitions that the expression “E1⊂E2” and “E1⊃E2” are not equivalent, thus the exclusivity notion is not appropriate to transform queries involving subexpressions of the second form, and we need to introduce the counterpart of exclusivity, obligation, for this purpose. Definition 5 (Obligation). Element type ETi obligatorily contains element type ETj if each element of type ET i in any document complying with the DTD has to contain an element of type ET j. The concept of obligation gives rise to the following equivalence:

ε12. E1⊃E2 ←→ E1 if τ(E1) obligatorily contains τ(E2) and free(E2) If two element types are not related by exclusivity and obligation, it may be worthwhile to check whether a third element type, called entrance location, exists that could render us opportunities for applying a potential structure index or shortening the navigation path needed for evaluating an involved containment operation. Definition 6 (Entrance location). Element type EL is an entrance location for τ(E1) and τ(E2) of a given DTD if in any document complying with the DTD, all paths from an element e1 of τ(E1) to an element e2 of τ(E2) pass through an element el of τ(EL). The notion of entrance location gives rise to the following equivalence:

ε13. E1⊂E2 ←→ E1⊂(E3⊂E2) if τ(E3) is an entrance location for τ(E1) and τ(E2), and free(E3) holds.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

86 Che

The correctness of this equivalence trivially follows from the entrance location definition. The equivalence corresponding to the ⊃ operation can be defined likewise. In general, the adding of an additional element type, as introduced by an entrance location, to a PAT expression is detrimental to the evaluation efficiency of the query. Therefore, the leftward application of these equivalences is obviously favorable, while the left to right-side transformation will only be applied under certain special conditions to ensure final, obvious improvements on the query expressions, e.g., to enable application of a structure index. We will show this by the following transformation rule that combines exclusivity and entrance location:

ε14. E1⊂E2 ←→ E1⊂E3 if τ(E3) is an entrance location for τ(E1) and τ(E2), τ(E3) is exclusively contained in τ(E2), and free(E2) and free(E3) holds. The correctness of this equivalence becomes evident when the omitted intermediate term, E1⊂(E3⊂E2), is added. Analogously, the equivalence that combines obligation and entrance location can be defined, but omitted herein.

Equivalences Applying Structure Indices Using the index operation introduced in Section 3, the equivalence exploiting structure indexes is given below:

ε15. E1 ⊃ ⊂ E2 ←→ Iτ(E1)(E2)∩E1 if a structure index between τ(E1) and τ(E2) is available. When free(E1) additionally holds, Iτ(E1)(E2) is subset of ext(τ (E1)), and the intersection is thus redundant and can be omitted.

Application of Equivalences We envision two types of applications of our equivalences. One typical way is to directly apply the equivalences on query expressions for generating more

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Transformation Based XML Query Optimization

87

alternatives for each input query expression, and then according to a certain criterion, e.g., cost comparison, choose the one with the cheapest cost. The second way is to conduct only beneficial transformations on query expressions toward the goal of optimization, which is usually achieved with resort to heuristics. The strategy adopted in our work is strongly heuristics-based as it applies only deterministic transformations on query expressions for optimization. Here, the determinism consists in: (1) all transformation rules are unidirectional, of which each obtains determinate improvement on its input queries, (2) once a new transformation is performed, the previous candidate (i.e., the input expression to the transformation rule) is immediately discarded. The whole optimization process conducted according to this strategy is a linear, deterministic process, which step-wise improves on an input query and leads to the unique, final, optimal alternative of the input query. Query optimization thus accomplished is highly efficient because of the determinism nature.

Control Strategy In our system, all the performed transformations are required to lead to step-bystep improvements on an input query expression until a final expression is reached. Examples of significant improvements may be a potential structure index being introduced or the input query expression being completely rewritten to be more evaluation-efficient. In both cases, equivalent transformations are performed according to XML-specific semantics at the PAT algebra level. So the key transformations pursued in our system are heuristics-based semantic transformations, which are usually conducted more efficiently by starting from a carefully chosen standard format. In our system, this standard format is achieved via a normalization step, called normalization phase, which also carries out necessary simplification on input query expressions. The second phase is, of course, the semantic optimization phase. During semantic transformation, in order to introduce a major improvement into an expression, element names may be substituted and the newly introduced element names may have redundancy with other parts of the expression. Therefore, a final cleaning-up or simplification phase is employed. In the following, we present sample transformation rules (Che, 2003), and then show an optimization example using these rules. Transformation rules are derived from the more general equivalences. Transformation rules are unidirectional and take the form “(E1)⇒(E2)”. An additional precondition may be added to some rules to determine the applicability of the rule to a specific input expression.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

88 Che

Transformation Rules Transformation rules are organized into three consecutive phases: normalization, semantics optimization, and simplification.

Normalization Rules Query normalization covers three aspects: DTD-constraint based simplification (like R1), operator reordering (like R 2 - R5), and a bottom-up simplification (like R6 - R7). Following are a few of such rules chosen from our rule set (Che, 2003):

R1. (E1⊂E2) ⇒ φ if τ(E2) doesn’t contain τ(E1) R2. (σR(E1)∩E2) ⇒(σR(E1∩E2)) R3. (E1∩σ R(E2))⇒(σR(E1∩E2)) R4. (E1 ⊃ ⊂ E2)∩E3⇒(E1∩E3) ⊃ ⊂ E2 R5. E1∩(E2 ⊃ ⊂ E3)⇒(E1∩E2) ⊃ ⊂ E3 R6. (E∩E)⇒(E) R7. ((E1 ⊃ ⊂ E2) ⊃ ⊂ E2)⇒(E1 ⊃ ⊂ E2) Instead of giving a proof, we provide brief explanations of the above rules. R1 directly comes from the DTD constraint used. As ⊃ ⊂ operations restrict the first argument’s extent by imposing a containment relationship with the elements of the second argument, we may simply consider ⊃ ⊂ operations as a special kind of selections. In this sense, R2 through R5 are communications of the intersection with a selection operation; their correctness is self-evident. R6 is yet another basic law of the set theory. R7 is to delete redundant restriction imposed by the containment relationship with the same argument.

Semantic Rules Semantic rules are developed with the predominant goal: to enable the exploitation of structure indices during optimization, which in most cases is not readily achievable, rather, relies on deep exploration of DTD knowledge such as obligation, exclusivity, and entrance location.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Transformation Based XML Query Optimization

89

Numerous cases have been identified for introducing structure indices into a query (Che, 2003). The simplest case is to directly use an available structure index between the two element types involved in a query:

R8. E1 ⊃ ⊂ E2⇒Iτ(E1)(E2)∩E1 if a structure index between τ(E1) and τ(E2) is available Iτ(E1)(E2) denotes a structure index operation defined between τ(E1) and τ(E2), where the subscript τ(E1) indicates the result type of the operation. This rule is based on the index substitution equivalence, ε15, to interpolate the index operation into a query expression. The second case is designed to reveal the applicability of a potential structure index that is not directly available. The corresponding rule combines the commutativity and associativity laws into a single transformation:

R9. (E1 ⊃ ⊂ (E2 ⊃ ⊂ E3))⇒((I τ(E1)(E3) ∩E1) ⊃ ⊂ E2) if a structure index between τ (E1) and τ (E3) is available. If applying indices into a query is not feasible, a less favorable opportunity — reducing the length of the traversal needed into the document’s structure — is explored.

Simplification Rules The third phase reapplies most of the simplification rules of Phase 1, and introduces additional rules such as R10 to simplify new subexpressions pertaining to the index operator Iτ(E1).

R 10. (I τ(E1)(E2)∩σR(E1))⇒σR(I τ(E1)(E2)) if free(E1) holds.

An Optimization Example Now, we present the transformations performed on the example query given in the earlier part of the chapter to show how our deterministic transformation idea works.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

90 Che

Transformation Example The query retrieves all the paragraphs containing both “xpath” and “xlink” from any article. ( (σr=“xpath”(Paragraph) ⊂ Article) ∩ (σr=“xlink”(Paragraph) ⊂ Article) )

⇒ (∩ pushing down by R5) ( ( (σr=“xpath”(Paragraph) ⊂ Article) ∩ σr=“xlink”(Paragraph) ) ⊂ Article)

⇒ (∩ pushing down by R4) ( ( (σr=“xpath”(Paragraph) ∩ σr=“xlink”(Paragraph) ) ⊂ Article) ⊂ Article)

⇒ (redundant ⊂ deletion by R7) ( (σr=“xpath”(Paragraph) ∩ σr=“xlink”(Paragraph) ) ⊂ Article)

⇒ (∩ pushing down by R2) (σr=“xpath”(Paragraph ∩ σr=“xlink”(Paragraph) ) ⊂ Article)

⇒ (∩ pushing down by R3) (σr=“xpath”(σr=“xlink”(Paragraph ∩ Paragraph) ) ⊂ Article)

⇒ (∩ deletion by R6) (σr=“xpath”(σr=“xlink”(Paragraph) ) ⊂ Article)

⇒ (index introduction by R8) (I Paragraph(Article) ∩ σr=“xpath”(σr=“xlink”(Paragraph) ) )

⇒ (preceding structure index by R10) (σr=“xpath”(σr=“xlink”(IParagraph(Article) ) ) ) In the optimized format, the structure index IParagraph(Article) is applied, which is followed by two other selection operations.

Conclusions and Future Directions Equivalences are the basis of transformation-based query optimization. Although a lot of work has been done on a broad spectrum of issues related to XML databases, the work addressing equivalences for XML query optimization is rare. In this chapter, we elaborated on the equivalences that we developed for XML query optimization. Our strategy differentiates transformations from equiva-

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Transformation Based XML Query Optimization

91

lences. Query transformation for optimization is accomplished in our implementation by a rule system that consists of only deterministic transformation rules. Transformation rules are derived from equivalences. Alternatively, our equivalences may be directly used to enumerate alternative plans from which an optimal one is decided based on cost comparison. Currently, a lot of work is continuing on various aspects related to XML data management, e.g., structured-document modeling (Abiteboul, 1997; Özsu, 1997; Navarro, 1997; Lee, 1997; Yan, 1994; Conse, 1994; Chaudhuri, 1996; Morishima, 1997; Gonnet, 1992; Fernadez, 2000; Florescu, 1999; Shanmugasundaram, 1999; Bohannon, 2002; Klettke, 2000), XML data indexing (Chan, 2002; Grust, 2002; Li, 2001; Milo, 1999), and advanced algorithms for fast query processing (Fernandez, 1998; McHugh, 1997; Gottlob, 2002; Li, 2001; Guha, 2002; Chien, 2002; Srivastava, 2002). Transformation-based query optimization for XML databases is a rarely explored but fascinating realm. Our optimization is performed at the logical level — PAT expressions. One interesting extension to our current work is to complement the logical (plan) optimization by introducing a compact physical (cost-based) optimizer to achieve further better optimization result.

Acknowledgments The author would like to show great appreciation to his former colleagues at Fraunhofer-IPSI (formerly known as GMD-IPSI), Germany. This continuing research at the author’s current affiliation was originally initiated at GMD-IPSI in close collaboration with Prof. Karl Aberer, Dr. Klemens Böhm, and Prof. M. Tamer Özsu (during his visit to GMD-IPSI on his sabbatical leave).

References Abiteboul, S., Cluet, S., Christophides, V., Milo, T., Moerkotte, G. & Simeon, J. (1997). Querying Documents in Object Databases. International Journal on Digital Libraries, 1(1), 5-19. Bertino, E. (1994). A Survey of Indexing Techniques for Object-Oriented Database Management Systems. In J.C. Freytag, D. Maier & G. Vossen (Eds.), Query Processing for Advanced Database Systems. Morgan Kaufmann Publishers, 383-418.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

92 Che

Bohannon, P., Freire, J., Roy, P. & Simeon, J. (2002). From XML Schema to Relations: A Cost-Based Approach to XML Storage. Proceedings of the 18th International Conference on Data Engineering (ICDE’02), (pp. 64-73). Böhm, K., Aberer, K., Özsu, T. & Gayer, K. (1998). Query Optimization for Structured Documents Based on Knowledge on the Document Type Definition. Proceedings of IEEE International Forum on Research and Technology Advances in Digital Libraries (ADL’98), (pp. 196-205). Chan, C. Y., Felber, P., Garofalakis, M. & Rastogi, R. (2002). Efficient Filtering of XML Documents with XPath Expressions. Proceedings of International Conference on Data Engineering, (pp. 235-244). Chan, C. Y., Garofalakis, M. N. & Rastogi, R. (2002). RE-Tree: An Efficient Index Structure for Regular Expressions. Proceedings of VLDB 2002, (pp. 263-274). Chaudhuri, S. & Gravano, L. (1996). Optimizing Queries over Multimedia Repositories. Proceedings of SIGMOD’96, (pp. 91-102). Che, D. (2003). Implementation Issues of a Deterministic Transformation System for Structured Document Query Optimization. Proceedings of 2003 International Database Engineering & Application Symposium. Che et al. (2003). Query Processing and Optimization in Structured Document Database Systems. Manuscript in preparation for publication on the VLDB Journal. Chien, S., Vagena, Z., Zhang, D., Tsotras, V.J. & Zaniolo, C. (2002). Efficient Structural Joins on Indexed XML Documents. Proceedings of VLDB 2002, (pp. 263-274). Consens, M. & Milo, T. (1994). Optimizing Queries on Files. Proceedings of the 1994 ACM SIGMOD International Conference on Management of Data, (pp. 301-312). Cover, R. (2002). Online Resource for Markup Language Technologies. Retrieved from the WWW: http://xml.coverpages.org Deutsch, A., Fernandez, M. & Suciu, D. (1999). Storing Semistructured Data with STORED. Proceedings of ACM SIGMOD 1999, (pp. 431-442). Fernandez, M. F. & Suciu, D. (1998). Optimizing Regular Path Expressions Using Graph Schemas. Proceedings of the 14th International Conference on Data Engineering, (pp. 14-23). Florescu, D. & Kossmann, D. (1999). Storing and Querying XML Data Using an RDMBS. IEEE Data Engineering Bulletin, 22(3), 27-34. Gonnet, G. H., Baeza-Yates, R. A. & Snider, T. (1992). Information RetrievalData Structures and Algorithms. New Indices for Text: PAT trees and PAT arrays. New York: Prentice Hall. Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Transformation Based XML Query Optimization

93

Gottlob, G., Koch, C. & Pichler, R. (2002). Efficient Algorithms for Processing XPath Queries. Proceedings of VLDB 2002, (pp. 95-106). Grust, T. (2002). Accelerating XPath location steps. Proceedings of SIGMOD Conference 2002, (pp. 109-120). Guha, S., Jagadish, H. V., Koudas, N., Srivastava, D. & Yu, T. (2002). Approximate XML joins. Proceedings of the ACM SIGMOD Conference, (pp. 287-298). Klettke, M. & Meyer, H. (2000). XML and Object-Relational Database Systems - Enhancing Structural Mappings Based on Statistics. Proceedings of the International Workshop on the Web and Databases (WebDB), (pp. 151170). Lee, K., Lee, Y. K. & Berra, P. B. (1997). Management of Multi-Structured Hypermedia Documents: A Data Model, Query Language, and Indexing Scheme. Multimedia Tools and Applications, 4(2), 199-224. Li, Q. & Moon, B. (2001). Indexing and Querying XML Data for Regular Path Expressions. Proceedings of the 27th International Conference on Very Large Databases, (pp. 361-370). McHugh, J., Abiteboul, S., Goldman, R., Quass, D. & Widom, J. (1997). Lore: A Database Management System for Semistructured Data. SIGMOD Record, 26(3), 54-66. McHugh, J. & Widom, J. (1999). Query Optimization for XML. Proceedings of the 25th International Conference on Very Large Databases, 315-326. Milo, T. & Suciu, D. (1999). Index Structures for Path Expressions. Proceedings of ICDT 1999, (pp. 277-295). Morishima, A. & Kitagawa, H. (1997). A Data Modeling and Query Processing Scheme for Integration of Structured Document Repositories and Relational Databases. Proceedings of the Fifth International Conference on Database Systems for Advanced Applications (DASFAA 1997), (pp. 145-154). Navarro, G. & Baeza-Yates, R. (1997). Proximal Nodes: A Model to Query Document Databases by Content and Structure. ACM Transaction on Information Systems, 15(4), 400-435. Özsu, M. T., Iglinski, P., Szafron, D. & El-Medani, S. (1997). An ObjectOriented SGML/HiTime Compliant Multimedia Database Management System. Proceedings of Fifth ACM International Multimedia Conference (ACM Multimedia’97), (pp. 239-249). Salminen, A. & Tompa, F. W. (1994). PAT Expressions: An Algebra for Text Search. Acta Linguistica Hungarica, 41(1), 277-306.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

94 Che

Shanmugasundaram, J., Tufte, K., Zhang, C., He, G., DeWitt, D.J. & Naughton J.F. (1999). Relational Databases for Querying XML Documents: Limitations and Opportunities. Proceedings of VLDB, (pp. 302-314). Srivastava, D., Al-Khalifa, S., Jagadish, H. V., Koudas, N., Patel, J. M. & Wu, Y. (2002). Structural Joins: A Primitive for Efficient XML Query Pattern Matching. Proceedings of ICDE’02, (pp. 141-150). Yan, T. W. & Annevelink, J. (1994). Integrating a Structured-Text Retrieval System with an Object-Oriented Database System. Proceedings of the 20th VLDB Conference, (pp. 740-749).

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Specifying Coherent Refactoring of Software Artefacts 95

Chapter V

Specifying Coherent Refactoring of Software Artefacts with Distributed Graph Transformations Paolo Bottoni, University of Rome “La Sapienza”, Italy Francesco Parisi-Presicce, University of Rome “La Sapienza”, Italy and George Mason University, USA Gabriele Taentzer, Technical University of Berlin, Germany

Abstract This chapter discusses the use of Graph Transformations for refactoring. Refactoring changes the internal structure of a software system, while preserving its behavior. Even though the input/output view of a system’s behavior does not change, refactoring can have several consequences for the computing process, as expressed for instance by the sequence of method

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

96 Bottoni, Parisi-Presicce, and Taentzer

calls or by state changes of an object or an activity. Such modifications must be reflected in the system model, generally expressed through UML diagrams. We propose a formal approach, based on distributed graph transformation, to the coordinated evolution of code and model, as effect of refactorings. The approach can be integrated into existing refactoring tools. Due to its formal background, it makes it possible to reason about the behavior preservation of each specified refactoring.

Introduction Software is subject to changes and a piece of software may need changes for several reasons. One such reason is the introduction of new requirements that cause the need for design changes. The introduction of a new requirement can be a consequence of either the iterative development process chosen for the project that constructs the system incrementally, or the fact that the requirement was overlooked in the initial specification and design of the system. As a simple example, consider an application developed around a single specific algorithm. If a new algorithm to perform the same calculations (graph layout, for example) becomes available, it may be useful to modify the application to add the option of using the new algorithm. Object-oriented programming has made many changes easy to implement, often just by adding new classes, as opposed to more traditional approaches requiring many modifications. But adding classes may not be sufficient. Even in the simple example above, the application must evolve by means other than class addition. If the designer has not foreseen the possibility of alternatives for the algorithm, the class with the original algorithm would probably need to be “split” into algorithm-specific elements and general ones, the latter to be “moved” to a new class that will then provide the means to choose between the two algorithms, placed in separate components. Another reason for wanting to modify an object-oriented program is to be able to reuse (part of) it. As an example, consider the case of two teams developing two class libraries independently. The two libraries may contain different classes implementing the same basic objects (windows, lists) or the same operations to manipulate them with different names. In order to integrate the libraries, it is best to remove these inconsistencies, by changing one library to use the basic classes or the operation names of the other one. Simple modifications such as the change of an operation name are not easy to implement, as they require searches for the procedures that can invoke them or for the other operations that they would override with the new name.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Specifying Coherent Refactoring of Software Artefacts 97

The two modifications above are examples of what is called refactoring of code. Actually, software applications consist of code as well as specifications and documentation, these being valuable company assets. The diffusion of UML in large programming organizations is creating a wealth of documentation as collections of UML diagrams, inspected by developers and designers, or used to communicate with the project shareholders. Documentation repositories facilitate software reuse and design pattern identification. Ideally, refinements or adaptations should be traceable to the original material, and code transformations reflected back to the documentation. When refactoring software, the internal structure of a software system changes, while preserving its behavior, in terms of input/output relations. However, this can have consequences for the computing process, as expressed for instance by the sequence of method calls or by state changes of an object or an activity. Several types of refactoring are now known and widely used (Fowler, 1999). It is demonstrable that they preserve program behavior, and it is usually known in which way they modify static specifications, i.e., class diagrams. Refactoring can also occur in design, involving modifications of interaction, state machine, or activity diagrams. However, not all transformations induced by refactoring are actually mapped back to the relevant documentation. Since refactoring is usually performed at the source code level, it may be difficult to maintain consistency between the code and its specification – expressed for example with UML diagrams – which usually refers to the code original version. In particular, one has to identify the relevant diagrams when modifying a piece of software. Two strategies can be adopted to preserve consistency: either restoring the specification after a chosen set of changes, or coherently defining the effects of each refactoring on the different artefacts of a software project. While changes in structural specifications are notationally equivalent to lexical transformations on source code, transformations of behavioral specifications may be significantly more intricated. We discuss an approach to the problem of maintaining consistency between source code and diagrams, both structural and behavioral, using the formal framework of graph transformation. In particular, Abstract Syntax Trees describe the source code, while UML diagrams are represented as graphs, conforming to the abstract syntax presented in the UML metamodel. The UML diagrams and the code are hence seen as different views on a software system, so that consistency between the views and the code is preserved by modeling coherent refactorings as graph transformations distributed on several graphs. Complex refactorings, as well as checking of complex preconditions, are decomposed into collections of distributed transformations whose application is managed by control expressions in appropriate transformation units.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

98 Bottoni, Parisi-Presicce, and Taentzer

In the rest of this introduction, we set the background for our work, by introducing the refactorings used in the motivating example and reviewing some approaches to refactoring and software evolution via graph rewriting, and illustrating motivations for the coherent refactoring of code and models. Background notions on graph transformation are then given. Next, the problem of maintaining consistency among specification and code is reformulated as the definition of suitable distributed graph transformations, and our approach is illustrated with two important refactorings. The next section discusses the principles under which one can establish correspondences between abstract representations of the code and of the model. Section 6 discusses forms of behavior preservation and sketches out how formal results for graph transformation help in reasoning about it. Conclusions are then given.

Selected Refactorings While a complete coverage of refactorings is beyond the scope of this chapter, we illustrate here the basic ones used in the example shown later in the chapter. A rich set of refactorings, both primitive and complex, is given in Fowler (1999). In general, all refactorings require that no name clashes be generated. For instance, if a new method is introduced or has its name changed, a check is needed to ensure that no method with the same signature is already present in the inheritance hierarchy. Here, we only mention additional checks other than checks for name clashes. RenameVariable and RenameMethod change the name of a variable or method to highlight structural or behavioral analogies in a set of classes: all references to these features must be renamed. RenameMethod is one of the constituents of the ChangeMethodSignature refactoring, with sub-refactorings such as ChangeReturnType and ChangeParameterType, or addition and removal of parameters. The EncapsulateVariable refactoring hides information by making a variable private and providing public getter and setter methods for accessing and updating it. All direct references to the variable are replaced by dynamic calls to these methods. InsertClass expands the inheritance hierarchy by introducing a new class B between a class A and its original superclass C. B becomes the superclass for A and has C as its superclass. PullUpMethod allows replicated methods to be moved from subclasses into a common superclass. To apply this refactoring, the body of the pulled up method must not refer to any variable only defined in subclasses. ExtractMethod derives from the composition of several primitive refactorings, but it is so widespread that it can be considered as a single one. It removes a block of code from a method and uses it to create a new method, substituting the code in the original method by a call to the new one. Beyond avoidance of name clashes, preconditions for it require that all variables

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Specifying Coherent Refactoring of Software Artefacts 99

which are accessed by the extracted code and have a local scope be passed as parameters, and that the removed code form a block, i.e., it has a single entry point and a single exit point.

Related Work Several tools have been developed to assist refactoring. Some are packaged as stand-alone executables, while others integrate refactorings into a development environment. Many tools refer directly and exclusively to a specific language, for example C# Refactory (http://www.xtreme-simplicity.net/) for C#, or CoreGuide6.0 (http://www.omnicore.com) for Java. Xrefactory (http:// www.xref-tech.com) assists in modifying code in C and Java. All of these provide a variety of refactorings, typically renamings and method extraction. However, none of them mentions diagrams and the effects on other views of the system, including documentation. The class diagram, referred to as “the model,” is instead considered in objectiF (http: //www.microtool.de/objectiF), which, in addition to supporting a variety of languages, allows transformations of both the code and the class model, with changes propagated automatically to both views. Other kinds of diagrams, especially those describing behavioral aspects of the system, are not refactored. Eclipse (http://www.eclipse.org) integrates system-wide changes of code with several refactoring actions (such as rename, move, push down, pull up, extract). Class diagrams are implicitly refactored, too. Finally, JRefactory (http://jrefactory.sourceforge.net) supports 15 refactorings including: pushing up/down, methods/fields, and extract method/interface. The only diagrams mentioned are class diagrams which are reverse engineered from the .java files. Reverse engineering is present in Fujaba (Niere et al., 2001), where the user can reconstruct the model after a chosen set of changes of the code. A more efficient option would be to define the effects of a refactoring on the different parts of the model. This is more easily realized on structural models, where transformations on such diagrams are notationally equivalent to the lexical transformation on the source code, than on behavioral specifications. Modern refactoring tools, however, work on abstract representations of the code, rather than on the code itself, typically in the form of an Abstract Syntax Tree (AST), following Roberts’ (1999) line. Refactorings are also defined on model diagrams. Sunyé et al. (2001) illustrate refactoring of statecharts, typically to extract a set of states to be part of a composite state. Transformations of concrete diagrams are specified by pre and post-conditions, written as OCL constraints. Metz et al. (2002) consider the UML metamodel to propose extensions to use case models, which would allow Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

100 Bottoni, Parisi-Presicce, and Taentzer

significant refactorings of such models and avoid improper current uses. These papers, however, do not consider the integration with possible source code related to these models. Current class diagram editors do not extend changes to all other related diagrams, limiting their “automation” to the source code, with the result that direct intervention is needed to restore consistency among possibly various UML diagrams representing the same subsystem. We adopt UML metamodel instances and draw a correspondence between these and abstract syntax trees representing code. Hence, a common graph-based formalism can be used as basis for an integrated management of refactoring both the code and the model in an integrated way. Graph rewriting has been introduced as a basis for formalising refactoring in work by Mens, alone (2000, 2001) and with others (2002). In these papers, a nonstandard graph representation of code is used, so that the availability of AST representations is not exploited. Moreover, integrated refactoring of model and code by graph transformation has not been considered up to now.

Outline of the Approach Our approach aims at precisely specifying integrated refactoring of model and code, while maintaining the consistency between them achieved during the development of a software project. Indeed, software development follows typical patterns. Authors can start some modeling activity, update and refine their models, start writing code, and modify it. If a CASE tool is available, code can be generated from models, e.g., skeletons of classes or collaborations for some pattern. Moreover, developers can decide to compile the code, reverse-engineer, and update some parts of the model from the generated abstract syntax tree. As a result of this first phase of development, parts of the code and the model have had an independent evolution, so that they can be consistent or not, while other parts have evolved in a coordinated way. The model and the code can then be checked to identify parts consistent with one another, either by circumstance or by construction. To this end, we assume that at any time a pair of graphs exists providing abstract representations of the code and of the model. The code representation is in the form of an AST, while the model is given through UML diagrams, and constitutes an instance of the UML meta-model. The checking phase produces an interface graph IG and a pair of graph morphisms from IG to the AST and UML graphs respectively. The morphisms establish the correspondence between code and models. The relations between these graphs must be managed by the CASE tool. The subsequent phases of refactoring can then be triggered on the code or the model. For any

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Specifying Coherent Refactoring of Software Artefacts 101

refactoring supported by the CASE tool in which the user acts on the code or on the model, the corresponding modifications on the other graph must be enforced. After refactoring, the cycle can start again with new developments, and so on. While refactoring tools work on both abstract and concrete representations of code, they are usually restricted to the manipulation of structural aspects of the model, namely class diagrams. Although this is intuitively justifiable by the stated assumption that refactoring does not affect the behavior of systems, the combination of refactoring with other forms of code evolution can lead to inconsistencies between the model and the code. This could be avoided by a careful consideration of what a refactoring involves, as shown in the following two subsections.

Modification of Collaborations In the refactoring ExtractMethod, a block of code blk is removed from a method morig in a class C, a new method mnew is created, block blk is inserted as the body of mnew and a call to mnew replaces the original code in morig. If the execution of morig is represented in a collaboration diagram (Figure 1a), but the refactoring tool cannot manipulate such a diagram, then the activation of mnew cannot be represented. Suppose that mnew is subsequently moved from C to a coupled class D, and finally modified so that it performs some new activities, involving a call to a method meth in a third class E. The designer can now assume that the last addition of the call to meth is significant enough to show in the collaboration. But, as we have lost the consistency between the model and the code, a simple-minded addition of the call to meth as a descendant of the call to morig would result in the inconsistent model of Figure 1b. This can be avoided if all the steps in this process are reflected in the collaboration: the extraction of blk to mnew would be reflected by a self-call stemming from the activation for morig, the movement of mnew to class D would transform the self-activation to a call to this new class and the consequent activation of mnew in it, so that the call of meth would now be in the correct position, as in Figure 1c).

Modification of Activity Graphs Activity graphs are special types of state machines used for describing complex processes, involving several classifiers, where the state evolution of the involved elements is modeled. Suppose that an action is performed to the effect of setting a field variable to some value, say x = 15. Hence, a state s appears in the model indicating that an assignment has to occur at that time. If the EncapsulateVariable refactoring is subsequently applied to the variable x, the code x = 15 is replaced

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

102 Bottoni, Parisi-Presicce, and Taentzer

Figure 1. (a) Original sequence diagram; (b) inconsistent situation from not reflecting the extraction of mnew; (c) desired consistent situation :C morig

:C

:E

morig

:C

:D

:E

morig meth

mnew meth

a)

b)

c)

by setX(15). The state in the activity diagram now becomes a CallState s’. (Compare similar modifications in activity diagrams in Figures 2c and 3c.)

Example of Refactoring We illustrate the refactorings with a typical case in software development. Let us consider the design of an intelligent Audio player, able to dynamically identify a MusicSource, for example, on the basis of some preferences, and to obtain from this source a piece of Music. It then sets up an environment, in which it causes the received piece to play itself. A first version produces the following, strongly coupled, set of classes. Moreover, the player must expose its preferences in a public variable for the music source to select some piece of music. This prevents reuse of the Audio class in a non-controlled environment. Figure 2 shows components of the UML model: class (2a), sequence (2b) and two activity diagrams (2c). class Audio { protected MusicSource ms; private Environment env; public MusicDescription preferences; protected findMusicSource() { // lookup for a music source } protected void playMusic() { ms = findMusicSource(); Music toPlay = ms.provideMusic(this); // code to set the playing environment env toPlay.play(env);

continued at top of page 103 Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Specifying Coherent Refactoring of Software Artefacts 103 } } class Music { void play(Environment env) { // code to play in the environment env } } class MusicSource { public Music provideMusic(Audio requester) { MusicDescription desc = requester.preferences; // code to retrieve music according to desc and sending it back as result } } class Environment { // fields and methods to define a playing environment }

With a view to the possibility of reuse, the programmer decides to protect the preferences, by applying the EncapsulateVariable refactoring. After this first step, the affected code looks as follows, where the parts in bold mark the changed elements. The new situation is reflected in the model diagrams of Figure 3. class Audio { protected MusicSource ms; private Environment env; private MusicDescription preferences; protected findMusicSource() { // same implementation as before } protected void playMusic() { // same implementation as before } public MusicDescription getPreferences() { return preferences; } public void setPreferences(MusicDescription desc) { preferences = desc; } } class MusicSource { public Music provideMusic(Audio requester) { MusicDescription desc = requester.getPreferences(); // same code using desc as before } }

The code above presents several possibilities for refactorings, allowing the introduction of an abstract notion of player able to retrieve a content source, interrogate it to obtain some content and set an environment for it to play. Concrete players will differ for the type of source they have to retrieve and the way in which they define the environment. On the other hand, content sources must have a generic ability to accept a player and sending the appropriate content to it, while the different forms of content will have specific realizations of the play

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

104 Bottoni, Parisi-Presicce, and Taentzer

method. To this end, a first step is to extract the code for playing in an environment from playMusic to a setEnvironment method. Method playMusic is then renamed to playContent, while findMusicSource is renamed to findSource and the variable musicSource to source, while in class Music, provideMusic is renamed to provideContent. Refactorings are then performed to introduce new classes and interfaces in an existing hierarchy, by creating and inserting the abstract class AbstractPlayer and the interfaces Content and ContentSource. We can now pull up methods and variables from Audio to AbstractPlayer. Finally, all return and parameter types referring to the concrete classes are now changed to the newly inserted types. The resulting code is reported below. Again, parts in bold show the modified parts with respect to the previous version. The reader can reconstruct the UML diagrams according to these modifications. abstract class AbstractPlayer { protected ContentSource source; private Description preferences; private Environment env; protected abstract ContentSource findSource(); protected abstract void setEnvironment(); protected void playContent() { source = findSource(); Content toPlay = source.provideContent(this); setEnvironment(); toPlay.play(env); } Description getPreferences() { return preferences; } void setPreferences(Description desc) { preferences = desc; } } class Audio extends AbstractPlayer { ContentSource findSource() { // code from findMusicSource } void setEnvironment() { // previous code used to set env } } interface ContentSource { Content provideContent(AbstractPlayer requester); } class MusicSource implements ContentSource { Content provideContent(AbstractPlayer requester) { Description desc = requester.getPreferences(); // previous code from provideMusic exploiting desc; } } interface Content { void play(Environment env); } class Music implements Content { // same implementation as before } class Environment { // same implementation as before }

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Specifying Coherent Refactoring of Software Artefacts 105

Figure 2. Components of the UML model for the first version of code – (a) class diagram; (b) sequence diagram; (c) activity diagrams Audio + MusicDescription preferences # MusicSource findMusicSource() # void playMusic()

# ms

MusicSource + Music provideMusic(Audio requester)

Music # play(Environment env)

- env Environment

(a) a:Audio

ms:MusicSource

toPlay: Music

playMusic() findMusicSource()

ms provideMusic(a) toPlay play(env)

(b) Audio::playMusic() a:Audio

ms:MusicSource

Audio::provideMusic()

toPlay:Music

a:Audio findMusicSource()

ms:MusicSource

provideMusic(a)

start retrieval ms: set environment

toPlay: Music [found]

a.preferences retrieve music

play(env)

(c)

The Formal Background The algebraic approach to graph transformation (Corradini et al., 1997) is the formal basis for our work, as graphs are natural means to represent code and Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

106 Bottoni, Parisi-Presicce, and Taentzer

model structures. To handle model and code-related graphs in a separate but consistent way, we apply concepts of distributed graph transformation. Finally, the concept of transformation units is used to obtain a global control on structured graph manipulations, useful to specify complex refactorings.

Graph Transformation Graphs are often used as abstract representations of code and diagrams, e.g., UML diagrams. Formally, a graph consists of a set of vertices V and a set of edges E such that each edge e in E has a source and a target vertex s(e) and t(e) in V, respectively. Each vertex and edge may be attributed by some data value or object, expressed by elements of an algebra on some algebraic signature Σ. Here, we consider typed attributed graphs. For graph manipulation, we adopt the double-pushout approach to graph transformation, DPO (Corradini et al., 1997),

Figure 3. UML diagrams after variable encapsulation – (a) class diagram; (b) sequence diagram; (c) activity diagram

Audio - MusicDescription preferences # MusicSource findMusicSource() # void playMusic() + MusicDescription getPreferences() + void setPreferences(MusicDescription desc)

# ms

- env Environment

MusicSource + Music provideMusic(Audio requester)

Music # play(Environment env)

(a) Audio::provideMusic() a:Audio

ms:MusicSource

toPlay: Music

a:Audio

playMusic()

ms:MusicSource

findMusicSource()

start retrieval ms provideMusic(a) getPreferences()

getPreferences() desc

retrieve music

toPlay play(env)

(b)

(c)

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Specifying Coherent Refactoring of Software Artefacts 107

based on category theory. Using typed graphs, structural aspects appear at two levels: the type level (modeled by a type graph T) and the instance level (modeled by an instance graph G). G is correctly typed if it can be mapped in a structurepreserving manner to T, formally expressed by a graph homomorphism. A graph rule r: L → R is a pair of T-typed instance graphs L, R such that L ∪ R is defined, i.e., graph objects occurring in both L and R have the same type and attributes and, if they have the same edges, will also have the same source and target vertices. The left-hand side L represents the modification pre-conditions, while the right-hand side R shows its effect. Vertex identity is expressed via names, while edge identity is inferred from the identity of the connected vertices. Additionally, graph rules comprise attribute computations where left-hand sides may contain constants or variables of set X, while right-hand sides capture the proper computations, denoted as elements of an algebraic term TΣ (X). A rule may also contain a set of negative application conditions (NAC), expressing graph parts that must not exist for the rule to be applicable. NACs are finite sets of graphs NAC={N i| L ⊆ N i, i ≥ 0 }, specifying a conjunction of basic conditions, and can refer to values of attributes (Fischer et al., 1999). For a rule to be applicable, none of the prohibited graph parts Ni – L present in a NAC may occur in the host graph G in a way compatible with a rule match m. A match is an injective graph homomorphism m: L ∪ R → G ∪ H, such that m(L) ⊆ G and m(R) ⊆ H, i.e., the left-hand side of the rule is embedded into G and the righthand side into H. In this chapter we use dotted lines to denote NACs. Nonconnected NACs denote different negative application conditions (see Figure 14 for an example). A graph transformation from a graph G to a graph H, p(m): G ⇒ H, is given by a rule r and a match m with m(L – R) = G – H and m(R – L) = H – G, i.e., precisely that part of G is deleted which is matched by graph objects of L not belonging to R and symmetrically, that part of H is added which is matched by new graph objects in R. Operationally, the application of a graph rule is performed as follows: First, find an occurrence of L in graph G. Second, remove all the vertices and edges from G matched by L – R. Make sure that the remaining structure D= G–m(L–R) is still a proper graph, i.e., no edge is left which dangles because its source or target vertex has been deleted. In this case, the dangling condition is violated and the application of the rule at match m is not possible. Third, glue D with R–L to obtain graph H. A typed graph transformation system GTS=(T,I,R) consists of a type graph T and a finite set R of graph rules with all left and right-hand sides typed over T. GTS defines formally the set of all possible graphs by Graphs(GTS)={G|I ⇒ *R G} where G ⇒*R H ≡ G ⇒r1(m1) H1 ... ⇒rn(mn) Hn = H with r1, ..., rn in R and n >= 0. It follows from the theory that each graph G is correctly typed.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

108 Bottoni, Parisi-Presicce, and Taentzer

Distributed Graph Transformation Distributed graph transformations (Fischer et al., 1999) are structured at two abstraction levels: the network and the object level. The network level contains the description of a system’s architecture by a network graph, and its dynamic reconfiguration by network rules. At the object level, graph transformations manipulate local object structures. To describe a synchronized manipulation on distributed graphs, a combination of graph transformations on both levels is needed. In a distributed graph each network vertex is refined by a local graph. Network edges are refined by graph homomorphisms on local graphs, which describe how the local graphs are interconnected. Each local graph may be typed differently, only restricted by the fact that an interface type graph is fully mapped to all connected local type graphs. We use distributed graphs where the network graphs consist of three vertices: for the model, for the code and for their interface. Furthermore, two network edges are needed, starting from the interface vertex and going to the model and code vertices, respectively. The corresponding refinement graphs are called model graph, code graph and interface graph. The interface graph holds exactly that subgraph which describes the correspondences between the other two local graphs. A distributed graph rule r is a network rule n — a normal graph rule — together with a set S of local rules — graph rules on local graphs — for all those network vertices which are preserved. Each preserved network edge guarantees a compatibility between the corresponding local rules. The rules must also be consistent with common attribute values. In this chapter, network rules are always identities, as the network is not changing. Two local rules, on the model and the code graph, are synchronized by applying a common subrule on their interface graph. We show here only local rules with grey levels indicating subrule parts. We introduce two operators to assemble a distributed rule from local ones: asOftenAsPossible means to apply a local rule as often as possible at different matches in parallel, while || just denotes the distributed application of rules.

Transformation Units Transformation units (Kreowski et al., 1997) are a general concept to control rule application, by control conditions specified by expressions over rules. We use it in the context of distributed graph transformation, in which a transformation unit consists of a set of rules and a control condition over C describing how rules can be applied. Typically, C contains expressions on sequential application of rules, as well as conditions and loops, e.g., by applying a rule as long as possible. We relate rule expressions to graph rules by giving names to rules and passing parameters to them, to be matched against specific attributes of some vertex. By Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Specifying Coherent Refactoring of Software Artefacts 109

this mechanism, we can restrict the application of rules to those elements which carry an actual reference to the code to be refactored. To this end, the rules presented in the transformation units are meant as rule schemes to be instantiated to actual rules, assigning the parameters as values of the indicated attributes.

Refactoring by Graph Transformation We present the general setting of refactoring by graph transformation and analyse a sample refactoring which involves transformation of the code and more than one UML diagram. Furthermore, we show the use of transformation units over distributed graph transformations to enforce synchronization and atomicity of the transformations in different diagrams.

Graph Representation of Diagrams and Code The abstract representations of code and UML models are given in the form of graphs, obeying the constraints imposed by a type graph. For the code, we refer to the JavaML definition of an abstract syntax for Java (Badros, 2000), and we consider the type graph provided by its DTD. Indeed, any JavaML document is structured as a tree, i.e., a special kind of graph where an XML element is represented by a typed vertex and its attributes by vertex attributes. The graph edges show the sub-element relation and are untyped and not attributed. We call this graph the code graph. For UML (OMG, 2002), the abstract syntax of the UML metamodel provides the type graph to build an abstract representation of the diagram that we call the model graph. As an example, Figure 4 shows the code graph for class Audio. For space reasons, we omit the representation of fields ms and env and method findMusicSource. Figure 5 presents the model graph for the class diagram of Figure 2a (without dependencies). Only the important fields of model elements are shown. Details of model elements occurring in more than one figure are shown only in one. Vertices that would be directly connected to a class vertex in the code graph, appear in the model graph as feature elements for which the class is an owner. Figures 6 and 7 present the components of the model graph for the sequence and activity diagrams of Figure 2. The model graphs, though presented separately, are different views of one large graph representing the whole model. Indeed, behavioral diagrams are associated with model elements which own or contribute to the model’s behavior. As an example, object m1:Method for playMusic appears in Figure 5 as a behavioral

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

110 Bottoni, Parisi-Presicce, and Taentzer

Figure 4. A part of the code graph for the first version of the code of class Audio : java-class-file

: java-source-program

c1‘: class name = „Audio“

a1‘: field name= „preferences“ visibility = public

: type name= „MusicDescription“

: type name= „void“ primitive = true

m1‘: method name= „playMusic“ id = „Audio:mth1“ visibility = protected 3

: block 2 : local-variable name = „toPlay“ id = „Audio:var1“

: assignment-expr op = „=„

1

: lvalue

c6‘: send message= „findMusicSource“

c7‘: send message= „provideMusic“

: type name= „Music“ : target

: arguments

c8‘: send message= „play“

a2‘: var-set name= „ms“

a2‘: var-ref name= „ms“ o8‘: this

: target i1‘: arguments

o5‘: var-ref name= „env“

o6‘: var-ref name= „toPlay“ idref=„Audio:var1“

Figure 5. Abstract graph for the class diagram of Figure 2a type c1:Class owner name= „Audio“ owner owner

type c2:Class feature a1:Attribute name= „preferences“ name= „MusicDescription“ visibility= #public

feature m1:Method name= „playMusic“ m2:Method visibility=#protected feature name= „findMusicSource“ specification visibility=#protected o1:Operation :Association o2:Operation specification

:AssociationEnd :AssociationEnd :Association

a3:AssociationEnd c4:Class name= „env“ name= „Environment“ visibility=#private c3:Class type name= „MusicSource“ type type c5:Class owner o3:Operation name= „Music“ a2:AssociationEnd owner feature specification name= „ms“ o4:Operation m3:Method visibility=#protected specification feature name= „provideMusic“ visibility=#public :Method :Parameter p1:Parameter name= „play“ name= „env“ name= „requester“ visibility=#protected kind=#in :Parameter kind=#in kind=#return

feature of class Audio, and in Figure 7 as the element whose behavior is defined by the component of the model graph for activity diagrams. Conversely, object o2:operation appears as the specification for playMusic in Figure 5, and as the operation for a CallOperationAction object in Figure 6. In the next section, transformation units for distributed transformations are used to

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Specifying Coherent Refactoring of Software Artefacts 111

Figure 6. Abstract graph for the sequence diagram of Figure 2b :CollaborationInstanceSet :InteractionInstanceSet

activator :Stimulus activator :Procedure activator

:Stimulus

:Procedure

:CallOperationAction

:Stimulus

predecessor o4:Operation name= „play“

:Procedure

c7:CallOperationAction o2:Operation name= „playMusic“ receiver o8: Object name= „a“ sender receiver

predecessor sender

c8:CallOperationAction o3:Operation receiver name= „provideMusic“

c1: Class name= „Audio“

destination :DataFlow source c3: Class :OutputPin name= „MusicSource“

o7: Object name= „m“

:Stimulus

o1:Operation name= „findMusicSource“

:Procedure

c6:CallOperationAction

i1:InputPin

sender

:ReadLinkObjectEndAction :LinkEnd

receiver o6: Object name= „toPlay“ c5: Class name= „Music“ o5: Object name= „env“

a3:AssociationEnd

Figure 7. Abstract graph for the activity diagram of Figure 2c for executing playMusic m1: Method name= „playMusic“

behavior :ActivityGraph :Partition :Transition contents :CallState source

:Partition target contents :SubActivityState source

:Procedure

:Partition ag:ActivityGraph

:Procedure :Transition

c6:CallOperationAction

c7:CallOperationAction

o1:Operation name= „findMusicSource“

o3:Operation name= „provideMusic“

target

contents :CallState :Procedure

contents :ActionState :Action

target source

c8:CallOperationAction o4:Operation name= „play“

:Transition

:Procedure

modify code and model graphs in a coordinated way, based on correspondences (further discussed in Section 5) between elements in the two graphs, indicated by similar names (e.g., “c1’ ” in Figure 4 and “c1” in Figure 5).

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

112 Bottoni, Parisi-Presicce, and Taentzer

Encapsulation of Variables The preconditions for EncapsulateVariable require that no method exists in the hierarchy with the same signature as the setter and getter methods to be created. Therefore, we use NACs on rules transforming the code and model graphs. Neither graph expresses the hierarchy directly, so that the transitive closure of inheritance must be evaluated by inserting edges of type gen from a class to its ancestors, before checking the precondition. This can be easily done by rules not shown here, but similar rules are presented in Bottoni et al. (2003). Figure 8. Rule for variable encapsulation in the code graph encapsulate_variable_code(in String cname, in String varname):

c‘: class name = cname

a‘: field name= varname visibility = x

c‘: class name = cname

: return

: var-ref name= „varname“

: type name = void

m1‘: method name= „set“+varname id = cname + „:mth“ + d visibility = x

a‘: field name= varname visibility = private

1: type name = t

1: type name = t

: type name = t

m2‘: method name= „get“+varname id = cname + „:mth“ + c visibility = x

: block : asignment-expr op= „=“

: formal_arguments : lvalue

: type name = t

p1‘: formal_argument name = „arg“ id = cname + „:frm“ + d

: var-ref name= „arg“ id = cname + „:frm“ +d

a1‘: var-set name= „varname“

Code graph transformation is now specified by rule encapsulate_variable_code in Figure 8, where cname identifies the class to be modified, and varname the variable to be encapsulated. This rule is complemented by NACs, two of which are shown in Figure 9, reporting also the nodes in the LHS of Figure 8. These two NACs check the absence of methods with the same signature in the class, while the others check the absence in the whole hierarchy. All accesses to the variable are substituted by method calls. Hence, Figure 10 shows the rule replacing direct access to the variable with a call of the getter, while Figure 11 shows the rule for value updates.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Specifying Coherent Refactoring of Software Artefacts 113

Figure 9. Two NACs for the rule in Figure 8, to check that no method exists with the same signature as the inserted setter and getter methods 1: type name= t c‘: class name = cname

c‘: class name = cname

: formal_argument name = „arg“ a‘: field name= varname

a‘: field name= varname

1: type

: type name = t

: formal_arguments

: method name= „get“+varname

: type name= void

: method id= „set“+varname

Rules operate locally on the components of the model graph for the diagrams above. Figure 12 shows the encapsulate_variable_model rule acting on the class diagram. Negative application conditions analogous to those for the code graphs are also used, guaranteeing a check of the overall consistency of the representations. Consequently, we also need to compute the transitive closure of the inheritance relation for model graphs (not shown). Rules encapsulate_variable_model and encapsulate_variable_code are applied in parallel along their common subrule shown in grey.

Figure 10. Rule to replace accesses to varname in cname with calls to the getter field-access(in String cname, in String varname): 1: var-ref id = i

1: var-ref id = i : target ac: field-access field= varname

m‘: method

c‘: class name= cname

a‘: field name= varname

m‘: method

c2‘: send name= „get“+varname

c‘: class name= cname

a‘: field name= varname

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

114 Bottoni, Parisi-Presicce, and Taentzer

Figure 11. Rule to replace updates of varname in cname with calls to the setter field-set(in String cname, in String varname): 1: lvalue

1: lvalue 2: expr-elems ac: field-set field= varname

: arguments

c3‘: send name= „set“+varname

m‘: method m‘: method

c‘: class name= cname

a‘: field name= varname

c‘: class name= cname

2: expr-elems a‘: field name= varname

The effect on activity diagrams is shown by the rule getEncVarInActivity in Figure 13, replacing variable access with a call of a getter. For simplicity, we omit all input and output pins. We do not present the whole distributed rule, but only the local rule acting on the model graph. If the variable replacement in the model graph corresponds to some variable replacement in the code graph, all possible rule applications of getEncVarInActivity have to be applied synchronously with code rule field-access along their common subrule, which is shown in grey. An analogous rule exists for replacing variable updates with calls of the setter method.

Figure 12. LHS and RHS of the rule for variable encapsulation on the class diagram component of the model graph encapsulate_variable_model(in String cname, in String varname): c: Class name = cname owner

c: Class name = cname owner

op2: Operation feature

owner owner

m2: Method name= „set“+varname visibility = x

feature a: Attribute name= varname visibility = x

feature a: Attribute name= varname visibility = private op1: Operation

type 1: Classifier name = t

feature m1: Method name= „get“+varname visibility = x

type 1: Classifier name = t type

type

p1: Parameter name = „arg“ : Parameter kind = #return

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Specifying Coherent Refactoring of Software Artefacts 115

Finally, we consider the required modifications for sequence diagrams, for the case of variable encapsulation. Since sequence diagrams do not show read and write actions on attributes, the encapsulation does not directly cause a refactoring. In order to maintain a consistent model, the user has to specify if and where the refactoring should be represented for this part of the model. In particular, whenever a method m is called, in which the encapsulated variable is used, it is necessary to introduce a Stimulus s’ to call the relevant setter or getter method. From the ordering of subtrees in the code graph of m, one can identify the stimulus s for which s’ is the successor (or predecessor) in the new activation sequence, and pass it as a parameter to the rule. For space reasons, we omit the representation of the relative rule getEncVarInInteraction. The rules in Figures 10, 11, and 13 must be applied at all possible instances of their LHS in the distributed graphs. There may be several such instances, and we want to apply a transformation in a transactional way, i.e., the overall application is possible only if corresponding parts can be coherently transformed. Hence, transition units specify some form of control on the application. In particular, the control construct asOftenAsPossible states that a local rule must be applied in parallel on all (non-conflicting) instances of the antecedent. Contextual elements can be shared by different instances, but no overlapping is possible on elements removed or transformed by the rule. Moreover, the construct || indicates the distributed application of two or more rules.

Figure 13. Rule for modifying variable access in activity diagrams getEncVarInActivity(in String cname, in String varname): 1:CallState 1:ActionState 2:Procedure 2:Procedure c2:CallOperationAction ac:ReadAttributeAction m1: Method

a: Attribute name= varname

c: Class name = cname m1: Method op: Operation name= „get“+varname

op: Operation name= „get“+varname

c: Class name = cname

a: Attribute name= varname

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

116 Bottoni, Parisi-Presicce, and Taentzer

To sum up, EncapsulateVariable is expressed by a transformation unit as follows: EncapsulateVariable(in String cname, in String varname):= encapsulate_variable_code(cname, varname) || encapsulate_variable_model(cname,varname); asLongAsPossible field_access(cname,varname) || (asOftenAsPossible getEncVarInActivity(cname,varname) end) end; asLongAsPossible field_set(cname,varname) || (asOftenAsPossible setEncVarInActivity(cname,varname) end) end;

The user can also decide to request a modification of interaction diagrams. In this case, he or she has to interactively provide a value for the stimulus after or before which to place the new call, and the transformation unit is completed by the following construct. asOftenAsPossible getEncVarInInteraction(cname,varname,stimulus) end asOftenAsPossible setEncVarInInteraction(cname,varname,stimulus) end

By applying the transformation unit, both code and model graphs are transformed to reflect the existence and usage of the new methods. As an example, the graph in Figure 14 is a subgraph of the resulting code graph (the body of playMusic is not shown as it remained unchanged) obtained by applying the transformation unit EncapsulateVariable, i.e., the local rule encapsulate_variable_code has been applied once with arguments cname = “Audio” and varname = “preferences”.

Extract Method In the words of Martin Fowler, “If you can do Extract Method, it probably means you can go on more refactorings. It’s the sign that says, ‘I’m serious about this’.” We present our approach to managing this refactoring, without figures due to lack of space. A more detailed version, but with a different code representation, is in Bottoni et al. (2003). The pre-condition that the name for the new method does not exist in the class hierarchy is checked for variable encapsulation. In general, we can assume that the code and model graphs are complemented by all the needed gen edges. The pre-condition that the code to be extracted be a block is easily checkable on the

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Specifying Coherent Refactoring of Software Artefacts 117

Figure 14. refactoring

Code graph for class Audio after the EncapsulateVariable

: java-class-file c1‘: class name = „Audio“

: java-source-program

m1‘: method name= „playMusic“ id = „Audio:mth1“ visibility = protected

m4‘: method name= „setpreferences“ id = „Audio:mth3“ visibility = public

: asignment-expr op= „=“

: type name= „Environment“

a3‘: field name= „env“ visibility = private

m3‘: method name= „getpreferences“ id = „Audio:mth2“ visibility = public

: block

: type name= „MusicDescription“

a1‘: field name= „preferences“ visibility = private

: lvalue : var-ref name= „arg“ id = „Audio:frm1“

: type name= „void“ primitive = true

: type name= „MusicDescription“

: return

: var-ref name= „preferences“

: formal_arguments

: type name= „void“ primitive = true

p2‘: formal_argument name= „arg“ id = „Audio:frm1“

a2‘: var-set name= „preferences“ : type name= „MusicDescription“

code graph. Indeed, this code can be a whole subtree rooted in a block, if, switch, loop, do-loop vertex, or a collection of contiguous subtrees of a same method vertex, composed of stmt-exprs not comprising any construct try, throw, return, continue, break, synchronized, and such that no label appears in them. We then need to identify all the variables to be passed to the new method. The code graph is inspected to identify all the var-set and var-ref elements where the name of the variable is not the name of a formal-argument of the original method or a name for a local-variable declaration present in the subtree to be moved. Additionally, if the subtree presents some local-variable vertex, we check that there are no var-set or var-ref elements for that variable in the subtrees remaining with the original method. The creation of the call for the new method is achieved by substituting the removed subtrees with a send element with the name of the new method as value of the attribute message, target this, and the list of formalarguments as derived before. In the model, we modify the class diagram by simply showing the presence of the new method in the class, as the effects on the referred variables and the existence of a call for this method are not reflected at the structural level. For the activity diagrams, we need to identify the Action associated with a given Operation. Such an Action can be further detailed through

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

118 Bottoni, Parisi-Presicce, and Taentzer

a collection of Actions associated with it. So, we need to identify all those vertices which correspond to roots of the moved subtrees, detach them from the description of the Operation, and exploit them to create the description of the Operation associated with the new method. For interaction diagrams, one must identify the existing instances of Stimulus occurring before and/or after the extracted code, and insert the Stimulus to a CallOperationAction, for an Operation with the name of the new Method, with equal receiver and sender. Moreover, each instance of CallOperationAction, originating from the original Operation instances and related to a vertex in the extracted subtrees, must now be related to an instance of Stimulus, whose activator is the Stimulus for the new Operation. The existing predecessor and successor associations for the first and last such instances of Stimulus are transferred to the new Operation. These transformations must be applied as often as possible, so as to affect all the descriptions of the behavior of the refactored method. Indeed, calls to such methods can occur in different scenarios, meaning that the sequence diagrams for all such scenarios must be modified.

Building Correspondences between Code and Model Graphs In order to manage distributed transformations involving the Abstract Syntax Tree (AST) viewed as a graph and the graph representing the UML model, we need to establish an interface graph IG and two morphisms µAST and µUML from it to the two graphs. This requires the construction of a correspondence between types of vertices in the two graphs. To this end, we adopt for AST the concrete representation given by JavaML, an XML-based specification, while the graph for the UML model is constructed in accordance with the UML metamodel. In this section, we sketch the principles directing the construction of the correspondences, as the complete construction is beyond the scope of this chapter. In particular, we consider the structural and behavioral aspects of the specification separately. From the structural point of view, we can proceed top-down and see that JavaML class vertices correspond to UML class vertices, and a JavaML field to a UML Attribute. However, care must be taken in managing aspects of the structural definition involving relations to other classes. For example, the subclass relation is represented in UML by the presence of a pattern involving two Class vertices, a Generalization vertex, and two associations relating the

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Specifying Coherent Refactoring of Software Artefacts 119

latter vertex to the other two, one with the role of specialization, the other a generalization. In JavaML, a superclass vertex, with a name attribute, constitutes a leaf of the tree rooted in the class vertex. In such a case, IG would contain only a CLASS vertex, mapping to a class vertex in AST through µAST, and to a class vertex in UML through µUML. The definition of the morphisms requires checking that the superclass relation is consistently represented in the two graphs. A similar situation occurs for the implements construct. As concerns behavioral aspects, the method vertex in JavaML contains all the information present in the code to characterize the method, in particular its signature and its body. However, in the UML metamodel, this information is distributed across an Operation vertex, maintaining information about the signature, and a Method vertex which simply contains the code of the method body. As regards the signature, similarly to before, we relate Method and Operation vertices and we check the agreement of the type information, without associating the type subvertices for method to the Classifier vertices describing those types in UML. This is due to the fact that a type vertex is present in JavaML every time it is necessary, but, in a UML diagram, it needs to be present only once, and associated with other vertices an arbitrary number of times. To model not only the static declaration of a method, but also its behavior through collaboration, sequence, state, or activity diagrams, we recur to action semantics as defined in OMG (2003). Here, a Method is associated with a Procedure, which has a Composition relation with an Action vertex. We put such an Action in correspondence with the stmt-elems vertex, usually a block, which is the root of the subtree for the description of the method vertex. In general, we want to put into relation semantically equivalent elements, so we will consider the different types of Action that can be associated with stmt-elems. A major difference exists, though. The JavaML file presents the stmt-elems of a block in an order which corresponds to the sequence of statements in the original code. The UML model on the other hand, does not require an order to be specified for independent actions. Control flow actions indeed exist, such as ConditionalAction or LoopAction, and idioms such as Iteration can be expressed. However, actions not related through some chain of DataFlow objects, need not be realized in any given order. If desired, though, the modeler can prescribe the existence of ControlFlow objects, defining predecessor and successor Actions. The process of building such correspondences, i.e., of introducing elements in the interface graph and establish the morphisms from this to the code and model graphs can be modeled by rewriting rules. Figure 15 shows two local rules whose distributed application on the code and model graph, respectively, produces the following effect: if there are, both in the code and model graph, elements representing a class s which is a superclass for a class c whose representations

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

120 Bottoni, Parisi-Presicce, and Taentzer

Figure 15. Two rules to establish correspondences concerning class inheritance : java-class-file c1‘: class name = c

: java-class-file : superclass name = s

c1‘: class name = c

c2‘: class name = s

: class name = s

c1:Class name= c specialization

: superclass name = s

:Class name= s generalization :Generalization

c1:Class name= c specialization

c2:Class name= s generalization :Generalization

in the two graphs have already been put in correspondence, as witnessed by the identifiers c1 and c1’ for the two instances, then the two representations of class s are put in correspondence, as witnessed by the generation of the identifiers c2 and c2’.

Behavior Preservation in Refactoring The refactorings in Section 4 are historical ones, whose behavior preservation properties are widely discussed. In general, it is important to have a methodology to formally verify properties of new refactorings. The approach based on graph transformations provides a formal basis to perform these checks and to generate conditions ensuring the desired preservation properties. We restrict our discussion to checks that can be carried out statically, not based on flow analysis. Several kinds of behavior preservation are relevant to the refactorings studied here. Some of them are briefly described next (Mens et al., 2002):

•

Type preservation: the refactoring does not change the type of any entity not deleted.

•

Access preservation: the refactoring maintains the access of a method to at least the same variables accessed before the refactoring, possibly through intermediate method calls.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Specifying Coherent Refactoring of Software Artefacts 121

•

Update preservation: each refactored method causes the same variable changes as before.

•

Call preservation: each method causes the execution of at least the same methods called before the refactoring.

The refactorings presented here are based on typed graph transformations. Since these transformations always guarantee that all the resulting graphs are correctly typed over the same type graph, type preservation (according to the type graph) is always exhibited by them.

Well-Formedness Constraints This kind of behavior preservation is not sufficient to ensure that the resulting graph is an acceptable code or model graph. Well-formedness constraints are needed to rule out undesired configurations of the produced graph (instance of the type graph). For example, we have seen the requirement that no names, whether for variable or method, are in conflict in any class. Refactoring-specific constraints addresses the problem of unwanted side effects. These constraints can be expressed with pre and/or post-conditions. With the latter, if the post-condition is not met, the transformation must be “undone” and the previous model restored. With the former (more efficient) method, application conditions are checked to prevent the transformation by a refactoring if it produces unwanted effects. For example, a new method m defined in class C should not override an existing method m with the same signature in a subclass of C, or be overridden by an existing method with the same signature defined in a superclass of C. This constraint is needed, for example, in both sample refactorings presented in the section, Refactoring by Graph Transformation. Not all constraints can be expressed by simple “forbidden” graphs. More general constraints can be defined by using propositional logic (Matz, 2002) to compose “atomic” constraints, formed by simple forbidden graphs, and injective graph morphisms describing the conditional existence of graph (sub)structures (Koch, Parisi Presicce, 2002). For example, to express the fact that no method of arity one is allowed to have the same name and a parameter of the same type as another method in the same class, we can write the formula NOT two_methods_with_same_signature where the constraint graph is presented in Figure 16. This formula is satisfied only if it there are not two methods named mnew in a model graph having each exactly one parameter of the same type.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

122 Bottoni, Parisi-Presicce, and Taentzer

Figure 16. Constraint graph expressing the well-formedness of methods with one parameter two_methods_with_same_signature: : Class name = target

owner feature

owner feature

parameter

: Method name=mnew parameter

: Method name=mnew

: Classifier

type

: Parameter kind = #in

type

: Parameter kind = #in

Knowing the maximum number of arguments for a method, we can construct similar forbidden configurations for each number of arguments. Generalizing this idea, we have to define graph constraint schemes, typically using set vertices (Bottoni et al., 2000), to be matched to any number of arguments.

Consistent Refactorings In order to show that well-formedness constraints are satisfied after the application of refactoring rules, we can enlist the help of established results on consistent graph transformations in Heckel et al. (1995). That approach consists of first translating graph constraints into post-conditions of transformation rules and then forming pre-conditions out of those post-conditions by applying the rules backwards. In the special case of a forbidden subgraph as constraint, this construction would yield a number of NACs for the rule, preventing the application of the rule if it causes the construction of the forbidden subgraph. For example, the application of this construction to the constraint in Figure 16 would yield NACs for the rule in Figure 13, in the case of method “get”+varname.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Specifying Coherent Refactoring of Software Artefacts 123

Conclusions We have presented a graph transformation-based approach to maintaining consistency between code and model diagrams in the presence of refactorings. The approach allows the coordinated transformation of two graphs representing the abstract syntax, as derived from the code by a parser, and the UML model of the software system. A correspondence is established between these two graphs, starting from the correspondence between types of vertices in the abstract syntax trees, as defined by the JavaML markup language, and types of elements and associations in the UML diagrams, as defined by the UML metamodel. Although the approach has been demonstrated using Java and the JavaML coding of its abstract syntax, it can be applied to any type of abstract syntax for object-oriented languages, provided that a non-ambiguous correspondence between the abstract syntax and the UML model components can be established. As a consequence, an integrated tool which is able to perform refactoring on code and model diagrams while maintaining the original correspondences between these components is imaginable. This would require integrating the ability of modern refactoring tools to manipulate ASTs, with a more general interpreter for transformation units. Indeed, it is not needed that the tool exploits graph transformations in order to manipulate the tree. As all refactorings are individually described by a transformation unit, and a tool has a finite number of them available, it is sufficient that the tree transformation is wrapped. In this way, the parameters can be communicated to the other parts of a distributed transformation. If the transformation occurs on a part of the code for which the corresponding parts of the model have been identified, the relevant modifications would automatically be performed. The opposite process could also be envisaged in which a refactoring of a model would reflect a modification of the corresponding code. This can be easily performed on structural diagrams, for which we have seen that there is a close correspondence between elements of JavaML and of the UML meta-model. Future work will have to identify refactorings in the behavioral diagrams for which it is possible to identify the needed transformations in the code.

Acknowledgments Partially supported by the EC under Research and Training Network SeGraVis.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

124 Bottoni, Parisi-Presicce, and Taentzer

References Badros, G. (2000). JavaML: A Markup Language for Java Source Code. 9th Int. World Wide Web Conference. JavaML-Homepage. Retrieved from the WWW: http://www.cs.washington.edu/homes/gjb/JavaML Bottoni, P., Parisi Presicce, F. & Taentzer, G. (2003). Specifying Integrated Refactoring with Distributed Graph Transformations. In J. L. Pfaltz, M. Nagl, & B. Böhlen (Eds.), Applications of Graph Transformations with Industrial Relevance. Second International Workshop, AGTIVE 2003, LNCS 3062, Springer, pp. 220-235. Bottoni, P., Schuerr, A. & Taentzer, G. (2000). Efficient Parsing of Visual Languages based on Critical Pair Analysis (and Contextual Layered Graph Transformation). Proc VL 2000, (pp. 59-61). Corradini, A. Montanari, U. Rossi, F. Ehrig, H., Heckel, R. & Löwe, M. (1997). Algebraic approaches to graph transformation part {I}: Basic concepts and double pushout approach. In G. Rozenberg (Ed.), Handbook of Graph Grammars and Computing by Graph transformation, Vol. 1. World Scientific, 163-246. Fischer, I., Koch, M., Taentzer, G. & Volle, V. (1999). Visual Design of Distributed Systems by Graph Transformation. In H. Ehrig, H.J. Kreowski, U. Montanari & G. Rozenberg (Eds.), Handbook of Graph Grammars and Graph Transformation, (Vol. 3, pp. 269-340). Fowler, M. (1999). Refactoring: Improving the Design of Existing Programs. New York: Addison-Wesley. Heckel, R. & Wagner, A. (1995). Ensuring Consistency in Conditional Graph Grammars: A Constructive Approach. Proceedings of SEGRAGRA’95, ENTCS, Vol.2. Retrieved from the WWW: http://www.elsevier.nl/ locate/entcs/volume2.html Koch, M. & Parisi Presicce, F. (2002). Describing policies with graph constraints and rules. In A. Corradini, H. Ehrig, H.J. Kreowski & G. Rozenberg (Eds.), Proc. ICGT 2002, LNCS 2505, Springer, 223-238. Kreowski, H.J., Kuske, S. & Schürr, A. (1997). Nested graph transformation units. International Journal on Software Engineering and Knowledge Engineering, 7(4), 479-502. Matz, M. (2002). Design and Implementation of a Consistency Checking Algorithm for Attributed Graph Transformation. (In German.) Diploma Thesis, Technical University of Berlin.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Specifying Coherent Refactoring of Software Artefacts 125

Mens, T. (2000). Conditional Graph Rewriting as a Domain-Independent Formalism for Software Evolution. In Proceedings of AGTIVE’99, LNCS 1779, (pp. 127-143). Springer-Verlag. Mens, T. (2001). Transformational Software Evolution by Assertions. Proc. Wks. on Formal Foundations of Software Evolution (pp. 67-74). Mens, T., Demeyer, S. & Janssens, D. (2002). Formalising Behaviour Preserving Program Transformations. In Proceedings of IGCT2002, LNCS, 2505 (pp. 286-301). Springer-Verlag. Metz, P., O´Brien J. & Weber, W. (2002, July). Use Case Model Refactoring: Changes to UML’s Use Case Relationships. Internal Research Report, Department of Computer Science, Darmstadt University of Applied Sciences. Niere, J., Wadsack, J.P. & Zündorf, A. (2001). Recovering UML Diagrams from Java Code using Patterns. Proceedings of the Second Workshop on Soft Computing Applied to Software Engineering. OMG. (2003). UML Specification 1.5. Retrieved from the WWW: http:// www.omg.org/uml Opdyke, W.F. (1992). Refactoring Object-Oriented Frameworks. PhD Thesis. University of Illinois. Retrieved from the WWW: ftp://st.cs.uiuc.edu/ pub/papers/refactoring/opdykethesis.ps.Z Roberts, D. (1999). Practical Analysis for Refactoring. PhD Thesis. University of Illinois at Urbana-Champaign. Sunyé, G., Pollet, D., LeTraon, Y. & Jézéquel, J.M. (2001). Refactoring UML models. Proceedings of UML 2001, LNCS, 2185 (pp. 134-138). SpringerVerlag.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

126 Bottoni, Parisi-Presicce, and Taentzer

Section II Elaboration of Transformation Approaches

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Declarative Transformation for Object-Oriented Models

127

Chapter VI

Declarative Transformation for Object-Oriented Models Keith Duddy, CRC for Enterprise Distributed Systems Technology (DSTC), Queensland, Australia Anna Gerber, CRC for Enterprise Distributed Systems Technology (DSTC), Queensland, Australia Michael Lawley, CRC for Enterprise Distributed Systems Technology (DSTC), Queensland, Australia Kerry Raymond, CRC for Enterprise Distributed Systems Technology (DSTC), Queensland, Australia Jim Steel*, CRC for Enterprise Distributed Systems Technology (DSTC), Queensland, Australia

Abstract This chapter provides a context and motivation for a language to describe transformations of models within an object-oriented framework. The requirements for such a language are given, and then an object-oriented Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

128 Duddy, Gerber, Lawley, Raymond, and Steel

model of the language’s abstract syntax is provided that meets these requirements. A concrete syntax is introduced along with some example transformations. Finally, we discuss the tools required to use the language within a model-driven software engineering paradigm. The authors aim to demonstrate the principles of model transformation within an objectoriented framework, and show how this can be applied to the development of software systems.

Introduction In Model-Driven Architecture - A Technical Perspective (2001), the Object Management Group (OMG) describes an approach to enterprise-distributed system development that separates the specification of system functionality from the specification of the implementation of that functionality on a specific technology platform. The MDA approach envisions mappings from Platform Independent Models (PIMs) to one or more Platform Specific Models (PSMs). The potential benefits of such an approach are obvious: support for system evolution, high-level models that truly represent and document the implemented system, support for integration and interoperability, and the ability to migrate to new platforms and technologies as they become available. While technologies such as the Meta Object Facility (MOF v1.3.1, 2001) and the Unified Modelling Language (UML, 2001) are well-established foundations on which to build PIMs and PSMs, there is as yet no well-established foundation suitable for describing how we take an instance of a PIM and transform it to produce an instance of a PSM. In addressing this gap, our focus is on model-to-model transformations and not on model-to-text transformations. The latter come into play when taking a final PSM model and using it to produce, for example, Java code or SQL statements. We believe that there are sufficient particular requirements and properties of a model-to-text transformation, such as templating and boilerplating, that a specialised technology can be used. One such technology is Anti-Yacc (Hearnden & Raymond, 2002) and we deal briefly with such concrete syntax issues later in the chapter. This chapter focuses on a particular program transformation language, designed specifically for use with object-oriented models and programming languages. We provide an overview of the general problem of software model transformation and survey some technologies that address this space. The technology we then describe is designed to satisfy a set of identified requirements and is

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Declarative Transformation for Object-Oriented Models

129

illustrated with a variety of example transformation specifications. We pay particular attention to how its features easily handle complex transformations and enable modular, composable, and extendable transformation rules without imposing an undue burden on the writer of the rules.

Transformation for EnterpriseDistributed Systems Development In order to build and maintain the IT systems supporting large-scale enterprisedistributed systems efficiently, descriptions of these systems at the domain level need to be automatically transformed into components, code, and configurations. The use of a transformation language designed to support the features of modern object-oriented specification and implementation technologies leads to more flexible, maintainable, and robust transformations than present ad-hoc approaches. The Meta Object Facility (MOF) is a technology specification standardised by the OMG in 1997. It provides an object-oriented framework for the specification of the abstract syntax of modeling languages. Space limitations do not permit a detailed description of MOF features, however, one can think of MOF models as corresponding to a slightly simplified UML Class Diagram. The benefits of using this facility for the specification of languages such as the Unified Modelling Language (UML) are that there are standard mechanisms for automatically deriving:

•

a set of interfaces in CORBA IDL or Java for programmatic access to object model repositories,

•

a concrete syntax based on XML DTDs and/or schemas known as XML Model Interchange (XMI), and

•

a customisable human-usable textual notation or HUTN (Human-Usable Textual Notation, 2002) for representing model instances.

However, to date, the common MOF foundation of OMG languages such as UML, the Common Warehouse Metamodel (CWM) and the Enterprise Distributed Object Computing (EDOC) model has not enabled the use of a model in one language to be transformed into a model in another language, except by the following limited means:

•

An XML document representing one model in the standard XMI form may be manipulated using XSLT to produce another model.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

130 Duddy, Gerber, Lawley, Raymond, and Steel

•

A program may traverse the model using CORBA or Java interfaces, and populate another model in a different repository.

•

Partial transformations of data may be described in the CWM.

All of these approaches have some usefulness. However, a language for describing the generic transformation of any well formed model in one MOF language into a model in some other MOF language (or perhaps in the same language) is not yet available in a standard form. The OMG has issued MOF 2.0 Queries/Views/Transformations RFP (2003), known as QVT for short. It requires submissions to:

• • • •

define a language for querying MOF models,

•

ensure that incremental changes to source models can be immediately propagated to the target models, and

•

express all new languages as MOF models.

define a language for transformation definitions, allow for the creation of views of a model, ensure that the transformation language is declarative and expresses complete transformations,

In developing our response to the QVT RFP, the authors considered a number of alternative approaches (Gerber, Lawley, Raymond, Steel & Wood, 2002). The results, along with a review of other submissions to the QVT RFP, are summarised below. Chapter 13 of the OMG’s Common Warehouse Metamodel Specification (2001) defines a model for describing transformations. It supports the concepts of both black-box and white-box transformations. Black-box transformations only associate source and target elements without describing how one is obtained from the other. White-box transformations, however, describe fine-grained links between source and target elements via the Transformation element’s association to a ProcedureExpression. Unfortunately, because it is a generic model and reuses concepts from UML, a ProcedureExpression can be expressed in any language capable of taking the source element and producing the target element. Thus CWM offers no actual mechanism for implementing transformations, merely a model for describing the existence of specific mappings for specific model instances. Varró and Gyapay (2000) and Varró, Varraó and Pataricza (2002) describe a system for model transformation based on Graph Transformations (Andries et al., 1999). In their approach, a transformation consists of a set of rules combined

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Declarative Transformation for Object-Oriented Models

131

using a number of operators such as sequence, transitive closure, and repeated application. Each rule identifies before and after sub-graphs, where each subgraph may refer to source and target model elements and associations between them (introduced by the transformation). This style of approach to model transformation introduces non-determinism in the rule selection, and in the sub-graph selection when applying a rule. Additionally, since rules are applied in a sequence, thus resulting in a series of state changes, one needs to be very careful about the order of rule application and repeated rule application to ensure termination of the transformation process. A common technique is to delete elements from the source model as they are transformed, but this requires that an element is transformed with a single rule rather than allowing multiple rules to address different aspects of the transformation. Peltier, Ziserman and Bezevin (2000) and later, Peltier, Bezevin and Guillaume (2001) and Alcatel, Softeam, Thales and TNI-Valiosys (2003) in their QVT RFP response propose that transformation rules are best expressed at the model level, and that they should then be translated into a set of rules that operate on the concrete representations of model instances. As such, they propose MOF as the common meta-model for representing models, XMI as the concrete expression of model instances, and XSLT as the transformation tool to operate on these concrete instances. Their rules have a mix of both procedural and declarative styles that is in part due to the fact that a given rule may only define a single target element per source element and that target element construction is explicit. They also require explicit ordering of the execution of rules. XSLT (W3C, 1999) is explicitly designed for the transformation of XML documents and, through the XMI specification, all MOF models have an XMLbased expression. However, being document-driven, XSLT is limited to sourcedriven transformations, and only provides for explicit object (or element) creation. Additionally, the XSLT syntax is both verbose and baroque, as it is based on XML. These reasons make it wholly unsuitable as an expressive model transformation language. The UML 1.4 specification introduces a new Action Semantics language (ASL), which has also been proposed by both Kennedy Carter (2003) and Tata Consultancy Services (2003) as a model transformation language. The ASL provides a number of low-level constructs that can be composed to provide specifications of actions. However, in order to be fully expressive for describing actions, ASL is defined at a very low level of abstraction, lower than appropriate for model transformation. Furthermore, it is an imperative language, so it does not provide for implicit creation, or unordered rule evaluation.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

132 Duddy, Gerber, Lawley, Raymond, and Steel

Codagen Technologies (2003) propose an extension to XQuery and XPATH for selecting/querying MOF models, and a procedural templating language (MTDL) for constructing target model elements. In contrast, Interactive Objects Software GmbH & Project Technology (2003) propose the use of OCL 2.0 (2003) for querying/selecting source model elements and a declarative language for constructing target model elements. Because there can only be a single creation rule that gives rise to a target model element, this proposal also effectively uses explicit object creation, which we believe is an unfavourable approach to building scalable and re-usable transformation definitions. Finally, Compuware & Sun Microsystems (2003) propose a language that is based on OCL 2.0 for queries/selection, is declarative, uses implicit creation, and results in links between target model elements and the source model elements that lead to their creation. However, the language is defined as a minimal subtyping of the MOF meta-model and consequently provides little in the way of structuring mechanisms to guide the user in writing transformation definitions.

Design Requirements for OO-Based Transformation Consider the problem of representing data described by an object-oriented class diagram in a relational database. Figure 1 shows possible models of the concepts involved in these two domains. This is an interesting transformation problem because the two models are familiar to many people, and they are not so similar that a transformation is essentially one-to-one. Additionally, there are simple alternative transformations

Figure 1. Simple source and target models NamedElt name : String

Named -name : String

1

Classifier isAbstract : boolean

type * DataType

*

attr

Class

Attribute

col

Table

type

Column

Type

multivalued : boolean super

1 sub

*

1

* 0..1

*

* key

1

*

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Declarative Transformation for Object-Oriented Models

133

possible that result in different performance characteristics that we can also explore. Let us consider what such a transformation might involve. Firstly, we might expect each Class to have a corresponding Table, each DataType to have a corresponding Type, and each Attribute of each Class to have a corresponding Column in the corresponding Table. Since our class model doesn’t contain any information about the combination of attributes (if any) that constitute a key, we will also need a Column per Table to represent an object-id. However, note that an Attribute may be multi-valued, so it would be rather inefficient to store all the other attributes multiple times in order to store the multivalued attribute. Instead, we want a separate table for each Column corresponding to a multi-valued Attribute along with a Column for the object-id so we can perform the requisite joins. If an Attribute’s type is a DataType, then the corresponding Column’s type would be the Type corresponding to the DataType. But if the Attribute’s type is a Class, then the corresponding Column would need to be a foreign key, so its type would be that of the (primary) key of the referenced table which is the object-id Column. So far, describing the transformation has been relatively straightforward. Things get more interesting once we consider that one Class can subtype another Class. As we have described the mapping so far, an instance of a subclass ends up with its attributes split between several tables. An alternative transformation involves creating a Column for every Attribute owned by a Class and for every Attribute owned by a supertype of the Class. Below we show, step by step, how one would express the transformations described above in our language. However, before we look at these details we reflect on the high-level natural-language description of the transformation we have just outlined in order to motivate the design of our language. The first thing to notice is that the description above is naturally declarative. It describes the things we want to generate and the reasons why we want to generate them, but it does not say how or when to create them, not does it involve a description of traversing either the source or the target models. Additionally, while there are natural dependencies between parts of the description, there is no explicit statement of “do this then do that”. Rule ordering is implicit, not explicit. Since the declarative style is a natural way to communicate a transformation description in a human language, we believe it is also suitable, indeed preferable, for a formal language for specifying transformations to be declarative. A set of functional and usability requirements for a transformation language has been developed by Gerber, Lawley, Raymond, Steel and Wood (2002). A

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

134 Duddy, Gerber, Lawley, Raymond, and Steel

detailed list of these is presented in our response (DSTC, IBM & CBOP, 2003) to the OMG’s QVT RFP. The major functional requirements are as follows. A model-transformation language must be able to:

•

match elements, and ad-hoc tuples of elements, by type (include instances of sub-types) and precise-type (exclude instances of sub-types);

•

filter the set of matched elements or tuples based on associations, attribute values, and other context;

•

match both collections of elements not just individual elements. For example, we may need to count the number of Attributes a Class has;

•

establish named relationships between source and target model elements. These relationships can then be used for maintaining traceability information;

•

specify ordering constraints (of ordered multi-valued attributes or ordered association links), either when matching source elements or producing target elements;

•

handle recursive structure with arbitrary levels of nesting. For example, to deal with the subclassing association in our example Class model;

• •

match and create elements at different meta-levels; support both multiple source extents and multiple target extents.

In addition, the following non-functional requirements, identified for readability and expressiveness concerns, require that:

•

there is no requirement to explicitly specify the application order of the rules, and all rules are matched against all relevant source elements;

•

creation of target objects is implicit rather than explicit. This follows from the previous requirement; if there is no explicit rule application order, then we cannot know which rule creates an object and are relieved of the burden of having to know;

•

a single target element can be defined by multiple rules. That is, different rules can provide values for different attributes of the same object;

•

patterns can be defined and rules are able to be grouped naturally for readability and modularity;

•

embedding of conditions and expressions in the language is explicit and seamless;

•

transformation rules are composable.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Declarative Transformation for Object-Oriented Models

135

Our experiences have shown that there are three fairly common styles to structuring a large or complex transformation, reflecting the nature of the transformation. They are:

•

Source-driven, in which each transformation rule is a simple pattern (often selecting a single instance of a class or association link). The matched element(s) are transformed to some larger set of target elements. This style is often used in high-level to low-level transformations (e.g., compilations) and tends to favour a traversal style of transformation specification. This works well when the source instance is tree-like, but is less suited to graphlike sources;

•

Target-driven, in which each transformation rule is a complex pattern of source elements (involving some highly constrained selection of various classes and association links). The matched elements are transformed to a simple target pattern (often consisting of a single element). This style is often used for reverse-engineering (low-level to high-level) or for performing optimizations (e.g., replacing a large set of very similar elements with a common generic element);

•

Aspect-driven, in which the transformation rule is not structured around objects and links in either the source or target, but more typically around semantic concepts, e.g., transforming all imperial measurements to metric ones, replacing one naming system with another, or the various parts of the object-relational transformation described above.

Indeed, aspect-driven transformations are a major reason why we favour implicit (rather than explicit) creation of target objects, since aspect-driven transformation rules rarely address entire objects, and thus it is extremely difficult to determine which of several transformation rules (that may or may not apply to any given object) should then have responsibility for creating the target object. Typically the target object is only required if any one of the transformation rules can be applied, but no target object should be created if none of the rules can be applied. This is extremely difficult to express if explicit creation is used.

A Declarative Object-Oriented Transformation Language We describe a declarative object-oriented transformation environment that satisfies the requirements described in the previous section. We present both a

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

136 Duddy, Gerber, Lawley, Raymond, and Steel

formal model for transformations and a concrete syntax and illustrate the transformation language through a series of simple examples. This section presents the transformation language that we have designed to address the problems faced when realising the MDA, by illustrating how the language would be used to solve the object-relational mapping problem at hand. A transformation in our language consists of the following major concepts: transformation rules, tracking relationships, and pattern definitions.

•

Transformation rules are used to describe the things that should exist in a target extent based on the things that are matched in a source extent. Transformation rules can be extended, allowing for modular and incremental description of transformations. More powerfully, a transformation rule may also supersede another transformation rule. This allows for generalcase rules to be written, and then special-cases dealt with via superseding rules. For example, one might write a naive transformation rule initially, then supersede it with a more sophisticated rule that can only be applied under certain circumstances. Superseding is not only ideal for rule optimization and rule parameterization, but also enhances reusability since general purpose rules can be tailored after-the-fact without having to modify them directly.

•

Tracking relationships are used to associate a target element with the source elements that lead to its creation. Since a tracking relationship is generally established by several separate rules, other rules are able to match elements based on the tracking relationship independently of which rules were applied or how the target elements were created. This allows one set of rules to define what constitutes a particular relationship, while another set depends only on the existence of the relationship without needing to know how it was defined. This kind of rule decoupling is essential for rule reuse via extending and superseding to be useful. Establishing and maintaining Tracking relationships is also essential for supporting round-trip development and the incremental propagation of source-model updates through the transformation to the target model(s).

•

Pattern definitions are used to label common structures that may be repeated throughout a transformation. A pattern definition has a name, a set of parameter variables, a set of local variables, and a term. Parameter variables can also be thought of as formal by-reference parameters. Pattern definitions are used to name a query or pattern-match defined by the term. The result of applying a pattern definition via a pattern use is a collection of bindings for the pattern definition’s parameter variables.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

-term

0..1

AndTerm

-defn

-lesser : MofFeature -greater : MofFeature

*

-extraction -feature

MofFeature

-type

-tgt

MofTerm

-name -srcModel -tgtModel

MofInstance

-ifTerm : Term -thenTerm : Term -elseTerm : Term

MofFeatureOrder

OrTerm

IfTerm

-transformation

1

1

*

Transformation

-supercedee * -superceder 0..1 TRule * -name * -trule -extendee -extender -trule * * -transformation

1

-scope

-extender -extendee

CompoundTerm

*

* *

PatternDefn

-name

-patternDefn

-body

0..1 0..1

-src

-compoundTerm

*

Term

0..1

Scope

Tracking

MofLinkOrder

PatternUse

-lesser : MofLink -greater : MofLink

TrackingUse

0..1

-arg

-use

-simpleTerm

-source -target -arg * 1 *

* -tracking -tracking

SimpleTerm

0..1

-domain

-domain

MofLink

0..1

-type

1

-trule

*

-tracking -trule

*

-expr

0..1

-kind

**

SimpleExpr

BooleanConstant

-representation

NamedExpr -name

IntConstant

FunctionExpr -operator

StringConstant

-var

1 -superceder

*

-supercedee

CompoundExpr

CollectionExpr

*

Expr

Var -name -local

-extender

-extendee

* *

-var

-use

*

EnumConstant

VarUse

Declarative Transformation for Object-Oriented Models 137

Figure 2. Transformation model

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

138 Duddy, Gerber, Lawley, Raymond, and Steel

The MOF model diagram in represents our current proposed Transformation model (DSTC, IBM & CBOP, 2002). The important classes are explained in the following text. The lower part of Figure 2 is an expression language metamodel constructed specifically for identifying MOF model elements in patterns and rules. Its two main abstract metaclasses are Term and Expression. Terms are evaluated to be either true or false with respect to the models supplied for transformation. An Expression represents some kind of value referenced by a Term. VarUse is a particular kind of Expression that represents the binding of a value to a variable. Variables in the language are dynamically typed, and a Var slot may bind to any valid MOF type. The top left part of Figure 2 shows the main structuring parts of the model. TRule represents a transformation rule, which is a statement that for all model elements that can be bound to the Vars of the TRule’s src Term such that this Term evaluates to true the TRule’s tgt Terms must also evaluate to true. Generally, this involves creating new MOF model elements in the target model(s) and setting their attributes. In order to know when and how many model elements to create, Trackings are used. A Tracking is a statement of a functional dependency between a set of source model elements and a target model element. This allows several independent rules to require a target object to exist without needing to explicitly coordinate which rule is responsible for actually creating the instance. Transformation rules may be related to other transformation rules in either or both of the following two ways:

•

A rule that extends another rule augments its source matching term with the source term of the extended rule.

•

A rule that supersedes another rule restricts the source matching term of the superseded rule with the negation of its source term.

Rule extending and superseding allow for Transformations, which supply a namespace for rules and pattern definitions, to be reused and specialised. In particular, they allow rules to be written simply for the general case, and then superseded for some special or exceptional cases.

Concrete Syntax by Example We now present the transformation described earlier using an SQL-like concrete syntax one rule at a time, based on one rule per concept-to-concept mapping. We then link these together and fill in details to provide the total transformation from our OO model to our Relational model.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Declarative Transformation for Object-Oriented Models

139

The first key element of our transformation is that a Class will be transformed into a Table with an object-id Column, so this becomes our first rule. We also want to make sure that we preserve a tracking relationship between the table we create and the class from which we create it. The next major mapping, from an Attribute to a Column, is similar, as is the rule for DataTypes. As such, we start with the following simple rules: RULE class2table FORALL Class Cls MAKE Table Tbl, Column idCol, idCol.name="id", Col.owner=Tbl LINKING Cls to Tbl by c2t; RULE attr2col FORALL Attribute Att MAKE Column Col LINKING Att to Col by a2c; Both Class and Attribute are subtypes of NamedElt, and we want their names to be mapped to the names of their corresponding Tables and Columns. We can make sure we have the right Class-Table or Attribute-Column pair by looking up the tracking relationships we established earlier. We can then write a rule from an OO NamedElt to a Relational Named like this: RULE named2named FORALL NamedElt n1 WHERE c2t LINKS n1 to n2 OR a2c LINKS n1 to n2 MAKE Named n2, n2.name = n1.name; We see here that trackings can be used to tie rules together, thus giving us the ability to express rules as fine-grained mappings rather than having to write complex, coarse-grained rules. However, further inspection of our class diagram reveals that DataType names must also be mapped. Rather than adding another OR clause to our rule, we introduce generalization to our tracking relationships. So, we make another Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

140 Duddy, Gerber, Lawley, Raymond, and Steel

tracking relationship that stands as a superset of the two we have already used, and look up the parent tracking rather than alternating over the children, like so: TRACKING c2t ISA named2named; TRACKING a2c ISA named2named; RULE named2named FORALL NamedElt n1 WHERE named2named LINKS n1 to n2 MAKE Named n2, n2.name=n1.name; Next, we need to make sure that the column resulting from the transformation of an attribute will be contained by the appropriate table, i.e., the table resulting from the transformation of the attribute’s containing class. We do this by again looking up the tracking relationships established in our earlier rules. This gives us the following rule: RULE clsAttr2tblCol FORALL Attribute Att, Class Cls WHERE Att.owner = Cls AND c2t LINKS Cls to Tbl AND a2c LINKS Att to Col MAKE Table Tbl, Column Col, Col.owner = Tbl; We already have a rule for transforming Attributes. However, we now find that we wish to transform multi-valued attributes differently. The values of a multivalued attribute will be stored in a separate table, with one column for the values and one column for the Class’s object-id. This new rule for attributes will need to match a subset of the cases that were true for the previous rule, and we can reuse the earlier Attribute rule’s matching pattern by using rule extension. However, we also want to indicate that the earlier Attribute rule should not run when this new Attribute rule runs, and we can do this using rule supersession. So now we have a rule for transforming Attributes to Columns, and another for linking the Column to a Table. However, we find that we want to map multivalued attributes differently. The Column for a multi-valued Attribute should Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Declarative Transformation for Object-Oriented Models

141

instead have its own Table, with another Column to link back to the key in the main Table for the Class. Therefore, we make a new rule that will supersede the rule that puts Columns in Tables, and link the Attribute’s Column to a new Table with a new key Column. RULE clsMultiAttr2tblCol extends and supersedes clsAttr2tblCol FORALL Attribute Att, Class Cls, WHERE Att.multivalued = TRUE MAKE Table AttTbl, Column KeyCol, Column AttCol, KeyCol.owner=Tbl, AttCol.owner=Tbl, KeyCol.name = Cls.name LINKING Att to AttTbl by mva2t; Having created and placed these Columns, we need to give them an appropriate type. So we need rules for mapping DataTypes to Types, and for assigning the appropriate Type to a Column. The latter case requires two rules, since an Attribute with a Class type is typed for a key value, but an Attribute with a DataType type is mapped for the corresponding Type. TRACKING dt2t ISA named2named RULE datatype2type FORALL DataType Dt MAKE Type T LINKING Dt to T by dt2t; RULE atype2ctype FORALL Attribute Att, DataType Dt WHERE a2c LINKS Att to Col AND dt2t LINKS Dt to T AND Att.type = Dt MAKE Column Col, Type T, Col.type = T;

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

142 Duddy, Gerber, Lawley, Raymond, and Steel

RULE actype2ctype FORALL Attribute Att, Class C WHERE Att.type = C AND a2c LINKS Att to Col MAKE Column Col, Type T, Col.type = T, T.name = "String"; The other approach to mapping attributes, as described above, is to include inherited attributes as columns in the tables of subclasses. To do this, we need to define a recursive Pattern for finding the inherited attributes of a class. PATTERN hasAttr(C, A) FORALL Class C, Attribute A, Class C2 WHERE A.owner = C OR (C.super = C2 AND hasAttr(C2, A)); Having defined this pattern, we can make a rule for creating a column for each inherited attribute. To handle the linking of these columns to their tables, we need to change the Attribute to Column tracking to include Class as a source, by modifying the earlier rules, attr2col and clsAttr2tblCol. The new rule, as well as these modified rules, is below: RULE superattr2col FORALL Attribute Att, Class Cls WHERE hasAttr(Cls, Att) AND c2t LINKS Cls to Tbl MAKE Table Tbl, Column Col LINKING Att, Cls to Col by a2c; RULE attr2col FORALL Attribute Att, Class C WHERE Att.owner = C MAKE Column Col LINKING Att, Cls to Col by a2c;

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Declarative Transformation for Object-Oriented Models

143

RULE clsAttr2tblCol FORALL Attribute Att, Class Cls WHERE c2t LINKS Cls to Tbl AND a2c LINKS Att, Cls to Col MAKE Table Tbl, Column Col, Col.owner = Tbl; While some people may be daunted by the idea of specifying transformations using a declarative language, we believe that the use of an appropriate concrete syntax such as the SQL-like one introduced above will allay their fears and allow them to reap the benefits of not needing to worry about rule order and explicit object creation that a declarative language affords. Additionally, an equivalent graphical syntax would be a useful addition for simple transformations, although our experiences indicate that more complicated transformations are better handled textually.

Advanced Transformations As with expert systems, a substantial transformation embodies a significant investment in capturing domain knowledge and, therefore, the careful organisation and structuring of the transformation will aid its long-term maintenance and evolution. Several features of the transformation language described in this chapter are key to supporting both re-use and maintenance of transformation definitions. These features are the supersedes and extends relationships, and dynamic typing of variables. Duddy, Gerber, Lawley, Raymond & Steel (2003) describe in detail how superseding and extending can be used in the context of a transformation to an Entity Java Bean (EJB) model. Specifically, they show how a mapping that results in remote access to EntityBeans can be modified to instead employ the Session Façade pattern (Brown, 2001) using a SessionBean that delegates methods to local EntityBeans. One could also use an extra source model as a parameterisation, or marking model, to provide finer grain control over which rules are applied to which source model elements. Dynamic typing simplifies rule writing: if an object bound to a variable does not have the attribute or reference mentioned in a term, then the term simply evaluates to false rather than requiring explicit runtime-type introspection and

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

144 Duddy, Gerber, Lawley, Raymond, and Steel

narrowing (down-casting). Dynamic typing also allows meta-transformations and libraries of transformations matching common structural patterns to be easily developed without the use of cumbersome reflective methods. Such transformations may, for example, deal with mapping names in nested namespaces to structured names in a single flat namespace.

Transformation Tool Set The transformation language described previously is just one piece of the automated development environment. Other tools that deal with the input and output of concrete textual representations of models, and the graphical visualisation of models and transformations are vital parts of an MDA tool set. The OMG has developed the XML Metadata Interchange (XMI) Format (2002; XMI Production of XML Schema, 2001) which defines rules for producing both an XML DTD and an XML Schema from a model, and a set of rules for transferring data between conformant XMI documents and MOF-compliant repositories. The main use of XMI is for interchange of models and model instances between tools, and for storage. The result of the OMG’s RFP for a Human Usable Textual Notation standard (1999) was the adoption of a more generic approach for automatically creating a human-friendly language for an arbitrary MOF model. By exploiting the inherent structure in a MOF model (containment, association cardinalities, etc.), and allowing for some on-the-fly customisation (e.g., for default values) it is possible to fully automate the generation of both parsers and printers of the HUTN language for a given MOF model. Most commonly, the ultimate goal of a transformation process is to generate code or some other textual representation of a system. While it would be reasonable to realise this goal by transforming an abstract model to a model corresponding to the concrete syntax of the target language (Java, Cobol, HTML, etc.), this task is particular enough to warrant and benefit from a specialised tool. The motivation of the AntiYacc tool is to provide a simple means to render the content of a MOF-based repository in a textual form corresponding to some specified syntax. It is capable of producing text that conforms to an arbitrary user-supplied EBNF grammar. Just as a parser generator such as Yacc (Johnson, 1974) takes a set of grammar rules decorated with code fragments to, typically, construct a parse tree from a stream of lexical tokens, AntiYacc takes a set of grammar rules decorated with code fragments to construct a stream of lexical tokens from a traversal of a graph of MOF objects. An AntiYacc specification also includes lexical rules that convert the token stream into text,

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Declarative Transformation for Object-Oriented Models

145

typically addressing issues such as horizontal and vertical white space, and delimiters. Finally, it would often be very useful to be able to visualise or edit a graphical representation of the models being transformed. However, since much of the time their metamodels may be purpose-designed and therefore have no standard graphical representation (let alone a tool to display/edit the model), it would be extremely useful to be able to generate such a tool in a manner analogous to the HUTN approach. That is, to employ a set of standard, simple visual concepts (box, line, label, containment, proximity, etc.) to render a given model. Such a tool is currently under development by the authors.

Conclusions In this chapter we have introduced the problem of model-to-model transformation for the purpose of building distributed systems from high-level models describing the system to be built in platform-independent terms then generating the system implementation for a particular, technology specific, platform. This is the vision embodied in the OMG’s Model Driven Architecture (MDA). We have described the functional and non-functional design requirements identified for a language suitable for writing transformation definitions and presented a language satisfying these requirements along with examples of a usable, familiar concrete syntax for the language. We have also briefly touched on issues relating to advanced transformations and mentioned a number of additional technologies required for dealing with textual and graphical forms of the models. It should be noted that the transformation language presented here is evolving as we gain further experience and as a result of the OMG’s RFP process. In particular, influenced by the Compuware/Sun submission, we are extending the concept of Trackings to more closely approximate a class model. Also, composition of transformations is essential for the use and extension of existing transformations. While there is no explicit mention of this in the language presented here, the ability to reference elements in one MOF model from another MOF model should be sufficient for simple composition of transformations. However, more sophisticated forms of composition, such as producing a transformation that maps A to C from one that maps A to B and one that maps B to C or producing a transformation that merges A and B to produce C from the A to B and B to C transformations, is the subject of future research. Additionally, the transformations discussed in this chapter have generally dealt with transformation in a single direction, from model A to model B. Another use

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

146 Duddy, Gerber, Lawley, Raymond, and Steel

of a declarative transformation language, and one in which the declarative nature offers an advantage over imperative alternatives, is in the application of a single transformation to perform transformations both from A to B, and back again. This is another aspect that is currently under investigation by both the authors and the larger OMG community.

References A Human-Usable Textual Notation for the UML Profile for EDOC: Request for Proposal. (1999). OMG Document ad/99-03-12. Alcatel, Softeam, Thales & TNI-Valiosys. (2003). Response to the MOF 2.0 Queries/Views/Transformations RFP. OMG document ad/03-03-35. Andries et al. (1999). Graph transformation for specification and programming. Science of Computer Programming, 34(1), 1-54. Boldsoft, Rational Software Corporation, IONA & Adaptive Ltd. (2003). Response to the UML 2.0 OCL RFP. OMG Document ad/2003-01-02. Brown, K. (2001). Rules and Patterns for Session Facades. IBM’s WebSphere Developer Domain. Retrieved from the WWW: http://www.boulder. ibm.com/wsdd/library/techarticles/0106_brown/sessionfacades.html Codagen Technologies Corporation. (2003). MOF 2.0 Query/Views/Transformations. OMG Document ad/2003-03-23. Common Warehouse Metamodel. (CWM) Specification (2001). OMG Documents ad/01-02-01, ad/01-02-02, ad/01-02-03. Compuware Corporation & Sun Microsystems. (2003). XMOF Queries Views and Transformations on Models using MOF, OCL and Patterns. OMG Document ad/2003-03-24. DSTC, IBM, & CBOP. (2003). MOF Query/Views/Transformations Initial Submission. OMG Document ad/2003-02-03. Gerber, A., Lawley, M., Raymond, K., Steel, J. & Wood, A. (2002). Transformation: The Missing Link of MDA. Proceedings of the First International Conference on Graph Transformation (ICGT’02), Barcelona, Spain (pp. 90-105). LNCS 2505. Hearnden, D. & Raymond, K. (2002). Anti-Yacc: MOF-to-text. Proceedings of the Sixth IEEE International Enterprise Distributed Object Computing Conference, Lausanne, Switzerland. IEEE. Human-Usable Textual Notation (HUTN) Specification. (2002). OMG Document ptc/02-12-01.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Declarative Transformation for Object-Oriented Models

147

Johnson, S. (1974). YACC - Yet Another Compiler Compiler. CSTR 32, Bell Laboratories. Meta Object Facility (MOF) v1.3.1. (2001). OMG Document: formal/01-11-02. Model Driven Architecture - A Technical Perspective. (2001). OMG Document ormsc/01-07-01. MOF 2.0 Queries/Views/Transformations: Request for Proposal. (2002). OMG Document ad/02-04-10. Peltier, M., Bezevin, J. & Guillaume, G. (2001). MTRANS: A general framework, based on XSLT, for model transformations. Proceedings of the Workshop on Transformations in UML, Genova, Italy. Peltier, M., Ziserman, F., & Bezevin, J. (2000). On levels of model transformation. In XML Europe 2000, Paris. Unified Modelling Language v1.4. (2001). OMG Document: formal/01-0967. Varró, D. & Gyapay, S. (2000). Automatic Algorithm Generation for Visual Control Structures. Retrieved February 8, 2001 from the WWW: http:// www.inf.mit.bme.hu/FTSRG/Publications/TR-12-2000.pdf Varró, D., Varraó, G. & Pataricza, A. (2002). Designing the Automatic Transformation of Visual Languages. Science of Computer Programming, 44(2), 205-227. XMI Production of XML Schema. (2001). OMG Document ptc/2001-12-03. XML Metadata Interchange (XMI) Version 1.2. (2002). OMG Document formal/2002-01-02. XSL Transformations (XSLT) v1.0. (1999). W3C Recommendation. Retrieved from the WWW: http://www.w3.org/TR/xslt

Endnote *

Jim Steel is now at INRIA/Irisa, University of Rennes 1, France.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

148 Badia

Chapter VII

From Conceptual Models to Data Models Antonio Badia, University of Louisville, USA

Abstract This chapter describes transformations between conceptual models (mainly entity-relationship diagrams and also UML) and data models. It describes algorithms to transform a given conceptual model into a data model for a relational, object-relational, object-oriented and XML database. Some examples are used to illustrate the transformations. While some transformations are well known, some (like the transformation into XML or into object-relational schemas) have not been investigated in depth. The chapter shows that most of these transformations offer options which involve important trade-offs that database designers should be aware of.

Introduction Conceptual models aim at capturing the structure of reality, are high-level and computer independent. Data models, on the other hand, aim at representing reality in the computer, and are therefore less abstract. It is assumed that, in creating an Information System, a conceptual model will be developed as part of the Requirements Specification, from which a data model will be derived later on,

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

From Conceptual Models to Data Models 149

in the Design phase (Davis, 1993). Thus, mappings between conceptual models and data models are one of the most vital transformations in the development of an Information System (Elmasri and Navathe, 2003). The purpose of this chapter is to present transformations between conceptual models and data models. Transformations between well known and used conceptual models (EntityRelationship diagrams and UML Class Diagrams) and the most common and important data models (relational and object-oriented) have been developed and are well understood. However, new data models like XML and the ObjectRelational data model are not included in these mappings. Translation into XML is the focus of some research lately, but this is a relatively new area and not much work has been done yet. Translation into Object-Relational databases is a virtually unexplored topic, perhaps because it is felt that existing mappings into the (pure) relational models are easy to extend to this case. However, ObjectRelational databases provide options to the designer that are not available in the relational case. Therefore, some guidance is needed for the choices that appear in the mapping process. In this chapter, we review existing mappings and extend them to include these new data models. We start with a review of the basic concepts, to establish some vocabulary and make the chapter self-contained, followed by a description of recent work in the area, including new and existing translations. One of the purposes of the chapter is to put all of this research in a wider perspective and examine the different approaches, something that is missing from the current literature.

Background For lack of space, we do not discuss conceptual or data models in depth; we assume the reader is familiar with the basic ideas. However, we review some basic concepts to establish a vocabulary.

Conceptual Models We briefly review Entity-Relationship models and UML class diagrams. There are other conceptual models, like Object-Role-Modeling (ORM); we will not include them here. An Entity-Relationship (E-R) (Chen, 1976; Thalheim, 2000) model is based on three basic concepts: entity types, attributes and relationships. E-R models are usually depicted in E-R diagrams; an example is given in Figure 1. Entity

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

150 Badia

Figure 1. E-R diagram ssn

dname

name (1,1)

locations

(1,M)

Employee

street

Department (1,M) (1,M)

(0,M)

(0,M)

Date

city

address zip

reviews hourly rate

salary

started

manages pname

(1,M)

Hourly employee

Salaried employee

(1,M)

(1,M)

budget

Project

types are depicted as rectangles with a name inside, attributes as ovals, and relationships as lines with a diamond shape on them. Entity types represent things either real or conceptual. They denote sets of objects, not particular objects; in this respect they are close to classes in objectoriented models. The set of objects modeled by an entity type are called its extension; particular objects are called entities. Relationships are connections among entity types. Relationships may involve any number of entity types; those involving two entity types (called binary relationships) are the most common. However, n-ary relationships (involving n > 2 entity types) are also possible (in particular, relationships relating one entity type to itself are allowed). Relationships are fundamental in an E-R model in that they carry very important information, in the form of constraints: participation constraint tells us whether all objects in the extension of an entity type are involved in the relationship, or whether some may not be. For example, Department and Employee have a relationship Works-for between them. If all employees work for some department, then participation of Employee in Works-for is total. However, if there can be employees which are not assigned to a particular department, then participation is partial. Cardinality constraint tells us how many times an object in the entity type’s extension may be involved in a relationship, and allows us to classify binary relationship as one-to-one, one-to-many or many-to-many. There are several notations to state constraints in an E-R diagram. The one chosen here associates with each entity type E and relationship R a pair of numbers (min,max), where min represents the minimum number of times an entity in E appears in R (thus, min represents the

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

From Conceptual Models to Data Models 151

participation constraint by being 0 for partial and 1 for total), and max represents the maximum number of times an entity in E appears in R (thus, max represents the cardinality constraint by being 1 for one-to relationships and greater than 1 for many-to relationship; the latter case is traditionally represented by using the letters M or N). Thus, the (1,1) by Employee and Works-For indicates that all employees work for exactly one department; the (0,M) by Department and Manages indicates that not all departments manage projects, but those that do may manage more than one. Entity types and relationships may have attributes, which are properties with a value. Attributes convey characteristics or descriptive information about the entity type or relationship to which they belong. Attributes may be simple or composite (made up of simpler parts, like the attribute Address of entity Department in the example, which is made up of parts named street, city and zip), single or multivalued (being capable of having one or several values for a particular entity; multivalued attributes are displayed as dual ovals, like locations in our example, meaning that some department may have multiple locations), primitive or derived (a derived attribute value is computable from other information in the model). A key attribute is an attribute whose value is guaranteed to exist and be different for each entity in the entity type. Therefore, this attribute (primary key) is enough to point out a particular entity. All entity types are assumed to have at least one key attribute. A contentious issue is whether attributes are required (i.e., every entity of the type must have a value for each attribute of the type) or optional (i.e., some entities of the type may or may not have values for some attributes). Different authors take different views on this issue, some even arguing that it is a mistake to consider attributes optional (Bodart et al., 2001). Since this has an impact when transforming E-R models into different data models, we will point out how to deal with each view. Some E-R models admit weak entities, entities with no key attributes; these entities are connected by a one-to-many relationship to a regular entity, called the strong entity. What characterizes a weak entity is its lack of clear identity (reflected in the lack of a key) and its dependence for existence on the strong entity. As a typical example, entity Employee has an associated weak entity Dependent (shown as a double box). Clearly, an employee may be associated with several dependents (hence the one-to-many relationship), and if an employee is deleted from the model (say the employee is fired), then the associated dependents also go away. Many proposals for additional features have been made over the years. The most successful one is the addition of class hierarchies, by introducing IS-A (class/ subclass) relations between entities. This addition, obviously motivated by the success of object-oriented methods for analysis, allows the designer to recognize commonalities among entities; usually this means shared attributes exist. Shared Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

152 Badia

attributes are removed and put together in a new entity (class) that is a generalization of the others, and a class-subclass relationship is created. As in object-oriented approaches, inheritance of attributes is assumed. In Figure 1, entity type Employee has two subtypes, Hourly-employee and SalariedEmployee. The IS-A relationship is indicated by a downward triangle in the line joining the involved entity types. The IS-A relationship can be annotated to distinguish several situations: whether the subclasses are disjoint or not; and whether the subclasses together cover the superclass (i.e., every entity of the superclass must also belong to one of the subclasses) or not. Note that both dimensions are orthogonal to each other; hence, two annotations are needed to determine the exact situation. The Unified Modeling Language (UML) (Rumbaugh et al., 1999) is the conceptual model for object-oriented design. In its current incarnation, it is a large, complex model made up of several different parts. Here we concentrate on static diagrams, which are the diagrams explaining the structure of information in the system and its interconnections. The previous E-R diagram is given in Figure 2 as a UML diagram for illustration purposes. A static or class diagram has many similarities to an E-R diagram. The world is made up of classes, which are the equivalent of entity types. As in E-R models, classes are defined based on their attributes. Composite attributes can be modeled by a structured domain, a domain with an internal structure (as an example, see address in Department). In theory, a multivalued attribute requires its own separate class; however, modern versions of UML allow specifying a cardinality constraint on the link between attribute and class. This seemingly minor point is very important since it gives the model the flexibility to deal with optional attributes (by setting minimum cardinality to 0) and multivalued attributes in a uniform framework, similar to XML (however, this is far from an accepted norm; see next subsection). It is usual in class diagrams to give a data type for attributes, and it is possible to give them a default value. In UML, classes also have methods, procedural attributes which describe the behavior of the objects of the class. Methods, though, cannot be completely specified in the conceptual model because they are not declarative, but procedural (i.e., code). In a class diagram, classes are depicted as boxes divided into three parts: the top one contains the class name; the bottom one, methods; and the middle one, attributes. Relationships are also present in UML, although there they are called associations. Associations are displayed as lines connecting the involved classes. Associations can have attributes (displayed on a box related to the association by a dashed line). However, unlike relationships, associations cannot involve more than two classes. Therefore, if a relationship relates more than two classes, it must be reified into a class (which is then connected by binary associations to the related classes; see Review in Figure 2); the resulting class is sometimes Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

From Conceptual Models to Data Models 153

Figure 2. UML diagram Department

Employee ssn name

1..N

1..1

dname address:{street,

1..N

1..N

city,zip}

Reviews

1..1 0..N

0..N

1..N

1..N

Date

Hourly Employee hourly rate

Salaried Employee

salary

1..N

locations

Manages

Project pname budget

called an association class. Associations can have roles and cardinality and participation constraints, though. Two special kinds of association are aggregation and composition. Their semantics are somewhat vague; aggregation is supposed to represent a part-whole relationship, while composition is supposed to be somewhat stronger then aggregation, in that an object contains another one as a part. The main difference seems to be that objects that stand in a composition association have a lifetime dependence: when an object that contains another is destroyed, the contained objects must be destroyed, too (in some versions, composition may involve an arbitrary number of objects). Thus, what we called weak entities in the E-R model should be captured as a class associated by composition with another class. However, the semantics of UML are informal and fail to capture the special meaning that a part-whole relationship has (Moschnig-Pitrik et al., 1999). Aggregations are displayed with a hollow diamond, and associations with a filled diamond. Another special association is the one between class and subclass, called a generalization. This is displayed as an arrowhead.

Data Models Here we briefly review the relational, object-oriented, object-relational and semistructured (XML) data models. The relational data model (Date, 2004) is well known; we simply overview its main concepts here. A domain is a non empty set; intuitively, it provides a pool of values. Every domain is assumed to come with a name (an infinite number of names, technically). Given a schema or list of domain names R = {A1,…, A n}, Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

154 Badia

a relation on R is a subset of the Cartesian product A1 x …x An. The elements of the relation are called tuples. A key K for relation r in schema R is a subset of R (i.e., a set of attributes) such that, for any two tuples in r, they are the same if they have the same value for K. A relational database schema is a set of relation schemas plus a set of integrity constraints. An integrity constraint is a predicate over relations expressing a constraint; by far the most used integrity constraint is the referential integrity constraint. A referential integrity constraint involves two sets of attributes S1 and S2 in two relations, T1 and T2, such that one of the sets (say S1) is a key for one of the relations (called the primary key). The other set is called a foreign key if T2[S2] is a subset of T1[S1]. Referential integrity constraints are the glue that holds the relations in a database together. It is therefore necessary to specify, when talking about a relational database, which are the primary keys and the integrity constraints that are supposed to hold. There are many variations of the Object-Oriented data model. In this chapter, we use the ODMG (Object Data Management Group) model (Catell et al., 2000), and its language ODL (Object Database Language), since it sets the standard data model for Object-Oriented databases. In ODL, classes are declared by giving them an interface (using the keyword interface); each class is a collection of attributes that are either primitive (of a basic data type) or relationship, i.e., their value is an object or set of objects of a given class. The interface includes: a name, a key (optional), an extent, and the properties and operations. A key is a set of properties (attributes or relationships) whose values are unique for each object in the extent (composite keys are called compound in ODMG). The extent is the name of the set of objects under the declaration. Obviously, inheritance among classes is supported directly in the model, and declared using the keyword extends. Values of attributes are typically literals (atomic or complex) but can be oids. Values of relationships are always object names or a collection applied to object names. In the ODMG model, only binary relationships are explicitly represented, through a pair of inverse references (using the inverse keyword). Literal’s value may be simple or complex. There are three types of literals: atomic, collection, structured. Atomic literals correspond to basic data types: integers (long, short, unsigned), float (float, double), Boolean, single character, string, and enumeration types. Structured literals have a tuple structure; they include types like Date and Time. Also, the user can define structured literals as needed (using the Struct constructs). Collection literals specify a collection of objects or literals. Types of collections are: Set, Bag, List (homogeneous), Array and Dictionary. Each collection has a group of built-in operators. Finally, objects have object identifiers (oids) and a value (unlike literals, which have value but no oid). Objects may have a name, and can be of atomic or

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

From Conceptual Models to Data Models 155

collection type. Atomic objects are not objects without internal structure; they correspond to atomic or structured literals. For each object, properties (i.e., attributes and relationships) and operations are specified. The Object-Relational (also called extended-relational, or universal) data model is exemplified by the latest version of SQL, SQL-99 (Melton, 2003). It can be seen as an attempt to capture as many as possible of the object-oriented concepts introduced in the previous subsection and wrap them around a relational shell. A more modest view of it regards the model as extending the base of the relational model (instead of the model itself) by making it easier to add more complex data types to serve as domain definitions. Here we will describe the basics of the standard (Melton, 2003), since each commercial DBMS has its own version of the model, with different names and different syntax. The basic idea is to substitute domains by (possibly complex) types, called User Defined Types (UDTs). The name comes from the fact that the model provides constructors so that users can define their own types as needed by an application; the emphasis is in extendibility. UDTs come in two types, distinct types and structured types. Distinct types are based on a single built-in data type. Distinct types are not compatible with any other type (including the one they are based on). The following is an example of an UDT called age based on the built-in type integer: CREATE TYPE age AS INTEGER (CHECK age BETWEEN 0 and 100) FINAL; FINAL refers to whether type can be extended with subtypes: distinct types cannot, structured ones can. Optionally, operations and comparisons can be defined for types. Structured types can have internal structure, with parts called attributes. Attributes do not have to be built-in; they may be complex (SQL99 offers “built-in” structured types ARRAY and ROW), but cannot be of the type being defined (i.e., recursion is not allowed). Structured types do not have identity. One cannot specify constraints on the attributes of a structured type. It is possible to define a hierarchy of types by defining a new type UNDER another; (single) inheritance applies. Types may be NOT INSTANTIABLE (cannot have values of that type; this is used for abstract superclasses). Structured types can be used as columns of tables, and also as tuples of tables. A typed table is a table whose rows are of a structured type. The attributes of the type become attributes of the table. Typed tables also have inheritance hierarchies, corresponding to the hierarchies of the types. Subtables cannot have primary keys, but supertables can. They can also have UNIQUE, NOT NULL, CHECK and integrity constraints. There is also a self-reference value created automatically, unique for each row of the table. SQL99 provides REFERENCE

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

156 Badia

types, which give structured types an id. Maximal supertables must specify the self-referencing column (it is inherited by subtables, which cannot specify such a column on their own). References can be generated by system, by using some built-in type, or by using some attributes of the type. Semistructured data is assumed not to have a strict type, but to posses irregular, partial organization (Abiteboul et al., 1999). Because, in addition, the data may evolve rapidly, the schema for such data is usually large, dynamic, and is not strictly respected. Data models have been proposed that try to adapt to those characteristics. Several of them have in common that they can be thought of as directed, labeled graphs (trees, in some cases) (Papakonstantinou et al., 1995). In this model, nodes are objects, and edges point to other objects which are components of the given object. However, this model, while very general, gives very few constraints as to what the data looks like. In practice, some information about the components of objects is usually available. Thus, in most cases the term semistructured data is reserved for data that does have some (loose) structure. XML (Bray et al., 1999) provides the tools to describe such information, either through a DTD or an XML Schema (note, however, that XML has additional properties over semistructured data — order, attributes — while semistructured data does not assume the existence of any schema. Hence, both terms are, strictly speaking, different). Basically, in XML an object may have attributes and elements. Simple parts of an object can be represented by attributes or elements, while complex parts are represented by elements, since they in turn are also objects. Elements are defined by a pair of matching start and end tags. Elements may have embedded subelements, indicated by nested pairs of tags (empty elements are also allowed). A root element may be declared which is not a subelement of any other element. Subelements have associated information to indicate a cardinality constraint, in the form of a pair of attributes minOccurs and maxOccurs: none or several occurrences (a Kleene star, indicated by ‘*’ in DTDs) is represented by setting minOccurs to 0 and maxOccurs to ‘’unbounded’’; none or one occurrences (indicated by ‘?’ in DTDs) is represented by setting minOccurs to 0 and maxOccurs to 1; one or several occurrences (indicated by ‘+’ in DTDs) is represented by setting minOccurs to 1 and maxOccurs to ‘’unbounded’’. Finally, exactly one occurrence (no mark in DTDs) is represented by setting minOccurs and maxOccurs to 1. There is also a “choice” construct (indicated by ‘|’ in DTDs) which represents union of types. Attributes in XML are associated with types, not with elements, and are expressed as an equation attribute name = value within the starting tag of the corresponding element; the names must be unique within an element. Attributes can be mandatory (expressed with the keyword #REQUIRED) or optional (expressed with the keyword #IMPLIED). Keywords ID and IDREF are used to establish links among elements, denoting the pointed at and pointing elements in Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

From Conceptual Models to Data Models 157

the link, respectively. Atomic elements and attributes are simply of #PCDATA or CDATA type (this is a string, but is used to represent any atomic type: integer, ...). Attributes are only for complex types; for simple types, there are facets (restrictions on simple types). Since in XML, unlike HTML, tags are not pre-defined, one must have an idea of what tags one may find in a given documents. A Document Type Definition (DTD) or an XML schema can be used to give information about the components of an object. An XML document is well-formed when it respects the general XML syntax (nested, balanced tags), and is valid (with respect to a given XML schema or DTD) if it respects the tag definitions of that XML schema or DTD. DTDs do not distinguish between entity types and relationships; relationships are implicitly expressed through the element-subelement connection. Because relationships are attached to objects, XML DTDs are much better at handling hierarchical relations than many-to-many relations (or n-ary relationships (n > 2) or relationships with attributes), and indeed this is the way most DTDs are used (Sahuget, 2001). XML schemas generalize DTDs. An XML schema has XML syntax (start/end tags), and enforces type checking and references. It classifies elements as simple type (integers, strings) or complex type (regular expressions, like in DTDs). For a complex type, it defines the structure. An XML schema also introduces the notion of Namespace, a collection of tags and their definitions. There is a notion of key and foreign key. In XML Schema, a key for a complex type T is declared by giving it a name, a path selector (that leads from the root to T) and a field (to say which element(s) within T form the key). Note that a key can be made up by several elements; also, several alternative keys can be specified. Foreign keys are defined with a keyref keyword and a reference for the key being denoted, using also a path and a field selector. Therefore, integrity constraints can be specified in XML Schema. One can specify mixed content, in which plain text and tags are intertwined. There is also an Any type that allows anything. Finally, inheritance can be modeled with derived types. There are two ways to derive: by extension and by restriction. In the second, types are derived by giving a base type and restricting the properties (cardinalities, choices) of the elements.

Translations We divide the translations according to the target data model, as this creates the more salient differences. We first give an algorithm that uses the E-R model as source, and then point out the differences needed to handle UML.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

158 Badia

From E-R and UML to Relational There is a well-known procedure to transform an E-R diagram into a relational design, given in Figure 3. Applied to the E-R model of Figure 1, the algorithm creates the tables of Figure 4 (keys are underlined, foreign keys in italics). Intuitively, the algorithm maps every entity (including weak ones) to a table, and every relationship to a separate table. This algorithm has one notable property: if all the functional dependencies that exist among the attributes in the E-R model are those generated by the key, then all the relations generated by the algorithm are guaranteed to be in fourth normal form (Elmasri & Navathe, 2003). Even when additional functional dependencies exist, they usually involve attributes of a certain entity, and therefore normalization of the database can proceed on a table-by-table basis. There are a few tricky cases, though, in which the E-R model does not have enough information to guarantee normal form, usually involving n-ary relationships (Camps-Pare, 2001). If all attributes are considered required, then the above translation works correctly. If attributes are considered optional, the above translation is still correct but allows for the introduction of nulls in the database. When using SQL to define the tables, required attributes should be considered non-nullable, and optional attributes should be considered nullable. Unfortunately, without any further information (for instance, dividing the attributes into required and

Figure 3. From E-R to Relational For every entity E (including weak entities) do create table RE; if entity is weak, key of table is discriminator plus key of strong entity; else key of table is the key of entity; For every n-ary relation R between entities E1, E2, ..., En do create a table RR; for all entities E1,...,En do add as an attribute the primary key of Ei; declare it a foreign key for the table corresponding to Ei; declare as a key for RR the set of all Ei such that R is many on Ei. For every attribute A of E (or R) that is not multivalued do if A is atomic then add A as an attribute to RE (RR); else add one attribute to RE (RR) for every simple subpart of A ; if attribute is (part of) a key, declare it as (part of) a key. For every attribute A of E (or R) that is multivalued do create a new table RA E; add as attributes the key of RE (RR) and A; declare the key of E (R) as a foreign key for RE; declare the set of all attributes as the key of RA E.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

From Conceptual Models to Data Models 159

Figure 4. Applying method to Figure 2 Employee(ssn, name) Dependant(ssn, lname, date-of-birth) Department(dname,address-street,address-city, address-zip) Dept-Loc(dname,location) Project(pname, budget) Works-for(ssn, dname, start-date) Manages(dname,pname) Started(dname,pname, date) Reviews(ssn,pname,dname).

optional), the only way to get rid of nulls is to break up every table into a set of tables, each one containing the key of the entity and one of its attributes. This way, entities with no value for a given attribute simply would have no entry in the appropriate table, but no nulls would be produced. However, this approach produces a heavily partitioned database, with most queries requiring a large number of joins. Hence, the issue is usually ignored in the process. The above algorithm admits some inlining, as follows: when a binary relationship R exists between entities E1 and E2, and R is one-to-many or one-to-one, it is possible to express R not by creating a new table, but by modifying the table corresponding to the entity on the one side (if R is one-to-one, either entity will do). Say this entity is E1; then the key of E2 is added to the schema of E1, together with any attributes R may have. Note that if participation of E1 in the relationship is not total, this will result in some nulls; hence, this procedure must be used only when participation of E1 is total or nulls can be dealt with adequately. In the database of our example, we could create, instead of tables Employee and Works-for, a single table Employee(ssn,name,dname,start-date). The procedure must be extended to deal with IS-A links. If entity type E1 is a subtype of entity type E2, a table must be created for each. However, there are two options as to how to configure the relations: 1.

Add to the table for the subtype (E1) all attributes of the supertype (E2). That is, repeat all attributes of the schema of E2 in the schema of E1. This implies that objects that are in subclass E2 will be stored in its own table. Applied to the E-R model of Figure 1, this approach creates tables Employee(ssn,name), Hourly-Employee(ssn,name,hourly-rate), Salaried-Employee(ssn,name,salary). Note that if the two subclasses are a cover of the superclass, the table for Employee would not be necessary (would remain empty); but then, queries asking for all employees would have no direct reference. Note also that if there were overlap (although obviously not the case in this example), information about employees could be duplicated.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

160 Badia

2.

Give as the schema of the superclass (E2) only those attributes that are proper of E2 and in the subclass (E1) only those attributes that are proper of the superclass. This implies that objects which are in subclass E2 will be stored in two tables: in the table for E1, the inherited attributes will be taken care of; in the table for E2, we will put only those attributes that are proper of E2. Applied to the E-R model of Figure 1, this approach creates tables Employee(ssn,name), Hourly-Employee(ssn,hourly-rate), Salaried-Employee(ssn,salary). Note that if the two subclasses are a cover, the table for the superclass is still needed. Note also that if the subclasses overlap, there would be no redundancy on this schema.

Which option is better depends entirely on the intended use of the database, (i.e., the applications it must support) and on the characteristics of the class/subclass relationship (overlap, cover). Clearly, queries that refer to the whole superclass frequently will be more efficient over the first option, while queries that refer to specific classes more frequently will be more efficient over the second option. As for the translation from UML, the basis is exactly the same: each class is translated into a table, and each association is also translated into a table. Note that all associations in UML are binary, and hence no rule is needed to handle other kinds of associations. Note also that when associations have attributes, or have an arity greater than two, they get promoted to classes (the so-called association classes); in this case, the class and the associated associations should be translated as one since they represent a single relationship. As an example, the class Review in Figure 2 represents a ternary relationship, and is connected by three binary associations to classes Employee, Department and Project. A single table should be created, containing the attributes of class Review and representing the associations by adding the primary keys of Employee, Department and Project as foreign keys in the new class. Otherwise (if we treated association classes like other classes), the resulting database would be technically correct but would introduce artificial (vertical) partitioning of the database, which in turn would make querying more expensive by having additional joins to compute. Finally, in those cases in which attributes have cardinality constraints, we still follow the same procedure as for the E-R model, simply identifying attributes as multivalued if their max cardinality is greater than 1 (attributes whose min cardinality is 0 are considered nullable in the database, but must still be present in the table).

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

From Conceptual Models to Data Models 161

From E-R and UML to Object-Oriented Translations from UML to Object-Oriented databases are straightforward, since both are based on the same underlying model. Thus, we start by explaining the translation from UML into ODL. Intuitively, each class in UML will become a class in ODL, and each association will be captured as a relationship attribute with an inverse (recall that all associations in UML are binary). Complex attributes become structured attributes, and multivalued attributes can be captured with a SET constructor. Weak entities, although not recognized in UML, should be identified to make them a complex attribute of the class they depend upon. The reason is that, since composition is not recognized in ODL, this is the best way to keep the semantics of weak entities (their life dependency and lack of key). Association classes can be treated as regular classes, since ODL cannot deal directly with relationships with attributes or with arities greater than two. Inheritance, of course, is modeled directly both in UML and ODL. The procedure can be described in Figure 5. Note that this procedure does not involve object identifiers (oids) at all; it is expected that any objects created in the database will get an oid because that is part of the object-oriented model. Applied to the E-R model of Figure 1, this approach creates the classes shown in Figure 6. Note that some arbitrary data types were chosen to complete the declaration. Note also that key declarations are optional in ODL, but that regardless of key declaration every object of the class has an oid.

Figure 5. From UML to Object-Oriented For each class C declare a class interface C’ in the database; if class C is a subclass of another class, use the “extend” declaration in C’; for each attribute A of C add an attribute A of the appropriate data type to C’ (complex attributes are mapped to STRUCT attributes); For each association R between classes C1 and C2 if R has no attributes then declare a relational attribute R1 in C1 with value C2, declare a relational attribute R2 in C2 with value C1, declare R1 as an inverse of R2, and R2 as an inverse of R1; make R1 be a if R is many on C1; make R2 be a if R is many on C2; else declare a class interface R’ for R; add as attributes of R’ all attributes of R; create two binary relations, one between R’ and C1’, and another between R’ and C2’, as indicated above.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

162 Badia

Figure 6. Applying procedure to Figure 2 interface Employee (key ssn) { attribute string ssn; attribute string name; attribute set{string lname, Date date-of-birth} dependant; relationship Department works-for inverse Department::works-here; relationship Reviews reviewer inverse Reviews::reviews; } interface Hourly-Employee extends Employee { attribute float hourly-rate; } interface Salaried-Employee extends Employee { attribute float salary; } interface Department (key dname) { attribute string dname; attribute Struct {string street, string city, int zip} address; attribute set locations; Relationship set works-here inverse Employee::works-for; Relationship set Manages inverse Project::managed; Relationship set started inverse Start::started-by; Relationship Reviews reviewed-for inverse Reviews::reviews-for; } interface Project (key pname) { attribute string pname; attribute float budget; Relationship set managed inverse Department::Manages; Relationship set inverse Start:project-started; Relationship Reviews under-review inverse Reviews::is-reviewed;} interface Start { attribute date Date; Relationship set started-by inverse Department::started; Relationship set project-started inverse Project::start;} Interface Reviews { Relationship Employee reviews inverse Employee::reviewer; Relationship Department reviews-for inverse Department::reviewed-for; Relationship Project is-reviewed inverse Project::under-review;}

When starting with an E-R diagram, all of the above applies. The most important difference is that there will be n-ary relationships. To translate an n-ary relationship R involving entities E1,…, En, one must do the following: R must be reified, just like a binary relationship with attributes. Thus, a class R’ must be created, with attributes all attributes of R, and then n binary relations between R’ and each Ei’ must be established. The final result is the same as the procedure shown above, since the process of reification is basically the same transformation that takes place in UML when an association class is created to represent n-ary relations (n > 2).

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

From Conceptual Models to Data Models 163

To conclude, we point out that, contrary to popular belief, the object-oriented data model does not handle optional attributes any better than the relational model. Each class declaration specifies a series of attributes that every object in the class must have; hence, optional attributes in the conceptual model create undefined attributes (the equivalent of relational nulls) in the objects.

From E-R and UML to Object-Relational The object-relational data model offers several degrees of freedom with respect to the (purely) relational data model. These degrees of freedom, in turn, raise different possibilities about the translation from conceptual models. Unfortunately, almost no research has been done on possible different translations, their weaknesses and strengths, and their comparison. Here, we outline two different approaches to the translations and evaluate them. The first translation is based on the intuition that the real goal of the objectrelational data model is to allow for new data types in the relational model, without changing its structure. Under this view, a translation into objectrelational databases would be the same as a translation into a purely relational database, with the following differences:

•

subtypes of an entity type can be directly mapped into subtables of a table, since the object-relational approach supports inheritance directly.

•

multivalued atomic attributes can be modeled as an attribute with internal ROW or ARRAY constructor, at least if an upper bound in the number of values is known1. This will help reduce table fragmentation and will allow, in many cases, to map one entity under an entity class into one tuple in one table.

Because this is almost the same translation as the purely relational, we call this approach conservative. There is also another possible approach, which is more extreme, in that it uses the full-blown potential of the object-relational model. In the extreme approach, we treat the data model as one of objects represented in tables (i.e., in tabular form). Hence, we not only map subtypes into subtables and multivalued attributes into complex attributes, we also map weak entities into complex subelements of the strong entity. That is, given weak entity E1 related by a 1-M relationship to strong entity E2, we create a type for E1, and include, as an attribute of E2, a collection attribute representing an ARRAY or SET of the type created for E1. 1-M relationships can also be mapped by embedding the entities in the M side into the table for the entities in the 1 side. If the relationship has attributes, a table is

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

164 Badia

created that contains, on each tuple, the attributes of the relationship plus a complex type corresponding to the entity. This table is then embedded as a subtable (complex) attribute of the table representing the entity on the 1 side. Note that there may be further references to the embedded entity, since it may be involved in other relationships; these are handled by using REF types. Binary relationships without attributes can be implemented through REFs (or sets of REFs), in a manner similar to the object-oriented model. Note that this approach creates a series of choices as to how to represent manyrelationships and complex types: directly by embedding or inlining, or indirectly through references. Since this creates differences in how the relations are physically stored on disk, and therefore on access patterns, which option to use depends on the expected workload (queries), size and performance requirements of the database. As an example, consider the entity type Department, connected by a 1-M relationship to entity type Employee (i.e., a Department may have several Employees, each Employee belongs to a single Department). One option is to create the complex type for Department using the complex type for Employee as a (multivalued, complex) element. In this approach, there is only one table (Department), and queries asking about Department information (including Employees per Department) may run somewhat faster, since Departments are explicitly joined to Employees. However, queries about Employees may be somewhat slower (of course, the final performance depends on many other factors, like clustering, indices, etc.). This approach is somewhat biased towards Department (considered a “primary” type), in detriment of Employee (considered a “secondary” type). Moreover, if Employee is connected by another relationship to another entity, this second relationship has to be handled by reference — unless it is also 1-M. In our example, Employee is involved in a 1-M relationship with Dependant. Then it is possible to inline the dependant type within the Employee type, which in turn is within the Department type. Direct access to dependants may be slow and complex in this schema, though. However, Employee is also involved in a many-to-many-tomany relationship with Department and Project. Then it is not possible to inline this relationship since this would cause redundancy (a project would be repeated on each employee that reviewed it). In this case, using a separate Review table with references to Employees may be a better choice. The problem applies to any relationship that is many-to-many, which cannot be embedded without causing redundancy. In contrast, the original option is to create a complex type for Department, one for Employee, a table for each, and handle the relationship either with a separate table (as in the relational model), or by using references (sets of references to Employee on each Department, and a reference to Department on each Employee). This approach is more neutral, in that both Departments and Employees are considered “top level” tables. Further relationships with other

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

From Conceptual Models to Data Models 165

tables, like Project, are easier this way. On the other hand, every join between Departments and Employees needs support from separate indices and/or algorithms. Note that both approaches would create subtables HourlyEmployee(hourly-rate) and Salaried-Employee(salary), under table Employee. A possible heuristic to decide between the conservative and extreme approaches is this: the extreme approach works well for weak entity types, and for entity types connected only by a 1-M relationship to other entity types. However, the presence of M-N relationships strongly suggests the use of the conservative approach. Finally, we note that both approaches can handle required attributes but have problems with optional attributes. Since both approaches ultimately make object types fit into table schemas, the absence of a value creates a null in any case.

From E-R and UML to XML Mapping from E-R to XML is made complicated by the fact that XML is decidedly a hierarchical model while E-R is more of a flat model (all entities are at the same level) (Mani et al., 2001; Lee et al., 2003; Conrad et al., 2000; Dos Santos et al., 2001). This presents two options as to how to transform an E-R model into XML: to somehow transform the E-R model to fit the hierarchical nature of XML, or to create a flat model that can be expressed in XML Schema. The choice is similar to the conservative and extreme approaches outlined in the previous subsection. In the case of XML, the first option may be taken to an extreme by choosing some entity type to become the root of the XML model, and embedding everything under it. M-N relationships, relationships with attributes, and n-ary relationships (n > 2) are difficult to represent in this manner; hierarchical relationships are, after all, 1-M. A less extreme approach would be to identify some entity types in the E-R model which are somehow “more important” and use them as first elements under an artificial root; this is the approach used in Bird et al. (2000). The second approach is basically to use the E-R to relational translation, and express the resulting (flat) database in XML Schema. Thanks to its ability to represent foreign key-primary key relationships, XML Schema can faithfully represent the flat structure of a relational database. Here, we apply both approaches to our example and compare the results. The first option requires us to choose a “root of the hierarchy” entity. Assume that, when applied to the E-R model of Figure 1, this method chooses Employee as the root; this results in the DTD of Figure 7. Note that this approach is inherently biased. Also, many-to-many relationships are described as one-to-many, since this is the only type of relation compatible with the hierarchical organization. Moreover, attributes in relationships are lost,

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

166 Badia

Figure 7.

Dependant (lname,date-of-birth)> Department (dname, locations*, address)> Address (street, city,zip)> Project (pname,pbudget)>

too. Hence, this approach leaves much to be desired. The work of Bird et al. (2000) creates hierarchies when the structure is favorable (one-to-many relationships), but breaks it up in favor of a flat approach in other cases. The second approach takes the flat approach for all of the models, and it can be expressed in the algorithm of Figure 8. To start the algorithm, we declare one complex element D to be the root element of the schema; D represents the whole database. Then, one element is created per entity type and one per relationship; links are established through keys and foreign keys. Inheritance is modeled with derived types. Constraints on inheritance (exclusive or not, total covering or not), cannot be modeled in XML Schema at the moment. All attributes (optional, choice, and multivalued) are treated equally, since they can be handled in XML Schema. If an attribute is optional but not multivalued, we use ‘?’; if an attribute is optional and multivalued, we use ‘*’. Hence, XML is the only data model that can adequately deal with optional attributes. The decision to consider attributes as subelements instead of XML attributes is based on two factors: it allows for more parallelism with the relational model, and hence facilitates integration of the two; and it allows saving the XML attributes for metadata. However, clearly the option exists to treat E-R single-valued, simple attributes as XML attributes. As for UML, the whole process is simplified by the fact that all relations (associations) are binary. Hence, if an association is 1-to-many, we can consider a hierarchical organization, by making the entity on the many side a subelement of the entity on the 1 side. However, many-to-many associations still require two separate elements and the use of primary keys and foreign keys in XML Schema to be adequately modeled.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

From Conceptual Models to Data Models 167

Figure 8. From E-R to XML For every non-weak entity E in the model, declare XE as a subelement of D; For each attribute A of E, declare A as a subelement of XE, within a sequence type. Declare the primary key of XE. For each weak entity W in the model, declare XW as a subelement of the strong entity E; For each attribute A of W, plus the key of E, declare A as a subelement of XW, within a sequence type. Declare the primary key of XW as the discriminant of W plus the key of E. For each binary relation R between entities E and E’ with no attributes, include a foreign key reference to E in the (complex) type description of E’, include a foreign key reference to E’ in the (complex) type description of E. If the relationship is 1-M, or M-N, the many side needs to define the refkey (foreign key reference) element as one occurring multiple times (’+’, if participation is total; ’*’, if participation is partial) For each binary relationship R connecting entities E and E’, and attributes A 1,…,A n Create a new element ER. A dd each attribute A i to ER following the same rules used for entity types above1. Declare a primary key for ER Declare a binary relation between ER and E as above, using primary keys and foreign keys. Declare a binary relation between ER and E as above, using primary keys and foreign keys. For each n-ary relation R connecting entities E1,…,En, and possibly attributes A1,…,A n Create a new element ER. A dd each attribute A i to ER following the same rules used for entity types above1. For each Ei Declare a binary relation between ER and Ei as above, using primary keys and foreign keys. Declare a primary key for ER, by combining all foreign keys Ei such that R is many on Ei.

Conclusions and Further Research In this chapter, we have surveyed transformations between conceptual models and data models. These transformations are vital in the design and development of Information Systems. While some of them are well understood, new data models (like object-relational, and XML) create new challenges in developing satisfactory transformations. Clearly, the most important difference among transformations comes from differences among data models, in particular the importance given to objects and relationships: in the object-based paradigm objects (actually, classes) come first, and only binary relations (as special attributes of objects) are considered. These models have trouble handling n-ary relations (n > 2) and relations with attributes. XML, with its strong hierarchical view, also has problems at handling certain types of relationships. On the other hand, the relational data model transforms

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

168 Badia

everything into relations, and hence objects are diluted in that the information about an object may be distributed in several tuples of several relations. Finally, the entity-relationship conceptual model treats them both as equally important; this has probably contributed to their popularity, since many applications may be best described using both constructs. Another conclusion is that all data models, except XML, fail to accommodate irregular data. In particular, all models except XML cannot deal with optional attributes without resorting to nulls or other special values. The most important conclusion of this overview is that there is a severe lack of research in transformations for the object-relational and semistructured data models, as can be seen from the fact that several translations, with different advantages and drawbacks, are possible, and none is considered the right translation. The object-relational data model offers several possibilities for all but the most trivial conceptual models; the trade-offs among these options are not well understood. Hopefully, new research will shed light into these issues in the near future.

References Abiteboul, S., Buneman, P. & Suciu, D. (1999). Data on the Web: From Relations to Semistructured Data and XML. Morgan Kaufmann. Bird, L., Goodchild, A. & Halpin, T. (2000). Object Role Modeling and XML Schema. Proceedings of E-R 2000, Springer-Verlag (pp. 309-322). Bodart, F., Patel, A., Sim, M. & Weber, R. (2001). Should Optional Properties Be Used in Conceptual Modeling? A Theory and Three Empirical Tests. Information Systems Research, 12(4), 384-405. Bray, T., Paoli, J. & Sperberg-McQueen, C.M. (Eds.). Extensible Markup Language (XML) 1.0.W3C Recommendation (2 nd edition). Retrieved from the WWW: http://www.w3.org/TR/REC-xml-20001006 Camps-Pare, R. (2002). From Ternary Relationship to Relational Tables: A Case Against Common Beliefs. SIGMOD Record, 31(2), 46-49. Cattell et al. (Eds.). (2000). The Object Data Standard: ODMG 3.0. Morgan Kauffman. Chen, P. (1976). The Entity-Relationship Model-Toward a Unified View of Data. ACM Transactions on Database Systems, 1(1), 9-36. Conrad, R., Scheffner, D. and Freytag, J. C. (2000). XML Conceptual Modeling Using UML. Proceedings of ER 2000, Springer-Verlag, (pp. 558-571).

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

From Conceptual Models to Data Models 169

Date, C. (2004). An Introduction to Database Systems (8 th ed.). New York: Adisson-Wesley. Davis, F. (1993). Requirement Specification: Objects, Functions and States. New York: Prentice-Hall. Elmasri, R. & Navathe, S. (2003). Fundamentals of Database Systems (4th ed.). New York: Addison-Wesley. Lee, D., Mani, M. & Chu, W. (2003). Schema Conversion Methods between XML and Relations Models (Invited Paper). In B. Omelayenko and M. Klein (Eds.), Knowledge Transformation for the Semantic Web. IOS Press. Mani, M., Lee, D. & Muntz, R. (2001). Semantic Data Modeling Using XML Schemas. Proceedings of ER 2001, Springer, (pp. 149-163). Melton, J. (2003). Advanced SQL:1999. Morgan Kaufmann. Mostchnig-Pitrik, R. & Kaasboll, J. (1999). Part-Whole Relationship Categories and Their Application in Object-Oriented Analysis. IEEE Transactions on Knowledge and Data Engineering, 11(5), 779-797. Papakonstantinou, Y., Garcia-Molina, H. & Widom, J. (1995). Object Exchange Across Heterogeneous Information Sources. Proceedings of the 11 th International Conference on Data Engineering, IEEE Computer Society, (pp. 251-260). Rumbaugh, J., Jacobson, I. & Booch, G. (1999). The Unified Modeling Language Reference Manual. New York: Addison-Wesley. Sahuguet, A. (2001). Everything You Ever Wanted to Know About DTDs, But Were Afraid to Ask. Extended Abstract. Third International Workshop WebDB, LNCS, Springer, (pp. 171-183). dos Santos Mello, R. & Heuser, C. A. (2001). Structure-Based Conversion of a DTD to a Conceptual Schema. Proceedings of ER 2001, Springer, (pp. 133-148). Thalheim, B. (2000). Entity-Relationship Modeling. Springer-Verlag.

Endnote 1

Technically, the SQL-99 standard does not have a SET constructor, and hence does not have the ability to capture multivalued attributes (or manyrelationships). Indeed, without such a constructor the standard does not even have the ability to represent nested relations (Melton, 2003). How-

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

170 Badia

ever, it seems that SET is going to be added to the standard soon, so we consider it available here.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

An Algorithm for Transforming XML Documents Schema 171

Chapter VIII

An Algorithm for Transforming XML Documents Schema into Relational Database Schema Abad Shah, University of Engineering & Technology (UET), Pakistan Jacob Adeniyi, King Saud University, Saudi Arabia Tariq Al Tuwairqi, King Saud University, Saudi Arabia

Abstract The Web and XML have influenced all walks of lives of those who transact business over the Internet. People like to do their transactions from their homes to save time and money. For example, customers like to pay their utility bills and other banking transactions from their homes through the Internet. Most companies, including banks, maintain their records using relational database technology. But the traditional relational database technology is unable to provide all these new facilities to the customers. To make the traditional relational database technology cope with the Web and XML technologies, we need a transformation between the XML technology

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

172 Shah, Adeniyi, and Al Tuwairqi

and the relational database technology as middleware. In this chapter, we present a new and simpler algorithm for this purpose. This algorithm transforms a schema of a XML document into a relational database schema, taking into consideration the requirement of relational database technology.

Introduction An electronic document on the Web contains regular and irregular structures that may not be understood by users (Suciu, 1999; Abiteboul & Vianu, 1997; Brayan, 1997). Such a document (or data) is referred to as semistructured data (Suciu, 1999; Abiteboul, 1997). Contrary to the data in relational databases (RDBs), the semistructured data is stored without any schema or with a vague schema (Buneman, 1997; Suciu, 1999; Buneman, 1997). Beside the Web, there are many other sources of semistructured data, such as heterogeneous networking of integrated systems, file systems, electronic mail systems, digital libraries, etc. (Abiteboul, 1997; Buneman, 1997). The introduction of Extensible Markup Language (XML) as a standard data/ information representation has facilitated the publication of electronic data on the Web (W3C, 2003). This language also provides a hierarchical format for the data exchange over the Web with structure (St. Laurent, 1999; Bray, Paoli, Sperberg-McQueen, & Maler, 2002). Information in an XML document is represented as nested element structures, which start with a root element. An element can have an attribute or a sub-element (for further details about XML see W3C (2003) and Bray et al. (2002)). An XML document has an optional part, which is called Document Type Declaration/Description (DTD). A DTD of an XML document is considered as the schema of the XML document (W3C, 2003; Bray et al., 2002; Men-Hin & Fu, 2001). A relational database (RDB) has two main components, a schema and data files (or operational files) which are created according to the schema. As said earlier, a DTD is considered as a schema of an XML document, but there are noticeable differences between a RDB schema and an XML document schema (DTD). We give a complete comparison between them in Table 1. The basic difference between them is that a DTD represents a hierarchical structure whereas a RDB schema represents a relational (tabular) structure. We can consider XML documents schema analogous to the classical hierarchical data model. XML is considered as the best tool for representing and exchanging information on the Web (St. Laurent, 1999; Bray, Paoli, Sperberg-McQueen & Maler, 2002). The language allows users to define and also display data on the Web. These features make XML powerful and different from Hypertext Markup Language

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

An Algorithm for Transforming XML Documents Schema 173

(HTML) (Suciu, 1999; Comer, 2000). XML enables the user to define his own structures using the syntax of the elements in a DTD. A DTD describes the structure of information in an XML document in a hierarchical way (Bray, Paoli, Sperberg-McQueen & Maler, 2002). The structure of a DTD consists of elements which are further specified by attributes and/or sub-elements. Recursive and optional types of the sub-element can be defined using the operations * (zero or more times), + (one or more times),? (optional), and | (or). Many types of data value can be assigned to attributes, i.e., string-type or entity. The data value ANY means that an arbitrary declaration can be made by the programmer. An element in an XML document is uniquely identified by a special attribute ID. This unique attribute of an element can be regarded as the primary key of the element. As it has been mentioned in Table 1, a DTD does not support the concept of the composite ID (or key). An attribute can be referenced in another element through a field called IDREF, and it is a type-less attribute. The concept of an IDREF is similar to the concept of a foreign key in relational databases. There is no concept of a root of a DTD (Bray et al., 2002). Nowadays, financial organizations want to empower their customers so that they can perform their transactions from their homes through the Internet. For these financial organizations to provide their customers with this facility, it is necessary and beneficial that the databases (which are mostly RDBs) should be presented and processed in the XML format. We therefore need a technique that will process and transform an RDB and queries into an XML format and vice versa. This technique (or transformation) is essential because most of the commercially available database management systems (DBMSs) are relational DBMSs. This transformation will integrate and handle heterogeneous RDBs in the same manner. Researchers agree that the currently available RDB technology independently is not adequate to achieve this objective of using them on the Web without such transformation (Shanmugasundaram et al., 1999). Recently, some researchers have proposed algorithms for this transformation (Shanmugasundaram

Table 1. Comparison between RDB schema and DTD RDB (Schema) Tabular format It supports many data types. Schema and tuples of a relation are considered as two different entities, and they are stored in two different files. It is mandatory for a database. It is based on the rational data model. It supports the concept of a composite key. It supports the concept of foreign key. A schema of a relation is defined before creating its instances (or tuples)

XML Schema (DTD) Hierarchical format It supports only character data types. XML document and its DTD can be stored in the same file or in two different files. It is optional for a XML document especially for small XML documents. It is not based on any such data model. The concept of composite key is not supported. Does not support any such concept. Since it is optional, it can, therefore, be defined after the creation of a XML document.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

174 Shah, Adeniyi, and Al Tuwairqi

et al., 1999; Williams et al., 2000; Marni & Lee, 2002; Men-Hin, & Fu; 2001). In these transformation algorithms, most investigators have considered a DTD as a schema of the XML document, and they have used the tree data structure during the transformation. We know that the processes of creating and maintaining tree data structures are costly and affect the performance of the transformation process as pointed out by Shanmugasundaram et al. (1999). Also, there are many syntax options that are available for writing DTDs. Most of the existing transformation algorithms (from DTD into RDB schema) are unable to handle DTDs written in different ways (Shanmugasundaram et al., 1999; Men-Hin & Fu, 2001). In this chapter, we propose a different approach for transforming any DTD of an XML document into relational database schema. This approach can handle DTDs written in different ways and transform them into relational database schema in a simple and elegant way. The remainder of this chapter is organized as follows. We describe and analyze the existing approaches for transforming a DTD of an XML document into a relational database schema. We then present our proposed approach for transforming a DTD into a relational database schema, and demonstrate the proposed approach in a case study. Finally, we give our concluding remarks and future direction of this work.

Background Related Work Investigators have produced many different techniques for transforming DTDs into relational database schemas (Eisenberg & Melton, 2002; Yan, 2001; Chidlovskii, 2001; Marni & Lee, 2002; Men-Hin & Fu, 2001; Shanmugasundaram et al., 1999). There are three main issues that need to be handled during this transformation. These issues are: (i) the complexity of the DTD element specifications, (ii) the resolution of the conflict between arbitrary nesting in a DTD and relational schema, (iii) set-valued attributes and recursion (Shanmugasundaram et al., 1999). In the following paragraphs, we give a brief description of the works of these investigators and give, in Table 5 (at the end of this chapter), a comparison of these transformation approaches and our proposed approach. Shanmugasundaram et al. initially proposed an approach in the form of algorithms for transforming a DTD of an XML document into a RDB schema. MenHin and Fu later proposed an improvement to the algorithms (Shanmugasundaram et al., 1999; Men-Hin & Fu, 2001). Men-Hin and Fu proposed two algorithms, both of which work in the same way, except that they differ mainly in their Step 2 and Step 5. In Step 2 of the improved algorithm they gave more rules for

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

An Algorithm for Transforming XML Documents Schema 175

determining the roots. The transformation algorithm by Men-Hin and Fu works in six steps, and they are briefly given below:

•

Step 1: Simplifying the DTD: This step simplifies DTDs of XML documents using the rules similar to regular expression rules. The information that is useful in constructing schema prototype trees is preserved in the DTDs. The value types (e.g., #IMPLIED, #FIXED, etc.) for the character data (CDATA) are removed from the DTDs.

•

Step 2: Determining the Root Node: In this step, roots of the prototype trees are determined from the simplified DTDs using the set of rules that are suggested for this purpose.

•

Step 3: Constructing Schema Prototype Trees: The prototype trees are constructed from the roots that are determined in the previous step using a set of rules.

•

Step 4: Generating a Relational Schema Prototype: This step realizes a prototype relational database schema from the prototype tree using the following rules:

(i) Regard all the necessary attributes and elements in the simplified DTD as the attributes that are treated in the entity- relationship (ER) Model.

(ii) Inline all the necessary descendants of the schema prototype tree starting from the root. The necessary descendants refer to all the leaf nodes in the schema prototype tree, and the nodes marked with a “#” if they exist.

•

Step 5: Discovering Functional Dependencies (FDs) and Candidate Keys: In this step the traditional relational database design techniques are applied in order to produce suitable relational schemas. These design techniques reduce the redundancy and inconsistency in a relational database schema, and discover the functional dependencies and the candidate keys by analyzing the newly constructed relational database schema.

•

Step 6: Normalizing the Relational Schema Prototypes: This step applies the normalization rules on the relational database schema, after determining the FDs and candidate keys of a relational database schema in the previous step.

In the first algorithm of Men-Hin and Fu, functional dependencies are found in Step 5, first by analyzing the XML data, and then by applying the algorithm: Efficient discovery of functional and approximate dependencies using partitioning. Step 6 of this algorithm is time-consuming, according to Men-Hin and Fu. They modified this step to make the first algorithm more efficient (Men-Hin & Fu, 2001). The modified algorithm decomposes a DTD into small prototypes in

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

176 Shah, Adeniyi, and Al Tuwairqi

Step 4: Tree Construction, and Step 5: Generating a Relational Schema Prototype. The reason for the decomposition is to minimize the cost of finding functional dependencies. Both of these algorithms-the first and the modified algorithms- use the tree data structure in their transformation processes (or algorithms). The use of this data structure affects the performance of the transformation process because creating and maintaining the tree structure are costly procedures. Also, these two algorithms are unable to handle all types of DTDs as it has been mentioned (Shanmugasundaram et al., 1999; Men-Hin & Fu, 2001). Eisenberg and Melton (Eisenberg & Melton, 2001; Eisenberg & Melton, 2002) gave an informal proposal for a bi-directional transformation between an XML document and a RDB. This transformation can do a complete or a partial transformation at schema level as well as tuple level (or row level). The partial transformation may however miss some semantics. This draft for the bi-directional transformations also suggests a transformation of the data types. The authors did not give any proper formal algorithm for these transformations. It is, therefore, difficult to comment about the real effectiveness of these transformations. Williams et al. have proposed 18 steps for transforming DTD into a relational database schema and 11 steps for the reverse transformation (Williams et al., 2000). Both of these transformations do not use the tree data structure, but some steps in both of these transformations are unclear. For example, in Step 9 and Step 13 of the transformation of a DTD into a relational database schema, data types are assigned to attributes of a DTD without any explanation and justification. This type of vagueness in the transformation processes makes them difficult to understand and to draw any conclusion about their correctness and accuracy. Marni and Lee (2002) have proposed a process for transforming a DTD into a relational database schema using a regular tree grammar. The use of the regular tree grammar is helpful in maintaining semantics constraints in the transformation process. The theory of regular tree grammars provides a useful formal framework for understanding various aspects of XML schema languages. The two normal forms (NF1 and NF2) are used for the representation of regular tree grammars. NF1 is used in the XML document validation process, and to check whether a given XML schema satisfies the structural constraints imposed by the schema languages. XML Schema (or DTD) provides several unique data modeling features such as union type “+”, which does not exist in the traditional database models, such as the relational database model. In Marni and Lee (2002), NF2 is used for representing the basic of the conversion definition of the two schema languages, that is, a schema that supports union types (e.g., XML-Schema Language), and a schema language that does not support union types (e.g., SQL). This conver-

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

An Algorithm for Transforming XML Documents Schema 177

sion definition is used as the first step in this transformation process of XML schema into relational database schema. The entities and relationships, which form the basic items of data modeling, are represented as elements and attributes of a DTD. The process of mapping XML schema (or DTD) into RDB schema has several issues, and they have been pointed out (Marni & Lee, 2002). One of the most important among them is the semantic constraint which exists in the XML model. Since relational database schema cannot express these constraints in the XML schema languages, a useful and meaningful subset of those constraints should therefore be found in the mapping process. This process of finding the subset needs simplification of an XML schema. The concept of inlining technique is used for generating an efficient relational schema (Marni & Lee, 2002). However, the inline technique that is presented in this work generates a huge number of relations. In addition, this work does not present any proposal for assigning data types to attributes of tables after or during the transformation process. The transformation process of an XML DTD to relational data schema is the mapping of each element in the DTD to a relation, and it maps the attributes of an element to the attributes of the relation. However, there is no correspondence between elements and attributes of DTDs and entities and attributes of ER model. The attributes in an ER model are often represented as elements in a DTD. The following DTD definition illustrates this issue of the transformation.





In the ER model, author would be taken as an entity and firstname, lastname and address would be taken as the attributes of the entity. But in defining a DTD, there is no incentive to consider author as an element and firstname, lastname, and address as attributes. In the syntax of a DTD, if firstname and lastname were defined as attributes, then they could not be nested under name because DTD attributes cannot have a nested structure. A direct mapping of elements to relations therefore leads to an excessive fragmentation.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

178 Shah, Adeniyi, and Al Tuwairqi

Proposed Transformation Approach Our proposed transformation approach (or process) works in two steps. The first step takes a DTD written in any form using the syntax available for writing DTDs and transforms it into a standard form that is devised keeping in view the syntax of a relational database schema. The second step of our proposed approach takes a DTD in the standard form and transforms it into a relational database schema. In our proposed approach, we do not include the processes of finding functional dependencies (FDs), assigning data types to attributes of relations after the transformation (note that DTD supports only one data type, i.e., PCDATA), and normalization of the relation schema. The reason for this is that these three processes are database design techniques and they depend on the perception of a database designer about the database schema (Elmasri & Navathe, 2000). These processes need a direct or indirect expert’s intervention and decision. Therefore, we separate these three processes from the actual transformation process, i.e., DTD in relational schema. In other words, these three processes are difficult to automate. Consequently, we have separated the processes that should be done manually from the processes that can be automated. This separation makes our algorithm simpler and has helped in our objective to transform any DTD into a relational schema. In this section, we propose only those processes of the transformation that can be automated. The proposed approach consists of two algorithms (or steps): (i) Pre-Processing Algorithm and (ii) Transformation Algorithm. Figure 1 shows a general view of our proposed transformation process. The first algorithm (i.e., Pre-Processing Algorithm) transforms any DTD that is written in any form (referred to as DTD a), into the standard DTD that is referred to as DTD s (Figure 1). The main objective of Pre-Processing Algorithm is to transform a DTD into a simple uniform and standard form. The second algorithm, (i.e., Transformation Algorithm), transforms a DTD in this standard form into an RDB schema (Figure 1).

Figure 1. General view of the algorithm DTD a

DTD s Pre-Processing Algorithm

RDB_Schema Transforming Algorithm

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

An Algorithm for Transforming XML Documents Schema 179

Pre-Processing Algorithm As it has been mentioned earlier, there is no standard and fixed method for writing a DTD of an XML document. In other words, different users can write DTDs in different ways using the options provided by the syntax of DTD. The main objective of the Pre-processing is to enable the overall transformation process to handle DTDs that are written in different ways. Hence, the main function of this algorithm is to transform DTDs written in different forms into a uniform and standard form. The output of this algorithm is the standard DTD denoted as DTD s, and it is used as the input to the Transformation Algorithm (Figure 1). Now we summarize the objectives of the standard DTD and give its format (or syntax). The main objectives of the standard DTD are listed below: (i)

to provide input to Transformation Algorithm in a uniform and standard form; and

(ii)

to enable the overall transformation process handle DTDs written in different ways.

The proposed syntax/format of the standard DTD is given in Figure 2. In the figure, the operators ? and * mean zero or one, and zero or more, respectively. The structure given in Lines 2-7 in Figure 2 can be repeated until it produces a standard DTD and the number of repetitions depends on the corresponding XML document structure. For example, in the standard Library DTD given in Figure 3, this structure (i.e., Lines 2-7 in Figure 2) is repeated four times. In the figure, we have given a blank line after each repetition of the structure. The steps in the

Figure 2. Format of the standard DTD (DTDs) 1.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

180 Shah, Adeniyi, and Al Tuwairqi

Figure 3. Example of Library DTD in the standard DTD format



ID

#REQUIRED

PAddr

CDATA

#FIXED

PCity

CDATA

#FIXED>

ID

#REQUIRED

Name

CDATA

#FIXED

Addr

CDATA

#FIXED

City

CDATA

#FIXED>

Books





Loans

Pre-Processing Algorithm are given in Figure 4. The algorithm takes any DTD (DTDa) and transforms it into the standard DTD (DTD s). In the Global Schema Extraction Algorithm (Men-Hin & Fu, 2001), elements and attributes are treated equally and represented as separate nodes in the prototype tree. The same approach is used in the DTD-splitting Schema Extraction Algorithm (Men-Hin & Fu, 2001) and Basic Inlining Algorithm (Shanmugasundaram et al., 1999). But our proposed Pre-Processing Algorithm

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

An Algorithm for Transforming XML Documents Schema 181

Figure 4. Pre-processing algorithm Algorithm Pre-processing (DTDa):DTDs (i)Scan DTD_Name in DTDa and make it DTDs_Name of DTDs and add 0 THEN Add Add

’ ELSE add main_elementi to DTDs as its;

/* it means m = 0; that is for the two cases: (i)

,and (ii) */

GOTO attribute; END FOR LOOP (vii)

/* outer loop */

add ‘]>’;

(viii) RETURN (DTDS); (ix)END Algorithm;

transforms all sub-elements of a main element into attributes of that main element.

Transformation Algorithm The Transformation Algorithm takes the DTDs of an XML document as an input and transforms it into a relational database schema (RDB_Schema). In Figure 5, we give the steps in the algorithm. In this algorithm, there are two nested loops.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

An Algorithm for Transforming XML Documents Schema 183

Figure 5. Transformation algorithm Algorithm Transforming (DTDs): RDB_Schema (i)

Scan Root_element of DTDs make it as Schema_name of RDB_schema;

(ii) Find total number of main_elements in DTDs (say they are n); (iii)IF n = 0 THEN EXIT(); ELSE FOR i = 1 to n Transform main_elementi into Tablei and give it the name of main_elementi; Find total number of attributes in main_elementi IF m = 0 THEN

(say they are m);

i++

ELSE FOR j = 0 to m Scan attribute_namej; IF attribute_namej is of type ID THEN make attribute_namej as primary key of Tablei ; ELSEIF attribute_namej is of type IDREF THEN make attribute_namej as a foreign key of Tablei ; ELSE make attribute_namej a normal attributej of Tablei; END FOR LOOP; /* inner loop END FOR LOOP; /* outer loop

*/

*/

(iv) RETURN (RDB_Schema); (v)

END Algorithm

The outer loop deals with elements of the DTDs and transforms them into corresponding tables/relations. The inner loop transforms every attribute of the element into the attributes of the corresponding relation. In Step (iii) of the algorithm (Figure 5), it transforms ID and IDREF attributes of an element into the primary key and foreign key of the relation, respectively. Note that since the syntax of DTD does not support the concepts of composite key, our proposed transformation process, therefore, does not support this concept.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

184 Shah, Adeniyi, and Al Tuwairqi

Case Study: Library DTD In this section, we take a DTD that is written in a general form and demonstrate the workings of our proposed transformation process. For this purpose, we take a Library DTDa of a Library XML document given in Figure 6. We first apply Pre-Processing Algorithm that takes the DTD given in Figure 6 as input and produces a DTD s in the standard form as its output given in Figure 3. After applying Pre-Processing Algorithm to the Library DTDa the complete trace and step-wise working of the algorithm are given in Table 2 and Table 3 respectively. In Table 3 we give the transformation of the markup statements of the DTDa, and the output of the Pre-Processing Algorithm on Library DTDa is given in Figure 3. After pre-processing of the Library DTDa and transforming it into the standard DTD s, then Transformation Algorithm is applied to DTDs. This algorithm transforms the Library DTDs (Figure 3) into a Library RDB-Schema given in Figure 6. DTDa of the Library XML document 1)

4)

5)

7)

9)

PName idref IDREF #REQUIRED>

LCNo id ID #REQUIRED>

10) 11) 12)

13) 14) 15) 16) 17) 18) 19) 20) 21) 22) 23) 24) 25)

26) ] >

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

An Algorithm for Transforming XML Documents Schema 185

Table 2. Trace of the pre-processing algorithm for Library DTDa Root_element

Library

n

i

Main_element

m

j

Books

4

4 1

2

Publisher

3

3

Borrowers

4

4

Loans

3

sub-elements

ID

IDREF

Lno

1

title

N

N

1

2 3 4 1 2 3 1 2 3 4 1 2 3

author PName LCNo PName PAddr PCity Name Addr City CardNo CardNo LCNo Date

N N Y Y N N N N N Y Y N N

N N N N N N N Y N

2 3 4 5 6 7 8 9 10 11 12 13 14

Table 3. Transformation of the markup statements Lno 1 2 3 4 5 6 7 8 9 10 11 12 13 14

title author PName LCNo Pname PAddr PCity Name Addr City CardNo CardNo LCNo Date

The Markup Statement CDATA #FIXED CDATA #FIXED IDREF #REQUIRED ID #REQUIRED ID #REQUIRED ID #REQUIRED CDATA #FIXED CDATA #FIXED CDATA #FIXED CDATA #FIXED ID #REQUIRED ID #REQUIRED IDREF #REQUIRED CDATA #FIXED

Table 4. Trace of transformation algorithm Schema_name

n

i

Main_element

Table

m

j

Attribute

ID

IDREF

Library

4

1

Books

Table_Books

4

1

title

N

N

2

author

N

N

3

PName

N

Y

4

LCNo

Y

-

1

PName

Y

-

2

PAddr

N

N

3

PCity

N

N

1

Name

N

N

2

Addr

N

N

3

City

N

N

4

CardNo

Y

-

2

3

4

Publisher

Borrowers

Loans

Table_Publisher

Table_Borrowers

Table_Loans

3

4

3

1

CardNo

Y

-

2

LCNo

N

Y

3

Date

N

N

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

186 Shah, Adeniyi, and Al Tuwairqi

Table 5. Comparative study of our approach with some existing approaches Basic Inlining

SI&HI *

Global Extracting

DTD-splitting Extracting

Williams et al.

Regular Tree Grammars

Our Approach

Data Structure Used

graph

graph

tree

tree

relational structure

regular tree grammars

no such abstract data structure is used

Operators Handling

creates a relation for every element in the DTD

ensures that each element is represented only once in a relation

eliminates operators from DTD

preserves some operators to preserve some sub-elements occurrences

some rules are specified to handle them

support XMLschema (not DTD)

pre-pressing algorithm processes them

the common elements are shared

actual data fields are available in relational schema

number of attributes of the schema is less than the algorithms basic inlining

preserves entities and definitions

maintains semantics constrains

simple, direct mapping & maintains the semantics of RDB

large no. of relations it creates

large number of joins in mapping for particular elements

number of possible minimal dependencies is exponential

works with limited number of elements and attributes

some rules of the mapping are vague such as assigning of data types to attributes

low

low

low

low

didn’t mention

handles Advantage

fragmentation

Problem **

Disadvs.

✦

Prfce✦✦

a complex mapping process

data types assigning is with human intervention

didn’t mention

high (expected)

*. A direct mapping of elements to relations leads to excessive fragmentation of attributes (for more details see (Marni, M & Lee, D. 2002 )). ** Performance.

).Disadvantages. )). Performance.

Figure 7. The trace of the algorithm is given in Table 4. In Figure 7, attributes, which are underlined, and those that are bold and underlined, denote primary keys and foreign keys, respectively. Note that all attributes of the relational database schema given in Figure 7 are typeless attributes and they can be given data types through user intervention. This issue and some other issues related to this work are identified in the section on Future Direction.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

An Algorithm for Transforming XML Documents Schema 187

Figure 7. RDB schema for Library DTDs Table_Books title

author

PName

LCNo

Table_Publisher PName

PAddr

PCity

Table_Borrowers Name

Addr

City

CardNo

Table_Loans CardNo

LCNo

Date

A Few Words About Automation We have automated the proposed transformation process by implementing both its modules, i.e., Pre-Processing Algorithm and Transformation Algorithm. We have used the programming language Delphi Version 7 as the implementation language. The first module parses the input DTD, checks its syntax and finds out syntax errors (if any), and transforms the input DTD into the standard form. The second module transforms the standard DTD in a relational schema.

Concluding Remarks and Future Directions In this chapter, we have given a new transformation algorithm (or an approach) to transform the DTDs (of an XML document) that are written in different syntax options, into a relational database schema. This transformation process consists of two algorithms: (i) Pre-Processing Algorithms, and (ii) Transformation Algorithm. We have defined: (a) the format of a standard DTD that is closer to a relational database schema, (b) Pre-Processing Algorithm that transforms any DTD (that is written in any form) of an XML document into this standard format, and (c) the Transformation Algorithm that transforms the standard DTD of the original DTD into a relational database schema. We have demonstrated the complete working of our proposed algorithm using a case study. Our proposed approach is simpler, formal and different from the existing approaches that have been suggested by Shanmugasundaram et al. (1999), and others (MenHin & Fu, 2001; Williams et al., 2000; Eisenberg & Melton, 2001; Eisenberg &

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

188 Shah, Adeniyi, and Al Tuwairqi

Melton, 2002). Most of these approaches use the tree data structure, which requires costly operations in creating and maintaining it. Our approach does not use a tree data structure and this makes it simpler and more efficient than those approaches that use the tree data structure. We have also given a brief comparative study of our proposed approach and some existing approaches in Table 5.

Future Directions We are actively working on the complete and comprehensive implementation of the proposed transformation algorithm. We have briefly mentioned some of the results of our initial implementation in this chapter. After a complete implementation of our proposed algorithm, we will analyze the results of our transformation process, and compare them with the results of others. We are also working on making our proposed algorithm to be bi-directional so that our new algorithm will transform relational databases schema into a DTD as well. We also suggest that investigation should also focus on issues that are related to the bi-directional transformation (i.e., XML documents schema into RDB Schema, and vice versa). These issues are: (i)

Handling of IDREFS: It can be an interesting study to translate the concept of IDREFS into the relational paradigm.

(ii)

Naming conflict between relations and attributes during the transformations.

(iii) Assigning data types to attributes, because DTD supports only character data type and RDB schema supports multiple data types. Upon the completion of the transformation process, we get typeless RDB schema. It therefore needs further study on when and how data types should be assigned to attributes of tables.

References Abiteboul, S. (1997). Querying semi-structured Data. Proceedings of International Conference on Database Theory. Abiteboul, S., Buneman, P. & Suciu, D. (2000). Data on the Web. CA: Morgan Kaufmann. Abiteboul, S. & Vianu, V. (1997). Querying the Web. Proceedings of the ICDT.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

An Algorithm for Transforming XML Documents Schema 189

Bray, T., Paoli, J., Sperberg-McQueen, C. M. & Maler, E. (2002). Extensible Markup Language (XML). Second Edition. Retrieved from the WWW: http://www.w3c.org/TR/REC-xml Brayan, M. (1997). SGML and HTML Explained (2nd ed.). New York: Addison Wesley. Buneman, P. (1997). Semi structured data. Proceedings ACM Symposium on Principles of Database Systems, (pp. 117-121). Chidlovskii, B. (2001). Schema Extraction from XML Data: A Grammatical Inference Approach. Proceedings of the Eighth International Workshop on Knowledge Representation meets Databases (KRDB2001), Italy. Comer, D. (2000). The Internet Book (3rd ed.). NJ: Prentice Hall. Eisenberg, A. & Melton, J. (2002). SQL/XML is Making Good Progress. ACM SIGMOD Record, 31(2),101-108. Eisenberg, A. & Melton, J. (2001). SQL/XML and the SQL Information Group of Companies. ACM SIGMOD Record, 30(3), 105-108. Elmasri, R. & Navathe, S. (2000). Fundamental of Database Systems (3rd ed.). Benjamin/Cumming Publishing. Marni, M. & Lee, D. (2002). XML to Relational Conversion using Theory Regular Tree Grammars. Proceedings of the 28th VLDB Conference, Hong Kong. Men-Hin, Y. & Fu, A. (2001). From XML to Relational Database. Proceedings of the Eighth International Workshop on Knowledge Representation meets Databases (KRDB2001), Italy. Shanmugasundaram J., Tufte, K., Zhang, C., DeWit, D. & Naughton, J. (1999). Relational Database for Querying XML Documents: Limitations and Opportunities. Proceedings of the 25 th VLDB Conference, Edinburgh, Scotland (pp. 302-314). St.Laurent, S. (1999). XML A Primer (2nd ed.). CA: M&T Books. Suciu, D. (1999). Semi Structured Data and XML. Proceedings of ACM SIGMOD (pp. 430-441). Williams, K. et al. (2000). Professional XML Databases. UK: Wrox Press. World Wide Web Consortium (W3C): Extensible Markup Language (XML), http://www.w3.org/XML. Yan, M. (2001). From XML to Relational Database. M. Phil Thesis, Chinese University of Hong Kong, Dept. of Computer Science.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

190 Ma

Chapter IX

Imprecise and Uncertain Engineering Information Modeling in Databases: Models and Formal Transformations Z. M. Ma, Université de Sherbrooke, Canada

Abstract Computer-based information systems have become the nerve center of current manufacturing systems. Engineering information modeling in databases is thus essential. However, information imprecision and uncertainty extensively arise in engineering design and manufacturing. So contemporary engineering applications have put a requirement on imprecise and uncertain information modeling. Viewed from database systems, engineering information modeling can be identified at two levels: conceptual data modeling and logical database modeling and correspondingly we have conceptual data models and logical database models, respectively. In this chapter, we firstly investigate information imprecision and uncertainty

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Imprecise and Uncertain Engineering Information Modeling

191

in engineering applications. Then EXPRESS-G, which is a graphical modeling tool of EXPRESS for conceptual data modeling of engineering information, and nested relational databases are extended based on possibility distribution theory, respectively, in order to model imprecise and uncertain engineering information. The formal methods to mapping fuzzy EXPRESS-G schema to fuzzy nested relational schema are developed.

Introduction Nowadays computer-based information systems have become the nerve center of current manufacturing systems. Engineering information modeling in databases is thus essential. From the point of view of database systems, engineering information modeling can be identified at two levels: conceptual data modeling and logical database modeling. Correspondingly, we have conceptual data models and logical database models for engineering information modeling, respectively. Product data models, for example, can be viewed as a class of conceptual data models that take into account the needs of engineering data (Shaw, Bloor & Pennington, 1989). Much attention has been directed at conceptual data modeling of engineering information because conceptual data models can capture and represent rich and complex semantics in engineering applications at a high abstract level. Limited by the power of traditional ER/EER (Chen, 1976) in engineering modeling, the International Organization for Standardization (ISO) has developed the Standard for the Exchange of Product Data (STEP, ISO 10303) in order to define a common data model and procedures for the exchange of information. STEP provides a means to describe a product model throughout its life cycle and to exchange data between different units. STEP consists of four major categories: description methods, implementation methods, conformance testing methodology and framework, and standardized application data models/schemata. EXPRESS (Schenck & Wilson, 1994), being the description methods of STEP and a conceptual schema language, can model product design, manufacturing, and production data and EXPRESS model hereby becomes a major one of the conceptual data models for engineering information modeling (Eastman & Fereshetian, 1994). Note that, however, not being the same as ER/EER and IDEF1X, EXPRESS is not a graphical schema language. In order to construct an EXPRESS data model at a higher abstract level, EXPRESS-G is introduced as the graphical representation of EXPRESS. Here EXPRESS-G can only express a subset of the full language of EXPRESS. In addition to EXPRESS-G, it is also suggested in STEP that IDEF1X or ER/EER can be used as one of the optional languages for EXPRESS data model design. Then EXPRESS-G, IDEF1X, ER/EER, or even UML data

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

192 Ma

model can be translated into EXPRESS data model. As to engineering information modeling in database systems, most of generic logical database models such relational databases, nested relational databases, and object-oriented databases can be used. Also some hybrid logical database models (e.g., object-relational databases) or some special databases based on these database models (active databases, deductive databases, and constraint databases) are very useful for this purpose. It should be noticed, however, that information is not always crisp in real applications. Imperfect Information exists extensively in engineering applications. Imprecision is most notable in the early phase of the design process and has been defined as the choice between several design alternatives. In addition to product design, imprecise and uncertain information can be found in other activities related to engineering. Concerning the representation of imprecision and uncertainty, attempts have been made to address the issue of imprecision and inconsistency in design by employing intervals (Kim et al., 1995). Other approaches to representing imprecision in design include using utility theory, implicit representations using optimization methods, matrix methods such as Quality Function Deployment, probability methods, and necessity methods. An extensive review of these approaches was provided in Antonsoon and Otto (1995). All of these methods have had limited success in solving design problems with imprecision. Currently, modeling fuzzy engineering information in conceptual data models and logical database models is receiving increasing attention. To model imprecise and uncertain information in database models, many traditional database models have been extended. Being the most common database model, the relational model has been extended to handle null values, fuzzy values, partial values, intervals, and probabilistic values. It should be noted that it is very difficult for a relational data model based on a flat relational table to model the complex structure and rich semantics of real-world objects. Therefore, the database models for modeling imprecise and uncertain information have increasingly been focused on the object-oriented database model (Bordogna, Pasi & Lucarella, 1999; George et al., 1996; Gyseghem & Caluwe, 1998; Vila et al., 1996) and the nested (NF2) relational database model (Yazici et al., 1999). Also some attention has been paid to information uncertainty and imprecision modeling in conceptual data models such as the entity-relationship (ER) model (Zvieli & Chen, 1986), extended entity-relationship (EER) model (Chen & Kerre, 1998), and IFO model (Vila et al., 1996; Yazici et al., 1999). But EXPRESS has no the capacity of modeling imprecise and uncertain information, although imprecise and uncertain information exists extensively in engineering applications. A classical database is generally created by first developing a high-level conceptual data model and then mapping this conceptual model into an actual

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Imprecise and Uncertain Engineering Information Modeling

193

implementation. The ER model, for example, is extensively used for the conceptual design of relational databases (Teorey, Yang & Fry, 1986). Similarly, engineering information modeling generally starts from conceptual data models and then the developed conceptual data models are mapped into logical database models. But less research has been done in developing design methodologies for implementing fuzzy databases, although some fuzzy conceptual data models and logical database models have been developed. It is especially true for fuzzy engineering information modeling in databases. Extending EXPRESS-G to model fuzzy information in a product data model was initially presented by Ma et al. (2000), where some main concepts and graphical representations were given. Modeling imprecise information, however, was not considered (Ma et al., 2000). In particular, the logical database model for product data with imprecise and uncertain information and its design methodology were not provided. In this chapter, we concentrate on a fuzzy nested relational database model and fuzzy EXPRESS-G model. After introducing the basic knowledge on fuzzy set and possibility distribution theories, EXPRESS-G, and nested relational databases, we extend EXPRESS-G and nested relational databases using possibility distribution theory, respectively, so that fuzzy engineering information can be modeled at two levels: conceptual data model and logical database model. Based on the investigation that nested relational databases can support an EXPRESS model, we develop the formal methods to mapping from fuzzy EXPRESS-G to fuzzy nested relational databases. The remainder of the chapter is organized as follows: The basic knowledge about possibility distribution theory, EXPRESS-G, and nested relational databases are given. Based on possibility distribution theory, EXPRESS-G and nested relational databases are extended. The formal methods to mapping fuzzy EXPRESS-G schema to fuzzy nested relational schema are developed. The chapter then concludes.

Basic Knowledge Information Imprecision, Uncertainty and Possibility Distribution Theory Inconsistency, imprecision, vagueness, uncertainty, and ambiguity are five basic kinds of imperfect information in database systems (Bosc & Prade, 1993; Klir & Folger, 1988). Intuitively, imprecision and vagueness are relevant to the content of an attribute value, and it means that a choice must be made from a

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

194 Ma

given range (interval or set) of values, but we do not know exactly which one to choose at present. For example, “between 20 and 30 years old” and “young” for the attribute Age are imprecise and vague values, respectively. In general, vague information is represented by linguistic values. Imprecise values generally denote range-values with the form [ail, ai2, ..., aim] or [ai1-ai2] for the discrete and continuous universe of discourse, respectively, meaning that exactly one of the values is the true value for the single-valued attribute, or at least one of the values is the true value for the multivalued attribute. Imprecise information hereby has two interpretations: disjunctive information and conjunctive information. In addition to range values, there exists one special kind of imprecise information, namely, null values (Codd, 1987; Motor, 1990; Parsons, 1996; Zaniola, 1986). Information uncertainty is related to the truth degree of its attribute value, and it means that we can apportion some, but not all, of our belief to a given value or a group of values. For example, the sentence, “I am 95 percent sure that Tom is married” represents information uncertainty. The random uncertainty, described using probability theory, is not considered in this chapter. Generally speaking, several different kinds of imperfection can co-exist with respect to the same piece of information. For example, that it is almost sure that John is very young involves information uncertainty and vagueness simultaneously. Imprecision, uncertainty, and vagueness are three major types of imperfect information. Vagueness and uncertainty can be modeled with possibility theory (Zadeh, 1978). Therefore, we mainly concentrate on fuzzy extension of EXPRESS-G model and fuzzy nested relational databases in this chapter. Fuzzy data is originally described as fuzzy set by Zadeh (1965). Let U be a universe of discourse, then a fuzzy value on U is characterized by a fuzzy set F in U. A membership function µF: U → [0, 1] is defined for the fuzzy set F, where µF (u), for each u ∈ U, denotes the degree of membership of u in the fuzzy set F. Thus the fuzzy set F is described as follows: F = {µ (u1)/u1, µ (u2)/u2, ..., µ (un)/un} When µF (u) is viewed to be a measure of the possibility that a variable, say X, has the value u in this approach, where X takes values in U, a fuzzy value is described by a possibility distribution πX (Prade & Testemale, 1984; Zadeh, 1978). πX = {πX (u1)/u1, πX (u2)/u2, ..., πX (un)/un}

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Imprecise and Uncertain Engineering Information Modeling

195

Here, for each ui ∈ U, πX (ui) denotes the possibility that X takes value ui. Let πX and F be the possibility distribution representation and the fuzzy set representation for a fuzzy value, respectively. Then X and F are usually regarded as the same things, i.e., πX = F.

EXPRESS-G EXPRESS-G is the graphical representation of EXPRESS, which uses graphical symbols to form a diagram (Eastman & Fereshetian, 1994; Shaw, Bloor & Pennington, 1989). Note that it can only represent a subset of the full language of EXPRESS. EXPESS-G provides support for the notions of entity, type, relationship, cardinality, and schema. The functions, procedures, and rules in EXPRESS language are not supported by EXPRESS-G. EXPRESS-G has three basic kinds of symbols for definition, relationship, and composition. Definition and relation symbols are used to define the contents and structure of an information model. Composition symbols enable the diagrams to be spread across many physical pages. Definition symbols. A definition symbol is a rectangle enclosing the name of the thing being defined. The type of the definition is denoted by the style of the box. Symbols are provided for simple type, defined type, entity type, and schema.

•

Simple type symbols. A number of predefined simple types offered by EXPRESS language include Binary, Boolean, Integer, Logical, Number, Real, and String. The symbol for them is a solid rectangle with a double vertical line at its right end. The name of the type is enclosed within the box. The symbols for these simple types are shown in Figure 1.

•

Type symbols. The symbols for the select, enumeration and defined data types are dashed boxes as shown in Figure 2.

• •

Entity symbols. The symbol for an entity is shown in Figure 3. Schema symbols. The symbol for a schema is shown in Figure 4.

Relationship symbols. There are three different styles of lines, namely, a dashed line, a thick solid line, and a thin solid line, which are employed to connect related definition symbols. The symbols for these three lines are shown in Figure 5. A relationship for an optional valued attribute of an entity is displayed as a dashed line, as is a schema-schema reference. A supertype-subtype relationship is displayed as a thick solid line. All other relationships are displayed as thin lines.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

196 Ma

Figure 1. Simple type symbols Binary

Boolean

Integer

Logical

Number

Real

String

Figure 2. Type definition symbols Select

Enumeration

DefType

Figure 3. Entity definition symbol Entity name

Figure 4. Schema definition symbol Schema name

Figure 5. Relationship line styles Dashed line

Thick line

Thin line

Note that the lines with open circles denote relationship directions in EXPRESSG. Composition symbols. Graphical representation of models often spans many pages. Each page in a model must be numbered so that we can keep track of where we are in the model and enable inter-page referencing. In addition, a schema may utilize definitions from another schema. Therefore, there are two kinds of composition symbols for page references and inter-schema references, which are shown in Figure 6 and Figure 7, respectively. EXPRESS-G provides two levels of modeling, namely, schema level model and entity level model. Therefore, we discuss the fuzziness in the entity level model and in the schema level model in the following, respectively.

Nested Relational Databases There is the restriction of first normal form (1NF) in traditional relational databases, which assumes that each attribute value in a relational instance must

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Imprecise and Uncertain Engineering Information Modeling

197

Figure 6. Position symbols for page references Page #, Ref # (#, #, ...

Page #, Ref # Name

Reference onto this page

Figure 7. Position symbols for inter-schema references

Schema.def

Schema.def

alias

alias

Definition referenced from another schema

Definition used from another schema

be atomic. It is clear that this assumption limits the expressive power of the traditional relational database model in modeling complex objects with rich data types and semantic relationships in the real applications. The first attempt to relax this limitation was made by Makinouchi (1977). In this initial work, attribute values in the relational instance may be atomic or setvalued. We call such relational databases non-first normal form (NF2) relational databases. After Makinouchi’s proposal, NF2 database model is further extended (Motor & Smets, 1997; Schek & Scholl, 1986; Yazici et al., 1999). The NF2 database model in common sense now means that attribute values in the relational instances are either atomic or set-valued and even relations themselves. So NF2 databases are called nested relational databases also. In this chapter, we do not differentiate between these two notions. It can be seen that the NF2 database model is a generalized relational data model, but it can model complex objects and relationships. A formal definition of NF2 database model (Yazici et al., 1999) is given as follows. Definition. Let a relation r have schema R = (A1, A2, ..., An) and let Dl, D2, ..., Dn be corresponding domains from which values for attributes (A1, A2, ..., An) are selected. Attribute Aj is a higher order attribute if its schema appears on the left-hand side of a rule; otherwise, it is simple. Definition. An NF2 schema may contain any combination of simple or higher order attributes on the right-hand side of the rules. An instance of relation r,

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

198 Ma

Figure 8. Hierarchy structure of car product Car

Car_Id

Interior

Interior_Id

Dashboard

Chassis

Seat

Gearbox

Engine_Id

Engine

Size

Turbo

denoted by r, consists of attributes (A1, A2, ..., An). A tuple of an NF2 relation is an element in r and denoted as consisting of n components. Each component aj (1 ≤ j ≤ n) may be an atomic or null value or another tuple. If Aj is a higher order attribute, then the value aj need not be a single value, but an element of the subset of the Cartesian product of associated domains D jl, Dj2, ..., Djm. Let us look at the hierarchy structure of car products shown in Figure 8 (Erens, Mckay & Bloor, 1994; Li, Zhang & Tso, 2000; Zhang & Li, 1999). The Car structure can be defined as a nested data model utilizing the following forms: Car = (Car _Id, Interior, Chassis, Gearbox, Engine) Interior = (Interior _Id, Dashboard, Seat) Engine = (Engine_Id, Size, Turbo) A NF 2 relational instance for the Car product data model is illustrated in Table 1. Based on NF2 database model, the ordinary relational algebra operations have been extended. In addition, two new restructuring operators, called the Nest and

Table 1. Nested relation for car product Interior Car_Id

Interior_Id

Dashboard

VA11 VA12

IN01 IN02

SE89 SE35

Gear-

Engine

Seat

Chassis

box

Engine_Id

Size

Turbo

DB24 DB35

CH01 CH02

GE5 GE5

E18 E20

1.8 2.0

No Yes

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Imprecise and Uncertain Engineering Information Modeling

199

Unnest (Ozsoyoglu et al., 1987; Roth, Korth & Batory, 1987; as well as Pack and Unpack in Ozsoyoglu et al., 1987), have been introduced. The Nest operator can gain the nested relation including complex-valued attributes. The Unnest operator is used to flatten the nested relation. That is, it takes a relation nested on a set of attributes and desegregates it, creating a “flatter” structure. The formal definitions and the properties of these two operators as well as the ordinary relational algebra for the NF2 data model can be found in Colby (1990) and Venkatramen and Sen (1993). The NF2 data model is useful in an engineering data modeling due to its capacity of modeling complex objects with hierarchical structure, which are very common in engineering areas. Let’s look at the Instance-As-Type (IAT) problem proposed in Erens, Mckay and Bloor (1994) and Li, Zhang and Tso (2000). The IAT means that an object appears as a type in one information base, but also as an instance in another information base at the same time. An IAT phenomenon, for example, occurs in the interior of car in the above example. The IAT problems can result in the more difficulty and cost in the maintenance of information consistency. So we must resolve them in product data modeling, or update anomaly occur. It is shown in Table 2.1 that the NF2 data model can avoid the IAT problems naturally.

Extended EXPRESS-G Since entity and schema are keys of the EXPRESS-G information model, in this section we extend EXPRESS-G for fuzzy information modeling in entity level and schema level models, respectively. In an entity level model, we mainly investigate the fuzziness in data types, attributes, entities, and relationships. In a schema level, we mainly investigate the fuzziness between schemas. The corresponding notations are hereby introduced.

Fuzziness in Entity Level Model An entity level model is an EXPRESS-G model that represents the definitions and relationships that comprise a single schema. So the components of such a model consist of type, entity, relationship symbols, role, and cardinality information.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

200 Ma

Figure 9. Simple range-valued type symbols Binary

Boolean

Integer

Logical

Number

Real

String

Number

Real

String

Figure 10. Simple fuzzy type symbols Binary

Boolean

Integer

Logical

Imprecise and Uncertain Role Names As we know, entities consist of attributes and an attribute corresponds to a domain of values with a certain type. In EXPRESS-G, the attributes of an entity are role named. The text string representing the role name is placed on the relationship line connecting an entity symbol to its attribute representation. For an aggregation attribute, the role name on the relationship line is followed by the first letter of the aggregation, such as LIST [m: n], ARRAY [m: n], SET [m: n], or BAG [m: n]. It should be noted, however, that the attribute values of an entity may be imprecise or fuzzy. An entity Engine, for example, has an attribute Size, which data type is Real and values may be either fuzzy or imprecise in the preliminary phase of its design. Such role names that may take imprecise and fuzzy values are different than ones that only take crisp values. In the EXPRESS, it is allowed that attributes take null values by using the keyword “optional” in explicit attribute definition. In order to represent range values and fuzzy values in EXPRESS-G, we give the symbols for range-valued and fuzzy attributes that take the values of simple data types as shown in Figure 9 and Figure 10, respectively.

Imprecise and Uncertain Type Modeling Fuzziness can also be found in type modeling. First let’s have a look at enumeration type. As we know, an enumeration type is an ordered list of values represented by name, where the list has a perfect boundary. A value named either belongs to the enumeration type or does not belong to the enumeration type. It is possible, however, that a value belongs to the enumeration type with degree, namely, the value is fuzzy. For example, an enumeration type is HairType = ENUMERATION OF (Black, Red, Brown, Golden, Grey) and the hair type of a person is red and brown.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Imprecise and Uncertain Engineering Information Modeling

Figure 11.

201

Ranged-valued type definition symbols DefType

Select

Enumeration

Figure 12. Fuzzy type definition symbols DefType

Select

Enumeration

A defined data type is created based on the underlying type. The defined data type generally has the same domain of values as the underlying type unless a constraint is put on it. The underlying type can be simple type, collection type, enumeration type, select type, and named type. We have shown that a value of simple type, collect type, or enumeration type may be fuzzy or imprecise. The imprecision and fuzziness for the values of select type or entity type are shown in the following. Thus, the value of a defined data type may be imprecise or fuzzy. A select type defines a named collection of other types called a select list. A value of a select type is a value of one of the types specified in the select list where each item is an entity type or a defined type. The imprecision or fuzziness of a value of select type comes from the imprecision or fuzziness of its component type, fuzzy or imprecise entity type and defined type. The symbols for modeling imprecise and fuzzy type are shown in Figure 11 and Figure 12, respectively.

Fuzzy Entity Modeling Fuzzy entity modeling can be classified into two levels. The first level is the fuzziness in the entity sets, namely, an entity has a degree of membership in the

Figure 13. Entity with fuzzy instances E

Figure 14. Entity with membership degree µ (E)/E

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

202 Ma

model. For example, an entity Engine may be fuzzy in the product data model. The second level is related to the fuzzy occurrences of entities. For an entity Research Student, for example, it is not uncertain if John is a Ph.D. student. Such an entity is represented using the definition symbol in Figure 13. For the first level fuzziness, memberships can be placed inside the solid rectangle as well as the name of the entities. Let E be an entity and µ (E) be its grade of membership in the model, then “ µ (E)/E” is enclosed in the solid rectangle. If µ (E) = 1.0, “1.0/E” is usually denoted “E” simply. The graphical representation of such entity is shown in Figure 14. In a classical situation, if there exist two entities E1 and E2 such that for any entity instance e, e ∈ E2 implies e ∈ E1, then E2 is called a subtype of E1, and E1 is called a supertype of E2. As mentioned above, an instance of entity, say e, may be fuzzy for an entity, say E. Therefore, there exists a fuzzy supertype/subtype in EXPRESS. Let E and S be two fuzzy entities with membership functions µE and µS, respectively. Then S is a fuzzy subtype of E and E is a fuzzy supertype of S if and only if the following is true: (∀e) (e ∈ U ∧ µS (e) ≤ µE (e)) Considering a fuzzy supertype E and its fuzzy subtypes S1, S2, …, Sn with membership functions µE, µS1, µS2, ..., and µSn, respectively, the following relationship is true: (∀e) (∀S) (e ∈ U ∧ S ∈ {S1, S2, …, Sn} ∧ µS (e) ≤ µE (e)) For the fuzzy subtype with multiple fuzzy supertypes, let E be a fuzzy subtype and S1, S2, …, Sn be its fuzzy supertypes, which membership functions are respectively µE, µS1, µS2, ..., and µSn. (∀e) (∀S) (e ∈ E ∧ S ∈ {S1, S2, …, Sn} ∧ µE (e) > 0 ∧ µS (e) ≥ µE (e)) The fuzzy supertype/subtype in fuzzy EXPRESS-G can be represented with a fuzzy relationship.

Fuzzy Relationship Modeling As mentioned above, there are dashed lines, thick solid lines, and thin solid lines in EXPRESS-G. Dashed lines and thin solid lines connecting attributes represent

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Imprecise and Uncertain Engineering Information Modeling

203

Figure 15. Fuzzy relationship line styles µ (A)/A

µ (A)/A

Figure 16. Fuzzy schema definition symbol Schema name

that attributes must belong to the corresponding entity. In fact, it is also possible that we do not know if an attribute belongs to the corresponding entity for sure. At this point, such an attribute should be associated with a membership degree to indicate a fuzzy attribute-entity relationship. We can place membership degrees upon dashed lines and thin solid lines. In addition, we use thick dashed solid lines to represent the fuzzy supertype/subtype above. The symbols for these three lines are shown in Figure 15, where A and µ denote the name of an attribute and its membership, respectively.

Fuzziness in Schema Level Model A schema level model is one that displays the schemas, and the relationships between these schemas. Since fuzziness can occur in entities, the relationships among these entities may hereby be fuzzy. Following two kinds of schema relationships Use and Reference in EXPRESS-G, fuzzy Use and Reference relationships in fuzzy EXPRESS-G are denoted by normal width relation lines and dashed relation lines with membership degrees as shown in Figure 16. Through the discussion above, three levels of fuzziness can be found in fuzzy EXPRESS-G, namely, the fuzziness at the level of attribute value (the third level), the fuzziness at the level of instance/entity (the second level), and the fuzziness at the level of entity and attribute (the first level). The fuzziness at the third level means that attributes take fuzzy values. The second level of fuzziness means that each instance of an entity belongs to the entity with a membership degree. The first level of fuzziness means that attributes comprise an entity with membership degrees or entities comprise a schema with membership degrees.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

204 Ma

Real Length

Material

Type

Real Volume

Real

Number Capacity

0.5/Thickness

0.7/Water tank

Light oil tank

Real String

Radius Tank_Id

Tank

Over height

Real

Air tank

Body_Id

String

Figure 17. Fuzzy EXPRESS-G data model

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Imprecise and Uncertain Engineering Information Modeling

205

An Example Illustration In Figure 17, we give a simple fuzzy EXPRESS-G data model utilizing some notations introduced above. Entity Tank is a supertype, which has three subtypes, namely, Air tank, Water tank, and Light oil tank. Among these three subtypes, it is known for certain that entities Light oil tank and Air tank are the subtypes of entity Tank. In other words, the membership degrees that Light oil tank and Air tank are the subtypes of Tank are 1.0, respectively. However, it is not known for certain if entity Water tank must be the subtype of entity Tank. It is only known that the membership degree that Water tank is the subtypes of Tank is 0.7. In addition, entity Light oil tank is a fuzzy entity with fuzzy instances. The entity Air tank has eight attributes. The attribute Body_Id is a perfect one with string type. Attribute Thickness is an attribute associated with a membership degree 0.5, which means the possibility that entity Air tank has attribute Thickness is not certainly 1.0 but is only 0.5. The attributes Volume, Capacity, and Over height are ones that can take fuzzy real values. The attributes Length and Radius are imprecise ones that can take range values. It should be noted that attribute Material is one of enumeration type.

Fuzzy Nested Relational Databases A fuzzy NF2 database schema is a set of attributes (A1, A2, ..., An, pM), where pM is a special attribute and used to represent the membership degrees of the tuples, and their domains are Dl, D2, ..., Dn, D0, respectively, where Di (1 ≤ i ≤ n) might be one of the following: 1.

The set of atomic values. For any an element ai ∈ Di, it is a typical simple crisp value without imperfectness.

2.

The set of atomic values as well as null values, where null values may be unk, inap, nin, and onul.

3.

The power set of the set in (1). The corresponding attribute values are multivalued ones with the form of {ail, ai2, ..., aim}.

4.

The power set of the set in (1). The corresponding attribute values are imprecise range-valued ones of the form [a il, ai2, ..., aim] and [ail-ai2] for the discrete and continuous universe of discourse, respectively.

5.

The set of fuzzy subset. The corresponding attribute value, say ai, is a fuzzy value represented by possibility distribution.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

206 Ma

6.

The set of relation values. The corresponding attribute value, say ai, is a tuple of the form which is an element of Dil × D i2 × ... × Dim (m > 1 and 1 ≤ i ≤ n), where each Dij (1 ≤ j ≤ m) may be a domain in (1), (2), (3), (4), or (5) and even the set of relation values.

Domain D0 of attribute pM is a set of atomic values and each value is a crisp one from the range [0, 1]. Let t be a tuple in a given NF2 relation. Then the attribute value of t on pM, denoted t [pM], represents the possibility that t belongs to the NF2 relation. Let us look at the preliminary design of a pressured air tank presented by Otto and Antonsson (1994). In this design, there are two design parameter values that need to be determined: length (l) and radius (r). In addition, there are four performance parameters in the design:

•

Metal volume m: the preference ranks of m are set because it is proportional to the cost;

•

Tank capacity v: the desired level of this performance ranks the preference;

• •

Overall height restriction Lo: it is fuzzy; and Overall radius restriction Ro: it is also fuzzy.

Here, we focus on the modeling of these parameter values as well as its structure information in product data model using fuzzy NF2 database model. A possible database schema and an instance are partially represented in Table 2 (for space limitation). Note that the attribute pM can be omitted from the fuzzy NF 2 relation when all tuples have value 1.0 on pM. Here, “[25, 600],” “[10, 100],” “[30,650],” and “[15,110]” are imprecise value intervals, and “about 2.5e+03,” “about 1.0e+06,” “about 2.5e+04,” “about 1.0e+07,” “less than 627.50,” “less than 106.75,” “less than 630.00,” and “less than 112.50” are all fuzzy values. Assume that these fuzzy values are represented by the possibility distributions as follows: “about 2.5e+03”: {1.0/2.5e+03, 0.96/5.0e+03, 0.88/7.5e+03, 0.75/1.0e+04, 0.57/1.25e+04, 0.32/ 1.5e+04, 0.08/1.75e+04}; “about 1.0e+06”: {0.05/1.0e+05, 0.18/2.0e+05, 0.37/3.0e+05, 0.55/4.0e+05, 0.69/5.0e+05, 0.78/ 6.0e+05, 0.87/7.0e+05, 0.93/8.0e+05, 0.96/9.0e+05, 0.97/1.0e+06};

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Imprecise and Uncertain Engineering Information Modeling

207

“less than 627.50”: {1.0/600.0, 1.0/605.0, 1.0/610.0, 1.0/615.0, 1.0/620.0, 1.0/622.5, 1.0/623.0, 0.9/ 623.5, 0.7624.0/,0.5/625.0,0.3/625.75, 0.1/626.5} “less than 106.75”: {1.0/100.0, 1.0/101.25, 1.0/102.5, 1.0/103.0, 0.9/103.25, 0.81/103.75, 0.5/105.0, 0.2/106.25, 0.1/106.5}

Mapping Fuzzy EXPRESS-G to Fuzzy Nested Relational Databases In this section, we define the formal approaches to mapping EXPRESS-G data model with imprecise and uncertain information to fuzzy NF2 database schema. The issue is essentially concerned with the design of logical databases for a product data model from a conceptual data model for a product data model.

Nested Relational Databases Support to EXPRESS Model The entity instances are identified by their unique identifiers in the EXPRESS information model. The entity identifiers are just like the keys in (nested) relational databases, but they are different. The keys are the component parts of information content whereas the entity identifiers are not. We can view an entity as a database relation and view the instances of the entity as the tuples of the database relation. When we would like to represent entity instances in relational databases, we have to solve the problem of how to identify entity instances in relational databases. In other words, we must indicate keys of the tuples originated from entity instances in the relational databases. As we know, in EXPRESS information, there are attributes with UNIQUE constraints. When an entity is mapped into a relation and each entity instance is mapped into a tuple, it is clear that such attributes can be viewed as the key of the tuples to identify instances. Therefore, an EXPRESS information model must at least contain such an attribute with UNIQUE constraint when relational databases are used to model an EXPRESS information model. In EXPRESS, there is the entity for which attributes are other entities, called complex entities. Complex entities and subtype/supertype entities in an EX-

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

208 Ma

PRESS information model can be implemented in relational databases via the reference relationships between relations. It is clear that, based on such organization principles, the objects related in structural relationships with one another are represented in separate relational databases. In order to obtain some information, one may have to query multiple relational databases by using join operations. Besides, it is very hard to have a complete and clear picture about an information model from the query answers. Relational databases obviously are not suitable to support the EXPRESS information model because of the restriction of first normal form (1NF) in traditional relational databases. Nested relational databases can solve the problems above very well.

Formal Mapping Generally speaking, an entity in EXPRESS-G can be mapped into a relational schema and the role names in the entity mapped into the attributes of the relational schema. But the following problems must be solved when mapping: (a) How to represent the subtype/supertype relationship in the fuzzy NF2 databases. (b) How to model the fuzziness of the EXPRESS-G data model in the fuzzy NF2 databases. (c) Data type transformation. It has been claimed above that the fuzziness in EXPRESS-G can be classified into three levels. The second level and the third level of fuzziness, namely, the fuzziness at the level of instance/entity and the fuzziness at the level of attribute value, can be represented in fuzzy NF2 databases. Relational database models and nested relational database models only focus on instance modeling and their meta-structures are implicitly represented in the schemas. The fuzziness at the level of entity and attribute cannot be modeled in fuzzy NF2 databases due to the limitation of NF 2 databases in meta-data modeling. The following three kinds of entities can be identified in an EXPRESS-G model: 1.

Member entities. A member entity is the entity that comprises other entities as a component part or that is the underlying types of enumeration and select types.

2.

Subtype entities. A subtype entity is the entity that is in supertype/subtype relationships and is the subtype entity of the supertype entity/entities.

3.

Root entities. A root entity is neither a subtype entity nor a member entity.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Imprecise and Uncertain Engineering Information Modeling

209

For the entities mentioned above, we have different transformation strategies. First, let us look at case (3): root entities. Each root entity is mapped into a fuzzy NF2 relational schema and its role names become the simple attributes or structured attributes of the relation, depending on their data types.

• • •

A role name with simple data type is mapped to a simple attribute.

•

A role name with enumeration type is mapped to a simple attribute. Here, assume that NF2 databases support such attribute domains that users define themselves.

•

A role name with select type is mapped to a structured attribute.

A role name with entity type is mapped to a structured attribute. A role name with defined type can be mapped to a simple attribute or a structured attribute, depending on the structure of the defined type.

It is generally true that the transformations above should be processed recursively because an entity type, a defined type, or a select type may contain some of the above components by itself. It should be noted that the fuzziness of values of simple attributes in EXPRESSG information models is represented in the attribute values of NF2 databases. For fuzziness of entity instances, the special attribute “pM” must be included in the structured attributes or the relational schema to indicate the membership degree of entity instances. Second, let us look at case (2): subtype entities. Following similar methods as described above, a subtype entity can be mapped into a fuzzy NF2 schema. It should be noted that, however, the key attributes of all the supertypes must be replicated in the subtype.

Table 2. Air tank relation Tank

Tank_body

_Id

Body_Id

Material

Length

Radius

TA1

BO01

Alloy

[25, 600]

[10, 100]

TA2

BO02

Steel

[30, 650]

[15, 110]

Overall Capacity

height

about

about

less than

2.5e+03

1.0e+06

627.50

about

about

less than

2.5e+04

1.0e+07

630.00

Volume

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

210 Ma

A member entity in case (1) is not mapped to a class, but to a complex attribute of another class that is composed of the member entities. But, the fuzziness in the member entity can be handled according to the same principles as the common entity. Following the formal rules given above, we map the fuzzy EXPRESS-G model in Figure 17 into the fuzzy nested relational database in Table 2. Note that attribute “0.5/Thickness” in Figure 17 cannot be mapped into the fuzzy nested relational database due to its first level of fuzziness. It should be noted that we do not discuss the mapping of data types in fuzzy EXPRESS-G models. We assume that fuzzy nested relational databases support the data types in fuzzy EXPRESS-G models. In fact, the data types supported by different database products vary. More and more data types are supported by some latest release of database management systems. Our focus here is on mapping the entities and the attributes associated with entities in fuzzy EXPRESS-G models. We have identified all three kinds of entities in fuzzy EXPRESS-G models and given the mapping methods that map fuzzy entities and attributes into fuzzy nested relational databases. So the mapping methods given in the chapter can be used to solve the problem of fuzzy engineering data model transformations.

Conclusions In this chapter, we have proposed a fuzzy extension to EXPRESS-G that can capture the imprecise and uncertain engineering information. In addition, fuzzy nested relational databases have been introduced. The formal approaches to mapping a fuzzy EXPRESS-G schema to a fuzzy nested relational database schema have been developed in this chapter. It should be noted that EXPRESS-G is only a subset of the full language of EXPRESS. Clearly, it is necessary to extend EXPRESS for imprecise and uncertain engineering information modeling and then map fuzzy EXPRESS models into databases. In addition, it is also very interesting to formally compare transformation modeling among EXPRESS-G and other conceptual data models, such as ER/EER, UML, and IDEF1X. We will investigate these issues in our future work.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Imprecise and Uncertain Engineering Information Modeling

211

References Antonsoon, E. K. & Otto, K. N. (1995). Imprecision in Engineering Design. ASME Journal of Mechanical Design, 117(B), 25-32. Bordogna, G., Pasi, G. & Lucarella, D. (1999). A Fuzzy Object-Oriented Data Model for Managing Vague and Uncertain Information. International Journal of Intelligent Systems, 14, 623-651. Bosc, P. & Prade, H. (1993). An Introduction to Fuzzy Set and Possibility Theory Based Approaches to the Treatment of Uncertainty and Imprecision in Database Management systems. Proceedings of the Second Workshop on Uncertainty Management in Information Systems: From Needs to Solutions. Buckles, B. P. & Petry, F. E. (1982). A Fuzzy Representation of Data for Relational Database. Fuzzy Sets and Systems, 7(3), 213-226. Caputo, M. (1996). Uncertainty, Flexibility and Buffers in the Management of the Firm Operating System. Production Planning & Control, 7(5), 518528. Chaudhry, N., Moyne, J. & Rundensteiner, E. A. (1999). An Extended Database Design Methodology for Uncertain Data Management. Information Sciences, 121(1-2), 83-112. Chen, G. Q. & Kerre, E. E. (1998). Extending ER/EER Concepts towards Fuzzy Conceptual Data Modeling. Proceedings of the 1998 IEEE International Conference on Fuzzy Systems, 2, 1320-1325. Chen, P. P. (1976). The Entity-Relationship Model: Toward a Unified View of Data. ACM Transactions on Database Systems, 1(1), 9-36. Codd, E. F. (1987). More Commentary on Missing Information in Relational Databases (Applicable and Inapplicable Information). ACM SIGMOD Record, 16(1), 42-50. Colby, L. S. (1990). A Recursive Algebra for Nested Relations. Information Systems, 15(5), 567-662. Dubois, D. & Prade, H. (1986). Possibility Theory: An Approach to Computerized Processing. New York: Plenum Press. Eastman, C. M. & Fereshetian, N. (1994). Information Models for Use in Product Design: A Comparison. Computer-Aide Design, 26(7), 551-572. Erens, F., Mckay, A. & Bloor, S. (1994). Product Modeling Using Multiple Levels of Abstraction Instance as Types. Computers in Industry, 24, 17-28.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

212 Ma

Francois, F. & Bigeon, J. (1995). Integration of Fuzzy Techniques in a CAD/ CAN System. IEEE Transactions on Magnetics, 31(3), 1996-1999. George, R., Srikanth, R., Petry, F. E. & Buckles, B. P. (1996). Uncertainty Management Issues in the Object-Oriented Data Model. IEEE Transactions on Fuzzy Systems, 4(2), 179-192. Gottlob, G. & Zicari, R. (1988). Closed World Databases Opened through Null Values. Proceedings of the 1988 International Conference on Very Large Data Bases, 50-61. Grabot, B. & Geneste, L. (1998). Management of Imprecision and Uncertainty for Production Activity Control. Journal of Intelligent Manufacturing, 9, 431-446. Guiffrida, A. & Nagi, R. (1998). Fuzzy Set Theory Applications in Production Management Research: A Literature Survey. Journal of Intelligent Manufacturing, 9, 39-56. Gyseghem, N. V. & Caluwe, R. D. (1998). Imprecision and Uncertainty in UFO Database Model. Journal of the American Society for Information Science, 49(3), 236-252. ISO IS 10303-1 TCI84/SC4. (1994a). Product Data Representation and Exchange-Part 1: Overview and Fundamental Principles. International Standard. ISO IS 10303-1 TCI84/SC4. (1994b). Product Data Representation and Exchange-Part 11: The EXPRESS Language Reference Manual. International Standard. Karwowski, W. & Evans, G. W. (1986). Fuzzy Concepts in Production Management Research: A Review. International Journal of Production Research, 24(1), 129-147. Kim, K., Cormier, D., Young, R. E. & O’Grady, P. (1995). A System for Design and Concurrent Engineering under Imprecision. Journal of Intelligent Manufacturing, 6(1), 11-27. Klir, G. J. & Folger, T. A. (1988). Fuzzy Sets, Uncertainty, and Information. Englewood Cliffs, NJ: Prentice Hall. Li, Q., Ma, Z. M. & Zhang, W. J. (1998). Modeling of Incomplete and Uncertain Information in Relational Database for Engineering Design and Production Management: Review and Future Work. Proceedings of the 1998 ASME Design Engineering Technical Conference. Li, Q., Zhang, W. J. & Tso, S. K. (2000). Generalization of Strategies for Product Data Modeling with Special Reference to Instance-As-Type Problem. Computers in Industry, 41, 25-34.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Imprecise and Uncertain Engineering Information Modeling

213

Ma, Z. M., Zhang, W. J. & Li, Q. (1999). Extending Relational Data Model to Resolve the Conflicts in Schema Integration of Multiple Databases in Virtual Enterprise. Proceedings of the 1999 ASME Design Engineering Technical Conference. Ma, Z. M., Zhang, W. J. & Ma, W. Y. (1999). Incomplete Information in Product Concept Design and Its Modeling in Relational Databases. Proceedings of the 1999 Lancaster International Workshop on Engineering Design, 99-114. Ma, Z. M., Zhang, W. J., Ma, W. Y. & Chen, G. Q. (2000). Extending EXPRESS-G to Model Fuzzy Information in Product Data Model. The 2000 ASME Design Engineering Technical Conference. Makinouchi, A. (1977). A Consideration on Normal Form of Not-Necessarily Normalized Relations in the Relational Data Model. Proceedings of Third International Conference on Very Large Databases, 447-453. McKay, A. (1988). The Structure Editor Approach to Product Description. Technical Report, University of Leeds, ISS-PDS-Report-4. Motor, A. (1990). Accommodation Imprecision in Database Systems: Issues and Solutions. ACM SIGMOD Record, 19(4), 69-74. Motor, A. & Smets, P. (1997). Uncertainty Management in Information Systems: From Needs to Solutions. Kluwer Academic Publishers. Otto, K. N. & Antonsoon, E. K. (1994). Modeling Imprecision in Product Design. Proceedings of Fuzzy-IEEE 1994, 346-351. Otto, K. N. & Antonsoon, E. K. (1994). Design Parameter Selection in the Presence of Noise. Research in Engineering Design, 6(4), 234-246. Ozsoyoglu, G., Ozsoyoglu, Z. M. & Matos, V. (1987). Extending Relational Algebra and Relational Calculus with Set-Valued Attributes and Aggregate Functions. ACM Transactions on Database Systems, 12(4), 566-592. Parsons, S. (1996). Current Approaches to Handling Imperfect Information in Data and Knowledge Bases. IEEE Transactions on Knowledge Data Engineering, 8, 353-372. Petrovic, D., Roy, R. & Petrovic, R. (1998). Modeling and Simulation of a Supply Chain in an Uncertain Environment. European Journal of Operational Research, 109, 299-309. Petrovic, D., Roy, R. & Petrovic, R. (1999). Supply Chain Modeling Using Fuzzy Sets. International Journal of Production Economics, 59, 443-453.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

214 Ma

Prade, H. & Testemale, C. (1984). Generalizing Database Relational Algebra for the Treatment of Incomplete or Uncertain Information. Information Sciences, 34, 115-143. Roth, M. A., Korth, H. F. & Batory, D. S. (1987). SQL/NF: A Query Language for Non-1NF Relational Databases. Information Systems, 12, 99-114. Schek, H. J. & Scholl, M. H. (1986). The Relational Model with RelationalValued Attributes. Information Systems, 11(2), 137-147. Schenck, D. A. & Wilson, P. R. (1994). Information Modeling: the EXPRESS Way. Oxford University Press. Shaw, N. K., Bloor, M. S. & Pennington A. de, J. (1989). Product Data Models. Research in Engineering Design, 1, 43-50. Teorey, T. J., Yang, D. Q. & Fry, J. P. (1986). A Logical Design Methodology for Relational Databases Using the Extended Entity-Relationship Model. ACM Computing Surveys, 18(2), 197-222. Tsourveloudis, N. G. & Phillis, Y. A. (1998). Manufacturing Flexibility Measurement: A Fuzzy Logic Framework. IEEE Transactions on Robotics and Automation, 14(4), 513-524. Venkatramen, S. S. & Sen, A. (1993). Formalization of An IS-A Based Extended Nested Relation Data Model. Information Systems, 20(1), 5357. Vila, M. A., Cubero, J. C., Medina, J. M. & Pons, O. (1996). A Conceptual Approach for Deal with Imprecision and Uncertainty in Object-Based Data Models. International Journal of Intelligent Systems, 11, 791-806. Wood, K. L. & Antonsoon, E. K. (1992). Modeling Imprecision and Uncertainty in Preliminary Engineering Design. Mechanism and Machine Theory, 25(3), 305-324. Yager, R. R. (2000). Targeted e-Commerce Marketing Using Fuzzy Intelligent Agents. IEEE Intelligent Systems, 15(6), 42-45. Yager, R. R. & Pasi, G. (2001). Product Category Description for WebShopping in E-Commerce. International Journal of Intelligent Systems, 16, 1009–1021. Yazici, A., Soysa1, A., Buckles, B. P. & Petry, F. E. (1999). Uncertainty in a Nested Relational Database Model. Data & Knowledge Engineering, 30, 275-301. Zadeh, L. A. (1965). Fuzzy Sets. Information and Control, 8(3), 338-353. Zadeh, L. A. (1978). Fuzzy Sets as a Basis for a Theory of Possibility. Fuzzy Sets and Systems, 1(1), 3-28.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Imprecise and Uncertain Engineering Information Modeling

215

Zaniola, C. (1986). Database Relations with Null Values. Journal of Computer System Science, 28(2), 142-166. Zhang, W. J. & Li, Q. (1999). Information Modeling for Made-to-order Virtual Enterprise Manufacturing Systems. Computer-Aided Design, 31, 611619. Zimmermann, H. J. (1999). Practical Applications of Fuzzy Technologies, Kluwer Academic Publishers. Zvieli, A. & Chen, P. P. (1986). Entity-Relationship Modeling and Fuzzy Databases. Proceedings of the 1986 IEEE International Conference on Data Engineering, 320-327.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

216 Ma

Section III Additional Topics

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Analysing Transformations in Performance Management

217

Chapter X

Analysing Transformations in Performance Management Bernd Wondergem, LogicaCMG Consulting, The Netherlands Norbert Vincent, LogicaCMG Consulting, The Netherlands

Abstract Performance Management (PM) is a way of managing in which the organisation’s goals and organisation model are made explicit. In this chapter, several forms of PM are investigated to meet these observations. The goals and organisation model together form the management model of the organisation. Steering the organisation towards its strategic goals then comes down to repeatedly transforming this management model. In this chapter, these transformations are systematically analysed. In order to do this, we first provide a framework in which several types of transformation can be identified. Second, properties of the transformations are stated and related to different styles of management. In addition, we sketch future trends in a resource-based view on performance management.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

218 Wondergem and Vincent

Introduction Performance management (PM) is a way of managing in which the organisation’s strategic goals and its organisation model are made explicit. In addition, the connection between those two issues is made by stating how the organisation model produces the organisations output. The explicitly stated goals (the what) and organisation model (the how) form the core of the management model for steering the organisation. In PM, running a business is all about transformations. First, of course, a general notion of transformation applies: the organisation transforms some form of input to some form of output. More specific to PM, steering the organisation towards its strategic goals is done by repeatedly transforming the organisation and the management model. In this chapter, we focus on these latter forms of transformation which describe the essence of performance management. This chapter sets out to do two things. First, we describe a framework for analysing transformations in performance management. Second, we use the framework to identify several types of transformations and describe which properties apply to them. The results of this chapter may enhance the understanding of performance management and thus lead to more effective management. This chapter has the following structure: it provides different views of, and approaches to, PM and presents our vision on the subject. Next, the framework for analysing transformations is presented: the performance management model. In the following section, we use this model for describing several types of transformations. The chapter then deals with future trends. Finally, we provide concluding remarks and an outlook on further research.

Background In general, organisations try to find, reach and sustain a strategic position in their environment. Mintzberg (1991) has classified the ways to do this into two categories: emergent strategies and planned strategies. “Organisations develop plans for the future and they evolve patterns out of their past” (Mintzberg, 1994). Performance management falls into the category of planned strategies. Performance Management has a typical set-up. First, the organisation formulates a strategy. Formulating a mission, creating a vision and formulating goals are often seen as preceding steps in strategy-formulation. However, these steps are not always explicitly taken or repeated in formulating or revising the strategy.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Analysing Transformations in Performance Management

219

Figure 1. General set-up of performance management

Second, the strategy and the corresponding goals are translated into performance indicators (PI). PI’s form measurable indicators which give a quantitative view on the organisation’s performance. The PI’s are often put on a scorecard, an instrument used for communicating and analysing the performance. The scorecard is used for steering towards the strategic goals. Therefore, it is used in a cycle for continuous improvement. Deming’s cycle (Deming, 1982), consisting of the steps “plan,” “do,” “check” and “act,” probably is the most wellknown variant. In this cycle, strategy formulation forms a part of the “plan” step. In addition, this step concerns setting up the organisation for the production of value. Figure 1 sketches the place of the Deming’s cycle in the general set-up of performance management. In the “do” step, the organisation produces its products or services and measures its performance through PI’s. This provides a fact-based insight into the current performance. The results are evaluated and actions to improve the future performance are defined in the “‘check” step. Finally, the “act” step consists of implementing the actions. After this, the strategy may be revised and a new cycle starts. The information that is explicitly used in the “check” and “act” steps constitutes the so-called performance management model. This is elaborated upon later in this chapter. In this chapter, we will consider three aspects of PM as its essence. We define Performance Management as the management method and instrument that: 1.

Translates the organisation’s strategy in measurable indicators. The “what” of the strategy is thus explicitly translated into quantitative performance indicators.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

220 Wondergem and Vincent

2.

Links the strategic goals of the organisation to the operational organisation. In other words: how the strategy is to be realised is explicitly incorporated in the management model.

3.

Incorporates a mechanism for the continuous improvement of performance. For this aspect, we use Deming’s cycle as a point of reference.

Different approaches to PM can be distinguished. Without trying to offer a complete overview, we provide a description of some approaches below. The Balanced Scorecard (BSC) is both an instrument and management model that aims at providing a balanced view on the organisation’s performance (Kaplan & Norton, 1992, 2000). It was developed as a reaction to primarily financial steering. Nowadays, it is a popular instrument in many businesses. The BSC looks at the business from four perspectives: financial, customer, internal and innovation & learning. In practice, however, this set-up is sometimes altered by changing the number and contents of the perspectives. In that way, the BSC is custom-made for the specific needs of organisations. The BSC is put to work by searching for key success factors (KSF) within the four perspectives. The KSF’s are derived from the organisation’s strategy, thereby making its causal dependencies explicit. For each KSF, measurable indicators are formulated, resulting in a set of key performance indicators (KPI). The Balanced Scorecard, as an instrument, forms the tool for steering the organisation with these KPI’s. Often, the Deming cycle is used for working towards continuous improvements. Many software tools that implement the (Balanced) scorecard explicitly support the Deming cycle, for instance by providing functionality for data-analysis (check) and action management (act). Value-based management (VBM) (an overview of this field is given in Scheipers et al., 2002) is an approach for performance management stemming from the areas of financial management and management accounting. VBM is a management control system that measures, encourages and supports maximizing shareholder value. It is based on the conviction that the interests of all stakeholder groups are best served by putting the shareholder first. The term “value” is more complete than earning-measures only, since it also takes the notion of risk, the impact of inflation and opportunity costs into account. VBM requires complete information to calculate “value” properly. In addition, a longterm strategic point of view is required to describe the expected outcomes. VBM is advocated to be used at the corporate and strategic business-unit level. Exploiting VBM requires substantial training, especially for non-experts in finance. In holistic approaches for VBM, continuous improvement of (financial) performance is mentioned. However, the link with operational processes appears to be only indirect.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Analysing Transformations in Performance Management

221

Six Sigma is an approach that aims at reducing variety in business processes (Harry, 1998). This can be done for processes that produce goods (products) as well as for processes that deliver services. Six Sigma is a form of “management by fact”: it uses statistical information and techniques to measure and analyse business performance. Six Sigma also exploits two variants of cycles for continuous improvements. First, the so-called DMAIC process, for Define, Measure, Analyse, Improve and Control, is an improvement system for existing processes. Second, the DMADV process, having Design and Verify as final steps, is a system for designing new processes. Designing new processes can be done based on Quality Function Deployment (QFD) (Mazur, 1993). QFD can be seen as a method for high-quality design, which starts with solicitation of customer requirements. QFD distinguishes three types of requirements: (1) revealed requirements (what customers say they want), (2) expected requirements (what customers demand as standard quality) and (3) exciting requirements (beyond the customer’s expectations). These requirements are analysed, prioritised and finally translated to design specifications. This results in a set of measurable product specifications. QFD is an explication of the “plan” phase of the Deming-cycle. The remaining three phases of the Deming cycle are not explicitly covered by QFD. QFD thus provides a structured approach for the design of a quality product and service portfolio and the processes that deliver them. In addition to the mentioned approaches, several other methods can be distinguished. A more elaborate overview of approaches to performance management is given in Wondergem (2003). Also, other fields provide approaches to quality and performance management that can be identified. As an example, we mention the approach to software quality, as described in Locuratolo (2002). These approaches to PM all formulate performance indicators for measuring performance and communicating strategic goals. How this is done differs for different approaches. In general, though, all approaches use an intermediate step between the goals (and strategy) on the one hand and the PI’s on the other hand. This step in between is formed by what we call “measured items” (MI). The measured items indicate which strategic subjects are important enough to be measured by PI’s. In the BSC, for instance, the MI’s are formed by the KSF’s. For Six Sigma, the MI’s are the measured characteristics of the processes and products, such as variability and quality. In the next section, a model for PM is described. This model is used in later sections to describe and analyse different transformations in PM.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

222 Wondergem and Vincent

Performance Management Model A Performance Management model (PMM) contains the information that is used for steering transformations in performance management (see also Figure 1). The PMM fits in our notion of performance management by making the goals and the organisation explicit. The cycle for continuous improvement is not incorporated in the model, since it does not constitute information for steering, but the process itself. Therefore, the cycle is described separately as a series of transformations. The PMM is input for these transformations and, with altered form and contents, forms its output as well. PM explicitly models the desired performance in a goal model (GM) and the factors that drive and produce output in an organisation model (OM). The GM and OM are parts of the performance management model (PMM). The PMM is completed by a connection between the GM and OM. This connection indicates how the OM produces or influences the GM. In that sense, the connection forms the handles (H) used to steer performance. A performance management model is thus defined as a tuple (GM, OM, H). Details of GM, OM and H are provided in the remainder of this section.

Goal Model The goal model explicitly states the desired results. It forms a strategic map, consisting of measured items, causal relations between them, and performance indicators. The causal relations between the measured items describe dependencies between the strategic goals and subgoals. The PI’s operationalise the measured items: they make the measurement of the performance considering the measured items explicit. Example: Customer Satisfaction: Contact centres are organisations for which customer satisfaction is of major importance. The management of the fictive unit “The Contact Centre” (TCC) of a company that sells products understands this. Therefore, they explicitly manage their organisation on customer satisfaction. To do this, they constructed the goal model as depicted in Figure 2. The measured item “Customer Satisfaction” is broken down into its constituents. Three other measured items appeared to influence customer satisfaction: satisfaction of the products, the quality of the contact with the customer and the image of the company. The contact center is directly responsible for the quality of contact only. The other two aspects refer to the company as a whole. The quality of

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Analysing Transformations in Performance Management

223

Figure 2. Example goal model for customer satisfaction

contact is influenced by three other measured items: the quality of the processes of customer contact, the satisfaction of the front-office personnel and the availability of the different channels for customer contact. Together, the measured items and causal relations (the arrows in Figure 2) form the strategic map of the company for steering towards customer satisfaction. In Table 1, example performance indicators for some of the MI’s in the goal model are provided. The third column of the table provides example numerical values for the PI’s. The goal model is defined as a tuple GM = (MI, CR, PI, O), where:

• •

MI is a set of measured items, CR is a relation on MI, the causal relations of GM,

Table 1. Example indicators Measured item

Performance indicator

Example

Satisfaction of product

% of customers rating the produt flexibility eight or higer

75%

Number of complaints about the product per month

35.6

Quality of process

Helpfulness of the call center agents as grade

7.4

% of complaining customers that accepts offer

65%

Channel availability

Waiting time in seconds before call is answered

13.1 sec

values

Availability (uptime %) of the voice response system

97%

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

224 Wondergem and Vincent

• •

PI is a set of performance indicators, and O is a relation on MI x PI, the operationalisation of the measured items.

A number of properties of the goal model that are used in PM are described below. The impact value of a causal relation (mi1 à mi2) denotes the part of mi2 that is explained by mi1. The completeness of an MI, say mi1, is the sum of all impact values of the causal relations (mi2 à mi1). The score of a PI expresses the current performance with respect to the PI. The score of a measured item is then given as the average of the scores of all PI’s that operationalise it. In order to compute the average of the PI’s, they first have to be made comparable (Chang & Morgan, 2000).

Organisation Model The organisation model provides the factors that drive and produce the organisation’s output. Numerous ways of modeling organisations exist. For our purposes, it suffices to see the organisation model as a directed graph: OM = (N, L), where N is a set of organisational elements and L a set of links. The links model dependencies on the organisational elements. Example: Contact centre: The management of the contact centre of the previous example has also constructed an organisation model. It has done so based on the COPC model, which is a performance management model for contact centres (COPC, 2002). The organisation model distinguishes goals, enablers and drivers (Figure 4). In short: the goals are produced by the enablers, which, in turn, are directed by the drivers. The goals of the contact centre consist of quality of contact (see the previous example) and costs. The enablers describe how the contact centre is organised: which processes are performed and how are employees and resources used in that? The drivers of the contact centre describe the “way of organising” and contains aspects such as strategy, leadership and planning. For PM, it is important that the OM is complete, i.e., that no relevant organisational elements and links are missed. This would diminish the steering capabilities of the OM. Furthermore, it is important that the OM provides enough detail, enabling focused steering.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Analysing Transformations in Performance Management

225

Figure 3. Example organisation model for contact centers

Handles Handles connect the goals (measured items) to elements from the organisation model. The handles can thus be formally described as a relation on GM.MI x OM.N. However, the handles are often not stated explicitly. Rather, they reside in the heads of managers. In that sense, the handles are subjectively described: each manager may have his own opinion about which factors influence results. This is in line with the “idiosyncratic set of knowledge” managers are stated to form in Van den Bosch and Van Wijk (2001). The scope of the handles is defined as the portion of all connections between organisation model and goal model that are included in the handles. The scope delimits the possible effectiveness of the management model. A large scope may consider many possibilities to influence a certain goal but may also be timeconsuming in decision making. A small scope may quickly lead to a decision about how to steer, but may miss relevant options.

Transformations in Performance Management Performance management can be seen as a cyclic process of transformations. Based on evidence of the current performance, the goal model indicates which (sub)goals need to be improved. The organisation model states how this can be done. Analysing the evidence in the light of the PMM thus leads to a series of actions that can be taken to improve performance. The actions are implemented and the results are measured again. This starts a new cycle in the management process.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

226 Wondergem and Vincent

Figure 4. Transformations in PMM

This cyclic management process can be described as a series of transformations. Each transformation uses evidence of the current performance (E) and the current status of the PMM as input. In Figure 4, this is schematically denoted as a model-theoretic statement. Implementing the actions leads to an altered PMM, denoted by PMM’. Which actions are actually taken, depends on the view management has of the organisation, its environment and their patterns of action-reaction. Together, these issues form the business model. In this sense, the business model (BM) forms the context in which the transformations take place. In Yilmaz and Chatterjee (1999), the BM is called the “theory of knowledge”. The remainder of this section focuses on actions that directly change the PMM: the transformations of the PMM. In general, three basic types of transformation of the PMM can be distinguished: transformation of the goal model, the organisation model and the handles. These transformations are elaborated in the following subsections. After that, the section, Future Trends in Transformations in the Organisation, focuses on future trends for transformations in the organisation itself.

Transformations of Goal Model Transforming the goal model can be identified at two levels. First, measured items and causal relations may be altered. Second, performance indicators and their connection to measured items can be changed. On the level of MI’s and causal relations, the notion of completeness is important. If the GM is not complete enough, MI’s may be added. The precondition of such a transformation can be stated as: the completeness of mi1 is too low, where mi1 is a measured item out of the goal model. The action in the transformation consists of adding a measured item mi2 and the causal relation (mi2 à mi1). The post-condition of the transformation then states that the completeness of mi1 is higher. The proof of this claim hinges on the assumption that the impact value of mi2 on mi1 is non-zero and positive. Example: Adding measured items: Consider the goal model of the first example (Figure 2). Suppose that the MI “Satisfaction of product” is not complete enough. Therefore, this MI is augmented with the following underlying Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Analysing Transformations in Performance Management

227

MI’s: “Satisfaction of product quality,” “Satisfaction of product price” and “Satisfaction of product flexibility.” Changing MI’s and causal relations can also be required by a change of strategy. In contrast to the previous transformation, which is internally oriented through the notion of completeness, a change of strategy is an external factor to the goal model. Since organisations reside in dynamic environments, strategies need to be frequently adjusted and sometimes rigorously changed. As said before, the goal model forms a strategic map. Therefore, the goal model should properly reflect the strategic goals and relations between them. Coherence between strategy and goal model is, however, a subjective issue. This means that transformations that aim at adjusting the goal model to the strategy heavily rely on the business model of the individual manager. Changes in PI’s may stem from several reasons, as is shown in Wondergem and Wulferink (2002). The corresponding transformations of the goal model therefore aim at different improvements. A first reason to alter PI’s is the availability of source data. With these data, the contents of the PI’s are filled. PI’s for which the source data is not sufficiently (easily) available, may be deleted or replaced by more suitable variants. Second, PI’s need to be recognisable, which means that management needs to either be familiar or become familiar with the PI’s. If PI’s are not recognisable, this hinders effective steering. Insufficiently recognisable PI’s are therefore replaced by PI’s that better suit the manager’s experience. As a third reason to change PI’s, we mention dysfunctional behaviour (Birnberg et al., 1983). PI’s measure specific aspects of the organisation’s performance, leaving room for “gaming”: giving insufficient attention to other important issues. These issues can in turn be covered by PI’s as well, resulting in a balanced set of PI’s. In addition, the measurement of PI’s should leave little room for “smoothing” the results. PI’s which are automatically delivered from a data warehouse may serve this purpose, since their computation from source data is strictly specified. Fourth, PI’s need to be consistent in definitions, providing the possibility to compare PI’s. This is for instance required for benchmarking. Finally, PI’s, or rather, the way in which performance is measured, should fit within the culture of the organisation. Measuring performance on individual levels, for instance, requires a culture that supports this. Otherwise, it may be seen as an intrusion on individual rights and as such may harm effective steering.

Transformations of Organisation Model The organisation model reflects the explicitly stated scope managers have for finding actions for improvement. Transformations of the organisation model may Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

228 Wondergem and Vincent

aim at providing a more complete (broad) model or at detailing certain components of the model. If the OM, for instance, appears to be not complete enough, new organisation elements may be added. Adding an organisation element and linking it to the existing OM, provides managers with a broader scope for steering the organisation. In certain quality models, such as the Dutch INK model (INK, 2001), several identified phases describe this growing complexity. Starting, for instance, with isolated actions, a proceeding phase considers series of actions and their interdependencies, i.e., processes. The final phase may describe a holistic view on the organisation and its place within its environment. Next to the completeness of the OM, its level of detail is also of importance. In practice, a first measurement of performance using a rather coarse grained model pinpoints the organisation processes which require improvement. Only these processes are then described in more detail, thus focusing the energy for improvement. Providing more detail for existing organisation elements may be done by decomposing the element. A process, for instance, can be decomposed into several sub-processes. This enlarges the insight in the problem areas and enables more specific decisions on improvement issues. Example: Detailing the organisation model: Consider the example organisation model of The Contact Center as depicted in Figure 3. Suppose that the processes do not deliver the expected quality. In order to be able to select (only) under performing processes, this organisational element is broken down into the following sub-elements: customer contact, instruction, training, supervision, reporting and planning. Furthermore, customer contact processes may be divided into pre-sales, sales, after-sales and service. Both types of transformation of the OM make it more complex. This coincides with the growing professionalism with which performance management is executed: over time, management learns to stepwise increase its focus. Simplifying the OM by deleting elements may also take place. The need for this may, for instance, stem from a simplification in the actual organisation or from a change of focus in management instruments.

Transformations of Handles Handles form the connection between the organisation model and the goal model. As such, they constitute the possibilities in the OM that a manager explicitly considers for reaching a certain goal in the GM. In general, managers

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Analysing Transformations in Performance Management

229

start with a certain view on the handles. By experience, they learn that certain handles do or do not work. This enhances or concentrates the set of options they consider in steering. In that way, the set of handles is personally defined and alters over time. In this respect, the professionalisation of management aims at providing managers with the right set of handles.

Future Trends in Transformations in the Organisation In this chapter, we discuss future and emerging trends in performance management by focusing on organisational transformations. To this end, we view organisations as a set of activities, coordination-mechanisms and responsibilities, in line with the set-up of Mintzberg (1983). In this set-up, we distinguish three categories of transformation, which will be elaborated on in the following subsections: 1.

Transformation of the responsibilities: from effort-driven towards result-driven management.

2.

Transformation of coordination: from yearly to dynamic resource allocation.

3.

Transformation in activities: from competitive advantage to sustained competitive advantage.

Toward Result-Driven Business Management Performance Management can be used to transform the control style and the budgeting process of the organisation. In addition, it makes the current way of working more explicit and thus transparent. This leads organisations toward a result-driven style of management. The control style and organisation’s budgeting process can have an impact on the way an organisation will set up its performance management model. Different control styles can be identified. Goold and Cambell, (as described in Strikwerda, 2000), have defined three control styles: strategic planning, strategic control and financial control. An organisation should select a dominant style to make clear what the organisation expects from the PMM (De Waal, 2001). In addition, this signals the dominant type of performance indicators which will be used. Financial control will use primarily financial indicators, while strategic planning requires information from non-financial sources as well.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

230 Wondergem and Vincent

The nature of the budgeting process influences the construction of the PMM. In result-based budgets, as opposed to cost-based budgets, the relationship between performance and patterns of action-reaction is more complex. This requires detailed insight into the causal relationship between the MI’s and the links with handles. The use of a balanced scorecard will thus coincide with the construction of more professional models of performance management. Example: Results and indicators: The contact center from example two is directly responsible for the quality of contact only. The contact center has two goals: (1) quality of contact, and (2) costs. In the case of financial control, the contact center mainly will be reviewed on the total costs. Examples of financial

Performance indicator

Measured item

control Costs per complaint

Total costs divided by the total number of complaints

Cost to service a product

Costs of personnel

Total costs of all personal in the contact centers

Personal costs to process the

Non-personal costs of the

Costs of tools used in the contact centers

The additional costs to process

complaints

channel

the complaint

In the strategic planning style not only the costs, but also the quality becomes more important. Then Table 1 gives a reflection of relevant performance indicators. Organizations that change their control style from financial to strategic or vice versa thus also transform their performance management model.

Toward Dynamic Resource Allocation The Deming circle, as stated before, is widely used to manage the process of continuous performance improvement. The Deming circle is executed at strategic, tactical and operational levels. The integration between the management levels (vertical integration) and between the departments on the same level (horizontal integration) should ensure that the actions are coordinated and all directed toward the strategic goals. The Deming circle can be seen as the operationalisation of management control. Management control consists of several control subsystems each with their own view on strategy (Simons, 1995). We mention two types: (1) the diagnostic control subsystem, where strategy is seen as a plan and performance indicators are used as control object and (2) interactive control, where strategy is viewed as a pattern of actions. In our consulting practice, we see that performance management often solely focuses

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Analysing Transformations in Performance Management

231

on the diagnostic control system (Wondergem & Eskens, 2003). The diagnostic control system, however, is a single-loop learning process, while successful implementation of performance management requires at least a double-looplearning process (Kaplan & Norton, 1992). In single-loop learning, the actions are directed towards realising the target of the PI’s. Double-loop learning enables organisations to analyse and revise the assumptions underlying the PMM and uses the evidence to define new norms. With double-loop learning, the organisation can plan the strategy and the necessary actions to realise the business strategy. In that, the organisation uses the PMM to operationalise the business strategy and can evaluate the strategic map and the causal relationships. Double-loop learning can make the connection between the diagnostic and interactive control system. Finally, deutero learning can be distinguished (Argyris, 1982). Deutero learning is about the speed and quality of the learning process and thus influences the flexibility and adaptability of the organisation. Considering the increased dynamic nature of the business environment, we envision that deutero learning will become a strategic necessity for many organisations. In general, the business environment transforms from a make-and-sell environment (industry era) into a sense-and-respond environment (information era). Changes in the environment make strategies obsolete. Therefore, the speed of evaluating the chosen strategy increases and the speed and frequency of walking, through the Deming circle should keep the same pace. As an effect, the strategic planning horizon shortens and budgets become obsolete sooner. As a consequence, information should be available in real time and actions are focused on short-term results. To make sure that the organisation develops the right product features and is able to adapt to the changes in a flexible manner, the performance results must be reviewed more frequently. We envision that, instead of making a yearly budget, organisations make quarterly rolling forecasts and align their resource allocation with the strategic requirements. This is in line with the vision of the Working Council for CFO’s (WCC, 2001). Example: The responsibility of the Contact Center for only the quality of the contact is based on the assumption that the product (make and sell) and the service of the product (sense and respond) can be separated and that the Contact Center is only a service entrance and not a sales point. Nowadays, however, customers do not separate sales and service; customers calling a contact center also want information about products or even want to buy products (cross-selling). Fulfilling customer needs with additional product features (extra games for a Nintendo) has a high impact on the satisfaction of the product. When the satisfaction of the product is declining and the customer Contact Center meets all of their goals (quality of contact, quality of process,

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

232 Wondergem and Vincent

employee satisfaction, channel availability), it is not the current planning that should be revised, but the assumptions that lie beneath it. Changing these assumptions will transform the performance management model.

Toward Sustained Competitive Advantage In the end, Performance Management should help an organisation to create superior performance and sustained competitive advantage. When taking an inside perspective on strategy, the actions should be focused on the enablers: processes, employees and resources. In this light, we mention several research questions that deserve renewed or ongoing attention. First, we mention the issue of process optimisation. A second question amounts to how the productivity of the employees can be increased. A third question centers around how to exploit synergies out of the resources, both tangible and intangible. Given the complexity of the causal relations between MI’s and the organisation model, a resourcebased view on performance management may well become more relevant. We thus propose more attention be given towards an “enabler-oriented” vision on performance management.

Conclusions We have described a framework for analysing transformations in performance management, including the performance management model. Using this framework, several types of transformation were described. The framework describes which elements of the organisation and its management model can be transformed by performance management, as well as the factors that play a role in the transformations. In addition, an initial description of the properties of the transformations was given and future consequences for the organisation were sketched. This chapter has focused strongly on the information that is used in transformations in performance management, as formulated in the goal model and the organisation model. As an additional aspect, the section about future directions sketched possible paths of evolution for organisations that use performance management. Actually using the information in PM was only briefly touched upon in this chapter. It is, however, an important issue since it heavily influences the success of the implementation of PM. The combination of insights into which information is necessary for steering, how to organise performance management

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Analysing Transformations in Performance Management

233

and knowledge of effective ways of actually using the information, will provide better means for successfully implementing performance management.

References Argyris, C. (1982). Reasoning, Learning and Action: Individual and Organisational. San Francisco, CA: Jossey-Bass. Birnberg, J.G., Turpolec, L. & Young, S.M. (1983). The organisational context of accounting. Accounting, Organizations and Society, 8, 111-130. Chang, R.Y. & Morgan, M.W. (2000). Performance Scorecards. San Francisco, CA: Jossey-Bass. COPC. (2002). COPC Performance Management System – Release 3.2B. Customer Operations Performance Center, Inc. Deming, W.E. (1982). Out of the crisis: Quality, productivity and competitive position. Cambridge: Cambridge University Press. De Waal, A. (2001). Towards world-class performance management. Tijdschrift Financieel Management. In Dutch. Harry, M.J. (1998). The Vision of Six Sigma, 8 volumes. Phoenix, AZ: Tri Star Publishing. INK. (2001). Manual for assessing the position of businesses. Zaltbommel, The Netherlands: INK. Kaplan, R. & Norton, D. (1992, January/February). The Balanced Scorecard – Measures that Drive Performance. Harvard Business Review. Kaplan, R. & Norton, D. (2000). The Strategy Focused Organization. Harvard Business School Press. Locuratolo, E. (2002). Designing Methods for Quality. Information Modelling and Knowledge Bases XIII. IOS Press. Mazur, G.H. (1993). QFD for Service Industries. Proceedings of the Fifth Symposium on Quality Function Deployment, Novi, Michigan. Mintzberg, H. (1983). Structures in Five: Designing Effective Organizations. Prentice Hall. Mintzberg, H. (1991). Strategy and intuition – A conversation with Henry Mintzberg. Long Range Planning, 24(2), 108-111. Mintzberg, H. (1994). The rise and fall of strategic planning: Reconceiving roles for planning, plans, planners. New York: The Free Press.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

234 Wondergem and Vincent

Scheipers, G., Ameels, A. & Bruggeman, W. (2002). Value-based management: an integrated approach to value creation. A literature review. In L. Peeters, P. Matthyssens & L. Vereeck (Eds.), Stakeholder Synergie: Referatenboek 25e Vlaams Wetenschappelijk Economisch Congres, Hasselt, (pp. 77-128). Leuven-Apeldoorn: Garant. Simons, R. (1995). Levers of Control – How Managers Use Innovative Control Systems to Drive Strategic Renewal. Boston, MA: Harvard Business School Press. Strikwerda, H. (2000). Het ontwerpen van een Organisatie: de Concernstructuur. Financial Times/Prentice Hall. In Dutch. Van den Bosch, F.A.J. & Van Wijk, R. (2001). Creation of managerial capabilities through managerial knowledge integration: A competencebased perspective. In R. Sanchez (Ed.), Knowledge Management and Organizational Competence. New York: Oxford University Press, 159176. WCC. (2001). Beyond Measurement Alone. Report of the Working Council for Chief Financial Officers, London. Retrieved from the WWW: www.cfo.executiveboard.com Wondergem, B.C.M. (2003). Het Selecteren van een Geschikte Methode Voor het Formuleren van Indicatoren. Proceedings of the Conferentie Informatiewetenschap 2003, Eindhoven, The Netherlands, November 2003. In Dutch. Wondergem, B.C.M. & Wulferink, H. (2002). Prestatie-indicatoren. Informatie, 48-52. In Dutch. Wondergem, B.C.M. & Eskens, J. (2003). Bestuurlijke Begrenzingen van de Balanced Scorecard. Management & Informatie. In Dutch. Yilmaz, M.R. & Chatterjee, S. (1999). Six-Sigma Quality in Small Businesses: A Genesis for Growth. Journal of Business and Management, 6(2), 100115.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Multimedia Conversion with the Focus on Continuous Media

235

Chapter XI

Multimedia Conversion with the Focus on Continuous Media Maciej Suchomski, Friedrich-Alexander University of Erlangen-Nuremberg, Germany Andreas Märcz, Dresden, Germany Klaus Meyer-Wegener, Friedrich-Alexander University of Erlangen-Nuremberg, Germany

Abstract This chapter describes an approach to a very complex subject of multimedia conversion. The authors hope to familiarize readers with the idea of multimedia objects transformation, especially time-dependent data (like video and audio streams). After a short introduction, the chapter gives fundamentals with broad references to the discussed field. The presented way of modeling conversions including an abstract model, categories of transformation and a processing model, leads to an application of the graph-based model in the deployment considering model-specific issues,

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

236 Suchomski, Märcz, and Meyer-Wegener

functional correctness, scheduling, management of resources and admission control. RETAVIC and memo.REAL are ongoing projects and the means of evaluation of the models of conversion as well.

Introduction Multimedia data are ubiquitous today. The formerly separated areas of music recordings, radio, and television are all moving to digital formats, which in essence means that all recordings are becoming data and can be stored and manipulated as data. Standard data storage systems can be used for sound and video, and both can be transmitted over computer networks. The multimedia computer as the endpoint is beginning to replace facilities such as telephone, radio, VCR, TV, and disk players. While this looks like integration and simplification, the computers themselves are anything but homogeneous. They are equipped with many different kinds of displays and audio/video boards, not to speak of software. Hence, the same piece of media content must be available in a large variety of formats. User requirements regarding platform and quality on one hand and resource limitations on the other even increase this variety. The simplest way is to create copies in all known formats, but this has many deficiencies, in particular when updates are necessary. As an alternative, transformations are available. So it seems to be a useful approach to keep media assets in a single copy and in a neutral format, and to transform them on request into the format needed by a particular user. This is even more useful in a large archive of media assets that is used by applications on many different platforms, e.g., in an authoring or teaching scenario. Assume for instance an archive of medical images and videos which must be kept without any loss of information and thus will potentially be rather large. In lectures and other presentations, however, a compressed version on a laptop computer will be more appropriate. While some of the images and videos can be transformed offline before the presentation, a discussion could create the need to access other objects online. Then a transformation at the time of the request is unavoidable. In the following, the term “media object” (MO) will be used for any kind of media data that belong to a single medium, i.e., text, image, audio, or video. Of course, media objects can be combined into multimedia objects (MMO), but their handling must be clarified before. If a media object is available in one form and is then requested in another, it must be transformed. In this chapter, the term “conversion” will be used to denominate all forms of transformation on multimedia data. Many conversion algorithms and programs are at hand, so they should be re-used in this context. In order to fulfill any kind of user request,

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Multimedia Conversion with the Focus on Continuous Media

237

however, it may be required to combine more than one converter. So conversions need to be registered and classified. Many conversions affect the quality of the media objects. In order to cope with the resource limitations in the network and in the end-user devices, the quality must often be reduced. It is important to describe the quality in a number of characteristics like resolution and number of different colours. This allows to find a balance between resource requirements on the one hand and perceived quality on the other. A specific problem arises with media objects that have inherent time, i.e., audio and video. They are often referred to as streams. To create the requested form before beginning the playout would lead to a significant latency that many users do not accept. So the media object should be transformed during the playout, which requires real-time processing. This imposes tough requirements on the system that manages and delivers the media objects especially according to these real-time requirements. Such a system could support a large variety of qualityof-service requests for media objects, but it is of high complexity. The goal of this chapter is to introduce the building blocks of such a system. The different forms of media objects — often defined by standards — must be well understood to be handled properly. Next, after looking at other projects and systems with similar goals, a model of conversion is defined. It allows for classification and provides an abstract view of conversion tasks. These individual conversions are then combined into conversion graphs. To execute them, scheduling and admission control must be handled in cooperation with a real-time operating system.

Fundamentals This section provides technical definitions in the discussed field. Obviously, the meaning of media data and multimedia data should be defined first. Media data are text, image (natural pictures, 2D and 3D graphic, 3D pictures), audio (natural sounds including human voice, synthetic) and video (natural video, 2D and 3D animation, 3D video). These data have a special digital representation when used in computers that is called media object (MO). Multimedia data combines more than one of mentioned media data and is represented by multimedia objects (MMO’s). While text and image are well known and reasonably simple to understand, the emphasis here is on audio and video — called audio stream and video stream respectively from now on. An audio stream consists of discrete values (samples)

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

238 Suchomski, Märcz, and Meyer-Wegener

usually obtained during the process of sampling audio with a certain frequency. A video stream consists of images (frames) that have also been captured with a certain frequency. The characteristic distinguishing these streams from other kind of data is the time constraint: The occurrence of the events (samples or frames) is ordered, and the periods between them are constant. These characteristics specify the continuous properties of a data stream (and justify why they are called time-dependent data or timed data). Thus a media or multimedia object with this time constraint is usually referred to as a timed MO or a timed MMO respectively. Because continuous properties are almost always present in MMO’s, the term timed is usually skipped (just as non-timed is skipped when referring to MO’s which are not time-dependent, e.g., images). By the way, the terms “timed” and “time-dependent” are often interchanged with the term “continuous”. In reality audio-video streams (as well as non-timed objects) often contain additional information that describes the stream (object) itself. It is called meta information, and it includes among others: stream properties (e.g., duration, bit rate) and quality properties (such as resolution, frame or sample rate, etc.). Meta information heavily depends on the data it describes, and because each type of media object differs from the other (e.g., stream properties are not present in non-timed objects), meta information contains different properties. Because an MMO consists of more than one MO, it must first identify the MO’s it includes and second it must store some arrangement information (additional properties in meta information), e.g., temporal and spatial layout. In available specifications like MPEG-4 (Battista et al., 1999) and H.263 (ITU, 1996), the pictures (frames) of a video stream are grouped into groups of pictures (GOP’s) and further into video sequences. To generalize this, a term is adopted here from Gemmel et al. (1995): A quant is a portion of data that is treated as one logical unit occurring at a given time. Representatives of quanta are a sample, a frame, a text, or a combination of them (e.g., a GOP), etc. So, an abstraction of the streams mentioned so far is a multimedia stream as a timed data stream that consists of quanta. In order to explain the transformation of MO’s and MMO’s in the following sections, a logical description is needed. Some models of MO’s and MMO’s have already been defined and discussed in the literature. It is common that each MO has a type, a format and a content. They will be referred to as MO.type (e.g., text, audio, etc.), MO.content and MO.format, respectively. Within MO.format, structure (e.g., frame rate, pixel depth, resolution) and coding scheme (e.g., QuickTime, MPEG-1/-2, DivX, XviD, MP3, AAC) can be further distinguished. An MMO is structured similarly, but it adds to MMO.format data about relations among the included MO’s, i.e., data on temporal/spatial relations.

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Multimedia Conversion with the Focus on Continuous Media

239

Figure 1. Conversion process using multiple converters converter graph converter chain converter

converter

converter

converter

converter

Based on this model of MO’s and MMO’s, all components of the conversion process will now be described. This process is depicted in Figure 1. A converter applies a conversion function to an MMO. It may add, remove, or change content as well as format and media type of any part. A chain of converters couples a few converters in order to perform a conversion process with a more complex functionality. A chain is a directed, one-path, acyclic graph, i.e., a sequential line of converters, which passes MMO’s from one node only to the next. A graph of converters consists of more than one connected converter chains. It is used for instance, if an MMO must be split into MO’s in order to use media-specific converters, i.e., converters working only with a given type and/or format of media object. Regarding timed (M)MO’s, this general model of conversion graphs must be extended to reflect the timing. Instead of whole (M)MO’s, only quanta are passed from one converter to the next in the chain, and that must be done at a particular point in time, e.g., every 40 milliseconds for a video with 25 frames per second. The main goal in modeling timed MO or MMO conversions is to define such an extended model independent of hardware, implementation, and environment (Marder, 2002). In our opinion, a very promising model is that of jitterconstrained periodic event streams proposed by Hamann (1997). The author in 2001 added jitter-constrained data streams which suit multimedia very well. These streams consist of a time stream and a volume stream. The former is defined as t = ( T, D, τ , t0 ), where T is the average event distance (the period), D the minimum distance, τ the maximum jitter (lateness), and t0 the starting point. Analogously, the volume stream is defined as s = ( S, M, á , s0 ), where S is the average quant size, M the minimum quant size, á the maximum jitter (deviation), and s0 the initial value. Later, these models will be used in the deployment phase to derive the important characteristics of the conversion process. Another important issue regarding transformation of timed (M)MO’s is providing and controlling the level of quality. Quality of Service (QoS) is defined by the ITU-T as, “a set of qualities related to the collective behavior of one or more

Copyright © 2005, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

240 Suchomski, Märcz, and Meyer-Wegener

objects” (ITU, 1997), i.e., as an assessment of a given service based on characteristics. Examples of these QoS characteristics (or parameters) are: latency (or delay), jitter in delay, delay bound, loss rate (lost data, delayed data), throughput, jitter in throughput, reliability (process failures, communication breaks), etc. (Vogel et al., 1995; Plagemann et al., 1995; Hafid and Bochmann, 1998). A more elaborate description and classification of QoS characteristics can be found in ITU (1997). In addition to QoS, it is useful to also take Quality of Data (QoD) and Quality of Experience (QoE) into account, as presented in Figure 2. Starting with the data source, a stored object has a certain quality, e.g., resolution and frame rate, which is the QoD. The server provides the data with or without transformation, i.e., it serves the data with a quality level equal to the transformed QoD — indicated by T(QoD) — and with respect to its own services QoSSRV. The network delivers the data to the client with QoSNET and influences as well the overall quality. Additionally, the client side responsible for receiving and presentation to the end user has its own QoSCLI. The end user finally perceives the delivered data and feels some level of experience (QoE; also called subjective QoS) that is most important in the whole process of multimedia delivery. However, discussing all those aspects in more detail goes beyond this work. A possibility of mapping QoE to objective QoS is explained by Plagemann et al. (1995). Here, we focus on the server side and assume that the QoD is given and that the T(QoD) requested by the user must be provided. Then it is necessary to control the quality during serving (or processing). In other words, only the QoSSRV is of interest at this point. In order to guarantee a certain QoS during a conversion process at the server side, the whole process with all involved components (mostly converters) must be scheduled in a real-time environment — with the possibility to control all parts of the process precisely.

Figure 2. Qualities involved in the process of delivering MMO’s to the client End -Point of Interest

NETWORK CLIENT

SERVER QoSNET

QoE