Research Design in Business and Management: A Practical Guide for Students and Researchers 3658343567, 9783658343569

Citation preview

Stefan Hunziker Michael Blankenagel

Research Design in Business and Management A Practical Guide for Students and Researchers

Research Design in Business and Management

Stefan Hunziker · Michael Blankenagel

Research Design in Business and Management A Practical Guide for Students and Researchers

Stefan Hunziker Wirtschaft/IFZ, Campus Zug-Rotkreuz Hochschule Luzern Zug-Rotkreuz, Zug, Switzerland

Michael Blankenagel Wirtschaft/IFZ, Campus Zug-Rotkreuz Hochschule Luzern Zug-Rotkreuz, Zug, Switzerland

ISBN 978-3-658-34356-9 ISBN 978-3-658-34357-6  (eBook) https://doi.org/10.1007/978-3-658-34357-6 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Fachmedien Wiesbaden GmbH, part of Springer Nature 2021 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Responsible Editor: Carina Reibold This Springer Gabler imprint is published by the registered company Springer Fachmedien Wiesbaden GmbH part of Springer Nature. The registered company address is: Abraham-Lincoln-Str. 46, 65189 Wiesbaden, Germany

Preface

Now more than ever, students, instructors, supervisors, and managers must understand the relevance of excellent research. The purpose of research is to find the best or at least good arguments to explain how (parts of) the world works (or how we construct it). Hence, we need to develop sound research designs. A research design is the plan for conducting a research project. It details the procedures used for gaining the information needed to solve research problems in business and management. Simply, it is the general plan of how researchers will answer their research question. Conducting research means the systematic use of a set of methods, tools, and measures to get a better understanding of specific phenomena or real-life events. In social sciences, these phenomena include individuals, groups, teams, organizations, and society. Opposed to many recommendations in the literature, we recommend students to start at the very end of the research process. This is done by envisioning the conclusion to the research project. Developing a well-thought-out conclusion enables to think about a proper research design. Preparing research designs seems to be a very challenging task for many students and researchers. Yet, it is a crucial stage of conducting excellent and relevant research. For this reason, the textbook Research Design in Business and Management embraces theory, concepts, and practical examples to get a sound understanding of how to develop a research plan. Students learn how a research design can be logically executed to achieve relevant conclusions. We encourage students to make use of the offered learning materials at the end of each chapter. The content of Research Design in Business and Management applies to all researchrelated questions in business and management. The focus of the textbook is on improving the quality of intellectual contributions as the result of a well-executed research design. It does not address specific statistical or other research methods in detail, as these topics are well covered in standard textbooks.Didactic Philosophy and Learning Objectives We encourage our audience to apply research designs to contribute to the practice and the scientific community. Students and professionals with research tasks begin their understanding of why research matters in today’s complex business environment. They progress to the challenges of how to develop relevant research questions. Stating v

vi

Preface

a fictitious answer to a research problem helps tremendously to design a sound research plan. Also, we discuss how to choose from different research designs depending on the research problem. To support the reader’s learning success, our approach is to introduce accessible straightforward concepts and means to facilitate the research journey instead of comprehensive deliberations of different approaches and opinions. We believe that decisive opinions and a clear structure support the students’ progress at the beginning of that journey more than meticulous discussions of all existing views on the subject. Thus, you might find sometimes (slightly) different notions or definitions than you already know from other textbooks on research. Hopefully, these divergences or simplifications help our audience to draw a consistent and easier picture about research. We try to complement our recommendations with practical examples from diverse research projects. To help students and professionals with their research project, we paid special attention to the following didactic elements: • clearly structured guidelines for the development of the research design with data collection and analysis, as well as interpretation of results from research aim, research question, and intended (hypothetical) answer. • collection, presentation, and qualification of research designs relevant to business and management, depending on the type of question to be answered. The research designs are enriched with relevant and transferable examples from the corresponding research area. • suitability for self-study. • designed for application-oriented academic work in business and management. The textbook Research Design in Business and Management has been developed for training and further education at the university level in German-speaking countries. Yet, it is also of high relevance for practice. It serves students, researchers, and practitioners facing research problems as a source of ideas on how research can generate value for individuals, companies, industries, and the scientific community. Acknowledgements We have received many valuable comments and suggestions for this textbook during the last few years from students, professors, and researchers. We cordially thank each of these contributors. Also, we wish to thank the following people and institutions: • Lucerne School of Business for its financial support. • Springer Gabler: all colleagues from the editorial, production, and marketing departments for their great support in making this textbook possible.

Preface

vii

• Dr Veronika Halene for her very valuable book review and all critical comments on the contents. • Our families and relatives, for their patience and understanding of the many “writing related absences”. Finally, students in our graduate and undergraduate classes on research design have inspired us to write this textbook. They contributed many thoughtful ideas.

Stefan Hunziker Michael Blankenagel

Contents

1

Introducing Research Designs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1 Why Research Designs Matter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Understanding the Purpose of Research. . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Define Research Design Properly. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.4 Check the Validity of Your Research. . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.5 Embrace Challenges to Implementation and Execution. . . . . . . . . . . . . 1.6 Make the Most of this Textbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 1 2 4 6 12 13 16

2

Understanding Intellectual Contributions. . . . . . . . . . . . . . . . . . . . . . . . . . . 2.1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Normative and Empirical Research. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Types of Intellectual Contributions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3.1 Theories and the Body of Knowledge. . . . . . . . . . . . . . . . . . . 2.3.2 Adding to the Body of Knowledge . . . . . . . . . . . . . . . . . . . . . 2.4 Value of Intellectual Contributions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

19 19 21 22 22 26 33 35

3

Setting-Up the Research Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1 Introduction to the Research Process . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Step 1: Start with the Conclusion you Would Like to Draw. . . . . . . . . . 3.3 Step 2: Defend your Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.4 Step 3: Produce Results for Good Arguments. . . . . . . . . . . . . . . . . . . . . 3.5 Step 4: Choose the Proper Analysis Method(s). . . . . . . . . . . . . . . . . . . . 3.6 Step 5: Collect Proper Data that Fit the Analysis Method(s) . . . . . . . . . 3.7 Step 6: Specify a Proper Preliminary Research Question. . . . . . . . . . . . 3.8 Step 7: Check the Consistency of Your Research Design. . . . . . . . . . . . 3.9 Step 8: Check the Feasibility of Your Research Project . . . . . . . . . . . . . 3.10 Step 9: Conduct Your Research Project. . . . . . . . . . . . . . . . . . . . . . . . . . References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

37 37 38 42 43 45 46 48 48 49 50 52

ix

x

Contents

4

Writing up a Research Report. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.1 Introduction to Writing a Research Report. . . . . . . . . . . . . . . . . . . . . . . 4.2 General Remarks on Scientific Writing. . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Overview of the Sections. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Sections of the Research Report in Detail. . . . . . . . . . . . . . . . . . . . . . . . 4.4.1 Management Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.2 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.3 Theoretical Background. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.4 Literature Review. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.5 Research Design. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.6 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.7 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4.8 Conclusion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

53 53 54 56 58 58 60 61 62 68 78 79 81 84

5

Comparing Types of Research Designs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.1 Recapitulation of Purpose and Content of Research Designs. . . . . . . . . 5.2 Mixed Methods and Staggered Research Designs . . . . . . . . . . . . . . . . . 5.3 Overview of the Most Important Research Designs. . . . . . . . . . . . . . . . References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

85 86 87 89 96

6

Design Science Research Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.1 General Description of Design Science Research. . . . . . . . . . . . . . . . . . 6.2 Peculiarities of Design Science Research Design. . . . . . . . . . . . . . . . . . 6.2.1 Characteristics of Design Science Research . . . . . . . . . . . . . . 6.2.2 Issues to Address in Design Science Research . . . . . . . . . . . . 6.2.3 Major Fallacies in Conducting Design Science Research. . . . 6.3 Writing a Design Science Research Paper . . . . . . . . . . . . . . . . . . . . . . . 6.4 Related Research Designs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

97 98 102 102 104 105 108 112 115

7

Action Research Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.1 General Description of Action Research. . . . . . . . . . . . . . . . . . . . . . . . . 7.2 Particularities of Action Research. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.2.1 Characteristics of Action Research . . . . . . . . . . . . . . . . . . . . . 7.2.2 Issues to Address in Action Research. . . . . . . . . . . . . . . . . . . . 7.3 Major Fallacies in Conducting Action Research. . . . . . . . . . . . . . . . . . . 7.4 Writing an Action Research Paper . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7.5 Related Research Designs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

117 118 121 122 124 126 130 134 137

Contents

xi

8

Single Case Research Design. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8.1 General Description of Single Case Research Design. . . . . . . . . . . . . . 8.2 Particularities of Single Case Research Design . . . . . . . . . . . . . . . . . . . 8.2.1 Characteristics of Single Case Research . . . . . . . . . . . . . . . . . 8.2.2 Issues to Address in Sinlge Case Research . . . . . . . . . . . . . . . 8.2.3 Major Fallacies in Conducting Single Case Research. . . . . . . 8.3 Writing a Single Case Research Paper. . . . . . . . . . . . . . . . . . . . . . . . . . 8.4 Related Research Designs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

141 142 148 148 150 156 159 165 168

9

Multiple Case Research Design. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.1 General Description of Multiple Case Research Design. . . . . . . . . . . . . 9.2 Particularities of Multiple Case Research Design. . . . . . . . . . . . . . . . . . 9.2.1 Characteristics of Multiple Case Research . . . . . . . . . . . . . . . 9.2.2 Issues to Address in Multiple Case Research. . . . . . . . . . . . . . 9.2.3 Major Fallacies in Conducting Multiple Case Research. . . . . 9.3 Writing a Multiple Case Research Paper . . . . . . . . . . . . . . . . . . . . . . . . 9.4 Related Research Designs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

171 172 173 173 175 176 177 181 186

10 Cross-Sectional Research Design. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.1 General Description of Cross-Sectional Research . . . . . . . . . . . . . . . . . 10.2 Particularities of Cross-Sectional Research . . . . . . . . . . . . . . . . . . . . . . 10.2.1 Characteristics of Cross-Sectional Research. . . . . . . . . . . . . . 10.2.2 Issues to Address in Cross-Sectional Research. . . . . . . . . . . . 10.2.3 Major Fallacies in Conducting Cross-Sectional Research. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10.3 Writing a Cross-Sectional Research Paper. . . . . . . . . . . . . . . . . . . . . . . 10.4 Related Research Designs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

187 187 189 189 190

11 Longitudinal Research Design. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.1 General Description of Longitudinal Research. . . . . . . . . . . . . . . . . . . . 11.2 Particularities of Longitudinal Research Design. . . . . . . . . . . . . . . . . . . 11.2.1 Characteristics of Longitudinal Research Design . . . . . . . . . . 11.2.2 Issues to Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2.3 Major Fallacies in Conducting Longitudinal Studies. . . . . . . . 11.3 Writing a Longitudinal Research Paper . . . . . . . . . . . . . . . . . . . . . . . . . 11.4 Related Research Designs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

201 202 203 203 204 211 213 216 219

193 194 197 199

xii

Contents

12 Experimental Research Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12.1 General Description of Experimental Research Designs . . . . . . . . . . . . 12.2 Particularities of Experimental Research . . . . . . . . . . . . . . . . . . . . . . . . 12.2.1 Characteristics of Experimental Research Design. . . . . . . . . . 12.2.2 Issues to Address in Experimental Research. . . . . . . . . . . . . . 12.2.3 Major Fallacies in Conducting Experimental Research. . . . . . 12.3 Writing an Experimental Research Paper. . . . . . . . . . . . . . . . . . . . . . . . 12.4 Related Research Designs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

221 222 223 223 224 228 229 231 233

13 Literature Review Research Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.1 General Description of Literature Review Design . . . . . . . . . . . . . . . . . 13.2 Particularities of Literature Review Research Design. . . . . . . . . . . . . . . 13.2.1 Characteristics of Literature Review Research Design. . . . . . 13.2.2 Issues to Address . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.2.3 Major Fallacies in Conducting Literature Review Research. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13.3 Writing a Literature Review Research Paper . . . . . . . . . . . . . . . . . . . . . 13.4 Related Research Designs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

235 236 237 237 238 242 243 245 250

1

Introducing Research Designs

Learning Objectives

When you have finished studying this chapter, you will be able to: • • • •

understand the function of a research design explain the main purposes of research differentiate between exploratory, descriptive, and explanatory research understand the main challenges of good research and why research is important to better understand real-world problems • understand the importance of internal, external, and conclusion validity

1.1 Why Research Designs Matter Nobody can explain the world in its entirety. This has been true throughout the history of humanity. Obviously, this is specifically relevant in today’s complex and dynamic world. We lack knowledge and insight, and the more we learn, the more we realize how little we know. Good research design helps us to fill up the gaps in our knowledge. We focus on the gap that we perceive as most pressing, interesting, or pertinent. Gaining and accumulating knowledge is an arduous process, taking up time and effort. Specifically, in business and management, researchers should spend these resources effectively and efficiently. And this is the purpose of research design. It ensures that the research process generates the answers that we seek, and the process focuses on the essentials to achieve this result. Hyped phrases like “fake news” and “conspiracy theories” imply divergent views on what is true and how the world works. This leaves us with three options:

© The Author(s), under exclusive license to Springer Fachmedien Wiesbaden GmbH, part of Springer Nature 2021 S. Hunziker and M. Blankenagel, Research Design in Business and Management, https://doi.org/10.1007/978-3-658-34357-6_1

1

2

1  Introducing Research Designs

• toss a coin or dice, • accept what we heard somewhere and what we somehow liked without proof, or • conclude how (or how we perceive how) the world works by finding and using valid and convincing arguments. The purpose of good research designs is to find the best or at least good arguments to explain how (parts of) the world works (or how we construct it). Thus, research design is rooted in logic. In fact, it is an effort in logic, enabling us to draw, argue, and assess conclusions and add to the knowledge about the world. de Vaus (2001) states: “the function of a research design is to ensure that the evidence obtained enables us to answer the initial question as unambiguously as possible.” (p. 9). Many textbooks today examine philosophical discussions about the reality (ontology), how we can gain what forms of knowledge (epistemology), how to conclude with sound deductive and strong inductive reasoning, and how to proof, confirm, and infer (logic). These underlying concepts are important. However, we find they rarely have a powerful impact on the research design itself. We will refer to them only if they are relevant for drawing arguments and conclusions from a specific research design or study’s results.

1.2 Understanding the Purpose of Research The term research is derived from the French word recherche with the meaning act of searching closely. Thus, research can be understood as a systematic and replicable process to add to the body of knowledge. This process encompasses the identification of research problems within specified boundaries and well-designed methods to collect and process information. Also, an important part of the research process is to disseminate the findings to develop, refine, or falsify theories. We can differentiate between three types of research, namely exploratory, descriptive, and explanatory research. Yet, these categories are not mutually exclusive, they are rather a matter of emphasis. Unfortunately, too often textbooks link these purposes with certain data types, methods and even research designs. This establishes a false and undifferentiated preconception of what specific methods and designs are good for. We like to replace this with a more systematic approach in this book. We believe that the understanding of the different purposes of research will help later to specify our desired conclusions and arguments. First, we briefly discuss these three types of research purposes below. Exploratory research We can characterize exploratory research by its flexibility. If a research problem is broad and not precisely defined, you may use exploratory research as a starting point. Exploratory studies are a valuable means to understand what happens; to seek new insights, to ask critical questions, and to assess phenomena in a new light (Yin, 1994).

1.2  Understanding the Purpose of Research

3

The goal of exploratory research is to identify and develop fresh problems, clarify, refine concepts, and create or develop hypotheses. Exploratory research produces initial and unsystematic data that could be used to identify a particular question for a subsequent study. It is not aimed at producing systematic, clearly defined data from which theories can be confirmed. Eventually, that is why it is called exploratory (Yin, 2014). Example

An example in business and management might be an exploratory study of pricing models for value-based health care. Let us assume that at the time this textbook was written, the amount of knowledge about value-based health care and the pricing was very limited. Thus an exploratory study can shed some light on this topic. Researchers can collect data and information on • • • • •

what is value-based health care, how this value might be determined, what impact the health care system of different countries might have, who the different players in health care are, what these player’s perspectives on “pricing” are, if somebody already has introduced value-based health care, and • what their opinion and experiences are. All the above are valuable contributions, even if it is yet too early to come up with any theories or designs for pricing models. ◄ Descriptive research Contrary to exploratory research, descriptive research tries to describe a specific phenomenon. Thus, such research defines questions, samples, and the research methods before the process of data collection and analysis. To put it differently, descriptive research depicts who, where, when, what, and sometimes how much aspects about the phenomenon, and even how and why aspects (from the perspective of the researched subjects). We identify some overlap in the research purposes. When we ask people why they bought this product, it is a description of their purchasing behavior but also the first step to explain the reasons for realized purchases. Descriptive research should be thought of as a means to an end rather than an end itself, as Yin (1994) suggests. As the term descriptive says, this research type aims to provide an accurate description of observations of real-world phenomena. For example, the goal of the collection of census data is to describe basic information about the EU population at a particular point in time. Descriptive studies have more guidelines than explorative research. They describe people, products, organizations, and situations. Descriptive studies have one or more guiding research questions. Data from descriptive research may be qualitative or quantitative, and quantitative data presentations are normally limited to frequency distributions

4

1  Introducing Research Designs

and summary statistics, such as averages. Customer satisfaction surveys, presidential approval polls, and class evaluation surveys are examples of descriptive studies (Sue & Ritter, 2007). Example

A detailed set of data on the profile of customers of a specific university degree program is an example of this type of research. By understanding the customer in more detail, sales and marketing executives will make more informed decisions on how to further develop and position the program on the market. ◄ Explanatory research The primary purpose of explanatory research is to explain why and how phenomena occur and to predict future occurrences. If the focus lies on cause-effect relationships, this type of study can explain which causes lead to what effects (Yin, 1994). Our primary interest is in the casual analysis of how one (set of) variable affects changes in another variable under which conditions. Thus, explanatory studies are characterized by research questions that specify the nature and direction of the relationships between or among variables being studied. We might accomplish this purpose by many different research designs. Example

An explanatory study may investigate why organization X reorganized its innovation process in a particular way. Another may investigate how different promotions affect customer preferences or which variables contribute the most to customer satisfaction and the relative weight of each variable. Here the explanatory research attempts to understand how different variables contribute to customer dissatisfaction (Sue & Ritter, 2007). ◄ As we have briefly addressed the purpose(s) of research, we can go on with the definition of the most relevant term of this textbook, i.e., research design.

1.3 Define Research Design Properly Following the famous Cambridge Dictionary, we define a design as “a drawing or set of drawings showing how a building or product is to be made and how it will work and look” or more broadly “a plan or specification for the construction of an object or system or for implementing an activity or process, or the result or specification as a prototype, product or process” (Cambridge n.d.a).

1.3  Define Research Design Properly

5

 We understand research design as the specific combination of decisions within a research process that enables us to make a certain type of argument by answering the research question. By “type of argument” we mean a theory improving (e.g., theory development or refinement) conclusion drawn from our research without any connotation about what the direction of the argument is. In this sense, it is a hypothetical or fictitious argument. Making the actual argument is obviously only possible after having conducted the research. For example, the type of argument could confirm the theory by testing a sample (we do not know yet). However, the actual argument might be that we could not confirm the theory by testing this sample. So, research design is the implementation plan for the research study. That plan allows us to accomplish the desired (type of) conclusion by answering the research question. A research design refers to the overall strategy that a researcher integrates the different components of the research study in a logical way (de Vaus, 2001; Trochim, 2005). By doing so, you will address the research problem; it serves as the blueprint for the collection, measurement, and analysis of data. No research design is better than another. It is only better suited to answer a specific research question. Each has its use in answering specific types of research questions and improving theories. But none suits all research questions. We often divide research into qualitative and quantitative research or methods. This is quite confusing on several levels. A method is “a particular way of doing something” (Cambridge n.d.b). For this textbook, we use the term methods for the operational procedures within a research study. We can differentiate data collection, methods of sampling, and methods of data analysis. For example, applying a “t-test” is one method. Grouping these methods into qualitative and quantitative methods does not add much value, especially since the classification of some methods is not clear. Is counting how many times the word “win” is mentioned in an interview a qualitative or quantitative method? To be more precise, we should include the transformation methods for the data collected and apply the terms quantitative and qualitative to each method. So, we can collect qualitative and quantitative data, we can transform qualitative data into quantitative data and vice versa, and we can analyze qualitative and quantitative data. Of course, the choice of methods is part of the research design. In contrast, we define a methodology as “a set of methods used in a particular area of study or activity” (Cambridge n.d.c). For example, this set comprises how you collect your data (e.g., document analysis, interviews), how you analyze your data (e.g., content analysis, correlation analysis) and the rationale for selecting this set of methods. Usually, differentiating between qualitative and quantitative research refers to such sets of methods. But as you can see from the previous argument, this neglects the possibility of combinations and transformations. Specifically, these methodologies are not the starting point of any research project. A researcher might prefer one methodology because he or she is well versed in its methods. Yet, limiting oneself to one methodology implies

6

1  Introducing Research Designs

only researching specific kinds of questions. It also implies only using specific research designs. On the other hand, we might employ a method in various research designs.

1.4 Check the Validity of Your Research The good news is that no research design is inherently better than another. But it may be better or even uniquely suited to draw the desired conclusion and to answer the research question. The quality of the research design is determined by its ability to make the desired type of argument. One important check for the quality of a research design is the study’s validity (Trochim, 2005). We can differentiate between four types of validity: conclusion validity, internal validity, external validity, and construct validity. Conclusion validity was originally labeled statistical conclusion validity. Here, we define conclusion validity as the degree to which conclusions we reach about detected relationships in our research are plausible and reasonable. The quality of excellent research is determined by the researcher’s ability to draw accurate (credible, believable) conclusions from the research process about relationships, independent of the chosen research design. Importantly, this is relevant in quantitative and qualitative research. For example, when a researcher investigates a relationship of elements (under certain temporal or contextual conditions), she or he can end up with two conclusions: either there is a relationship according to the data or there is not. In either case, however, she could be wrong in her conclusion, as Fig. 1.1 illustrates. Conclusion validity only deals with the question if there is a relationship or not given the data analyzed in the research project. You might be more familiar with the term internal validity. Internal validity is related, but not the same as conclusion validity. Internal validity is concerned with the causality of the relationship (Trochim, 2005). Thus,

research result

Fig. 1.1   Relationship or no relationship between elements element X

actual

element Y

research result research result

element X

actual

research result

element Y

1.4  Check the Validity of Your Research

7

internal validity raises whether the research findings accurately reflect how variables or constructs are really connected to each other. For example, we may be concerned about if there is really a causal relationship where A causes B? Maybe, the true relationship is that A causes C, and C essentially causes B. For example, in a program evaluation of a master’s program, we might conclude that there is a positive relationship between accomplishing the optional course “research design (RD)” and the grade of research design related courses (like the master’s thesis). The research design is a cross-sectional study. We can collect data and analyze the participation in the RD course and the grades in the master’s thesis). We may find that students in the RD course get higher grades in the master’s thesis and students not in the RD course get lower ones. Conclusion validity is essentially whether that relationship is reasonable, given the data. What we have detected is that there is a relationship. That is perfectly fine for the argument: there is a (positive) relationship between participation in the RD course and the grade in the master’s thesis (conclusion validity). However, the research design is totally unsuitable for an argument like “by visiting the RD course, students learn a lot about research, so they improve their grade in the master’s thesis. Is the detected relationship causal? We do not know. Maybe another factor (not the RD course) is responsible for better grades. Maybe the “RD group” is just smarter than the comparison group. Confirming the assumed causal relationships (internal validity) requires another research design and is often hard to establish. Thus, it is possible to conclude that our RD course and the grades of our students are indeed related (conclusion validity). We can conclude that the better grades were caused by some variable other than the RD course (i.e., internal validity is violated). In this sense, internal validity deals with causal relationships and conclusion validity deals with the chance to detect relationships: if there is none or vice versa (also referred to as type I and type II errors). The following two examples illustrate what we mean by internal validity. Example

Example 1 (adopted from Brahler, 2018) The managers of two apartment buildings in the same city with electrical heating were concerned about the electricity consumption of the residents. Electricity was included in the monthly rental fee. They decided to conduct an experiment. First, in April, they read each meter three times a week at the same time of day in both buildings. Then, on May 1, a meeting was called for residents in Apartment 1. They were told that the total savings earned by the building, based on a comparison to their consumption in the previous month, would be divided by the number of apartments, and a cheque for that amount would be sent to each of them. Residents in Apartment 2 also had a meeting on that same day, but they were told only of the ecological benefits of

8

1  Introducing Research Designs

energy conservation. Electricity consumption in Apartment 1 was 15% lower in May than in April, but in Apartment 2 it remained unchanged. Can we conclude that the group contingency (whether they belonged to group “apartment 1” or “apartment 2”) was effective in reducing electricity consumption? Internal validity: YES. In this example, we might suspect history to be a threat to internal validity, but the proper control is in place to rule it out as a confound. The decrease in electricity consumption by residents in Apartment 1 may be due to the group contingency or outside events during the experiment, a likely candidate being warmer weather in May than in April. However, residents in Apartment 2 would also have experienced the warmer weather and yet we do not observe a reduction in their electricity consumption. Thus, the group contingency appears to be the cause of the reduced use of electricity by residents in Apartment 1 (example adopted from Brahler, 2018). Example 2 (adopted from Brahler, 2018) Based on her own experience, Professor Lucas strongly believed that study groups enhance student learning. During the very first class she taught, she encouraged her students to contact Joe, a Learning Skills Specialist, at Counselling Services. She told them that Joe and his colleagues would help them form groups, decide for where and when to meet, and supervise the meetings. About half the students took Professor Lucas’ advice. They met once a week in their respective study groups for the duration of her course. Counselling Services kept a record of participating students, which Professor Lucas used for her data analysis. Her dependent measure was the final exam grade, which was marked by her teaching assistant. She discovered that students who met weekly with study groups (Study Group condition) scored two letter grades higher than students who did not meet weekly with study groups (Control condition). Decide if it is correct that she attributed the higher exams scores to the study group meetings. Internally valid: NO. This is an example in which selection is a threat to internal validity. The students in the two comparison groups are unlike with respect to whether they met weekly with study groups and with respect to whether they volunteered to be part of study groups in the first place. The volunteers in Study Group condition may be more highly motivated to succeed than the non-volunteers in the Control condition, and they may have done better on the final exam regardless of meeting with their study groups. The higher final exam score by the Study Group subjects may be due to the study group procedure or to a subject-related variable more likely to be found in volunteers, such as a higher motivation to succeed (example adopted from Brahler, 2018). ◄ Another type of validity is called external validity. We relate external validity to generalizing the findings to the intended population or setting. It refers to the extent to which the results of a research project can be generalized. For example, if researchers draw

1.4  Check the Validity of Your Research

9

unjustified conclusions that their results from the insurance industry are also valid for other industries, external validity is violated. We refer the fourth type of validity to as construct validity. Construct validity checks if a variable, a measurement tool, or a construct really represents the content you like to measure. We define the main idea of validity as “the degree to which a test measures what it claims, or purports, to be measuring” (Brown, 1996, p. 231). To put it a bit more complex, construct validity represents the degree to which inferences can be made from the operationalization of a construct to the theoretical construct on which your operationalization is based. In this sense, construct validity is also somewhat related to generalizing (Trochim, 2005), but that might be to narrow an approach. More broadly we might understand construct validity as a “labeling” issue. For example, if we use the construct “risk culture” as a dependent variable, do we really measure “risk culture” or something different? This leads to the question of interpretation. Interpretation includes the issue of labelling and the issue of measurement itself. Can we measure without interpretation? This is sometimes referred to as instrument validity, but we would like to subsume it under construct validity as well. Overview

As we describe in the next chapter, all researchers seek to contribute to the body of knowledge, but they do it with different approaches. Quantitative researchers (we dislike that term, but for the following discussion it might be helpful to use this differentiation) focus generalization and statements for an entire population. In contrast, qualitative researchers produce intellectual contributions more from the perspective of purpose and intent. In social sciences, the two approaches may deal with the same real-world phenomena (e.g., fraud, innovation, motivation, leadership, etc.) but with different methods, interests, and logics. Research that draws primarily on qualitative methods is characterized by constructing comprehensive, plausible, meaningful, and coherent results that consider the actions of different individuals and groups. This kind of qualitatively dominated research needs specific skills and competencies with which qualitative researchers produce intellectual contributions. We can express interpretation as the quest for the meaning of a phenomenon. Ever since researchers conduct qualitative research, this has been the critical cornerstone of the research process. Qualitative researchers are usually challenged to explain the criteria they apply to support their interpretations and measurements. The crucial question here is how meaningful interpretations can be distinguished from creative or even spurious conclusions. Of course, interpretation is a vital part of all research, as researchers deal with interview protocols, business reports, annual reports, informal talks, observations, and many other data sources.

10

1  Introducing Research Designs

Interpretation poses–as the experience of both authors confirms–a huge challenge, specifically for (unexperienced) students. Interpretation is a task that requires practice, experience, sensitivity, and a great deal of imagination. This task might appear daunting that some researchers try to do completely without interpretations. Many of our undergraduate and graduate students eliminate the risk of interpreting things by citing extracts of interviews from different interviewees in a rather fragmented way and then label this as “results”. Surely, no interpretation is complete and represents the full truth. New interpretations may find their way into the current body of knowledge, previous interpretations may become obsolete. Yet, this inability to accomplish definitive interpretations does not make every interpretation equally useful or valid. How then can we test the scientific quality of our interpretations? Remember, no interpretation can be scientifically proven. However, we may corroborate interpretations through some useful techniques. For example, eliminating counter-interpretations may be an effective starting point for supporting an interpretation. The aim of these techniques is to produce robust and plausible interpretations (Gabriel, 2018). This issue of interpretation is not limited to qualitative methods. It is just more openly apparent. But quantitative methods also use measurements like for example profit (that is not directly observable but needs to be calculated following localized norms that all offer some freedom) or EVA (where you not even know which of the originally prescribed conversions have been applied) that involve interpretations. Thus, in many research projects, it has become the standard to incorporate a statement of the researcher’s position and its influence on the research process and outcome, i.e., its validity.

Concept of reflexivity One major concept in the quality’s context of primarily qualitative research is called reflexivity. Reflexivity serves as an evaluating criterion that assesses the quality and rigor of qualitative research (Hall and Callery, 2001; Cohen and Crabtree, 2008). The concept of reflexivity is nothing new. It has been discussed for many decades (Charmaz, 2017). Generally, reflexivity addresses the crucial question of subjectivity in research and turns this threat into an opportunity for (qualitative) researchers. Reflexivity and its impact on the research process are well documented and acknowledged (Subramani, 2019 and cited literature). We may differentiate between prospective and retrospective reflexivity. Prospective reflexivity deals with the impact of the researcher on the research project. In contrast, retrospective reflexivity considers the impact of the research process on the researcher (Attia & Edge, 2017). If we conduct a research design that is dominated by qualitative methods, it is important to understand the bidirectional relationship between researcher and research.

1.4  Check the Validity of Your Research

11

Still, many researchers consider qualitative methods inferior to quantitative methods. Yet, qualitative methods have become increasingly relevant and more accepted within business and management research. With this, also reflexivity has become more important. Simply, reflexivity is defined as the awareness of the researcher’s role in its pursuits of the research goals. We may define it as the process by which research turns back upon and considers itself (Alvesson, 2004). Thus, reflexivity requires thinking about how we think, how we constantly challenge the existing body of knowledge, how we revise it in the light of new intellectual contributions, and how this in turn affects the research process. Alvesson and Skoldburg (2000) suggest that both interpretation and reflection represent key aspects of reflexive research. We base interpretation not only on a simple analysis of data. Rather, we must be aware that interpretation is influenced by e.g., personal values, use of language, and the political position. The second element is reflection. Reflection means where the researcher turns attention to themselves and their research community. Also, reflection deals with the researchers intellectual and cultural conditions and traditions informing the research. So, reflection becomes a sort of interpretation of the interpretation. This makes the research so called reflexive. Researchers reflect on how their perceptual, theoretical, ideological, cultural, and cognitive assumptions inform the interpretation. It is clear now that reflexivity goes far beyond pure reflection on the research process and outcomes (Haynes, 2012). Reflexivity has lot do to with self-critique. This rather unsettling aspect is common in many reflexive approaches. Researchers consciously challenge their basic assumptions and values. We may also say that by doing so, ontology and epistemology interact, i.e., the researcher’s self and knowledge challenge each other. Reflexivity requires the researcher to reassess the views on theory, methods, data, and his self. Specifically, in research projects where researchers and practitioners collaborate, politics, hierarchy, and authority are crucial elements to be reflexive upon. Ignoring reflexivity in this context means also ignoring the different dynamics that come into play and impact the research process (Haynes, 2012; Orr & Bennett, 2009). Researchers may consider the following steps to foster reflexivity and reflexive research: • designing research that includes multiple perspectives can increase the dialogue. This may lead to complementary and divergent understandings of a phenomenon under investigation. The approach of involving multiple perspectives on the same topic or question (e.g., several interviewees) in a research project supports a reflexive dialogue. • develop a reflexive journal. This might be comparable to a booklet where researchers make entries during the research process regularly. Such entries may explain methodological decisions taken during the process, the logistics of the research project, and reflection upon what happens about the own values and interests (Lincoln & Guba, 2006).

12

1  Introducing Research Designs

• report research perspectives, positions, and values found in the research literature. It may be useful for researchers to briefly disclose how their preconceptions, values, and assumptions may affect the research process (Koch & Harrington, 1998; Lincoln & Guba, 1985; Malterud, 2001). To sum up, reflexive research is very important, specifically in qualitatively dominated research designs. The overarching reason to embrace this concept is to ensure credibility of the research results by reducing the likelihood of the researcher (significantly) biasing the research. Researchers have to deal with biases that could negatively affect the quality of the research project (e.g., data collection tools, data analysis, interpretation of data). Remember that theses biases are also relevant in pure quantitative studies. Yet, it is easier to control for many of them compared to qualitative research designs. To increase the different validity aspects in a research project is not always easy. Researchers face many challenges to implementation and execution of valid research designs. In the following, we briefly discuss some of these challenges. We will delve into more details in the corresponding main chapters on each research design later in this textbook.

1.5 Embrace Challenges to Implementation and Execution This textbook is designed to support students, researchers and professionals contributing to the body of knowledge by developing and executing appropriate research designs. Yet, this seems to be a very challenging task. We, the authors of this textbook, can confirm this based on several decades of experience as degree program directors and supervisors of many research projects. In our experience, the main challenges of research projects are manifold: • • • •

• • • • •

difficulties to produce an adequate research question, problems to design a plan for a research study that answers the question, poor envisioning of the conclusion researchers like to draw, bad conclusion validity, especially due to, – choosing proxies that do only partially reflect the underlying concepts – unjustified generalizations – unjustified claim of a causal relationship (vs. correlation) lack of thought through operationalizations (how do I measure, transform, etc.), lack of transparency, lack of theoretical references, difficulties to write a sound and critical literature review, and confusion of existing literature with theory or theoretical background.

1.6  Make the Most of this Textbook

13

In our experience, the first two challenges are the most crucial ones. Even worse, these aspects aggravate each other. Language problems can also exacerbate the difficulties. Being precise and accurate is important in determining research question and design. A brief example may help to illustrate the first challenge: “how often do companies go bankrupt?” might contain good ideas, but the answer is too simple: “once”. Or as in an analogy from medicine, patients might die of cancer once, never twice. Stating an inappropriate research question makes it difficult to define the reasoning and evidence needed that enables students to answer the question. Yet, there is more to add to this challenge. For example, one study came to the startling conclusion that football spectators feel safe in stadiums. The researchers interviewed spectators sitting in the stadiums. It is quite compelling that if they were afraid, they would not have attended the football match. What can we conclude from this? Simply, the research design is flawed as the sample of interviewees is heavily biased. The reversal of the research process that leads to the above two problems is also quite common. Students start with a research methodology they are already familiar with, such as conducting and analyzing interviews (i.e., a qualitative research approach). Motivated by their preferred research methods, they then look for a research question that might fit their research methodology. Before we now delve into the next chapter, let us give you some recommendations on how you may use this book.

1.6 Make the Most of this Textbook Research projects are usually the culmination of any degree program. Such a project provides you with an opportunity to explore, confirm, pursue, explain, or identify additional aspects of a business and management subject you are interested in. This textbook supports you with your research projects in the following ways: • As a textbook, it provides a comprehensive guideline to your research project. You can use it as accompanying course literature or for self-studying to gather sufficient input for setting up your own research project: where to start, what to do and how to report it. It allows you to make informed decisions at all the major decision points you will run into in your own research project. To reap the full benefits of this book, we recommended to read it from start to finish. • As a reference it provides additional inputs, crucial aspects to remember and pitfalls to circumvent for specific issues you will deal with throughout the research process. Maybe you have questions about how to conduct your literature review, or how to write up the results of your literature review. Or you look for hints about what to do when “discussing” the results of your research. For any of those issues, the book can act as a reference for your future research projects. For this purpose, please jump directly to the specific chapters. Yet, at this point, you already need to be acquainted

14

1  Introducing Research Designs

with the structure of the book to find all the input you look for. For example, you will find information about the literature review in Sect. 4.4.4 as well as more research design specific information in the corresponding section in your research design. To facilitate finding the pertinent information you are looking for, we did our best to envision various requirements you might have for your upcoming research project: • you have not yet been confronted with “research” or have not been given a comprehensive overview of what is involved in conducting research: You may look at the first three chapters of this book. • you have “research modules” in your degree program and you like to know what this is all about and want to understand what is expected of you: You may look at Chaps. 3 and 4 • you must conduct a research project and you look for guidance on how to tackle your task: – get an overview of what you need to do: Look at Chap. 3 and Sect. 4.3. – an initial idea crossed your mind about what your research might be about, and you need to plan your research: Go to Chap. 3 – you have a great idea about what you like to achieve, and you now look for the appropriate research design. Here, go directly to Chap. 5 and get acquainted with the suggested research designs and their adjacent research designs. – you are asked to write up your research report: Chap. 4 and the third section (x.3) of each research design might support you in this. Having said this, we would like to conclude this introductory chapter and start with the discussion of intellectual contributions in the next chapter. Key Aspects to Remember

Explain what we mean by research design Research design specifically combines decisions within a research process that enables us to make a specific type of argument by answering the research question. It is the implementation plan for the research study that enables reaching the desired (type of) conclusion. The research design specifies, lists, or characterizes the constructs, proxies and data and the methods used in the research process. Understand the difference between methodology and method A methodology is a (given) system or set of methods used to establish a data source, collect data from it and analyze this data. In the broadest meaning of the term

1.6  Make the Most of this Textbook

15

“research” itself is a methodology: it is a system of ways to generate intellectual contributions to the body of knowledge. In contrast, a method is a particular way of doing something. (Cambridge n.d.b) There are a lot of well-established methods available in science. They usually refer to a specific task like “comparing the mean of a variable among different groups” or “collecting data by conducting semi-structured interviews”. Be familiar with the purpose of research Basically, research is a systematic and replicable process to add to the body of knowledge. This process consists of the identification of research problems within specified boundaries and well-designed methods to collect and process information. Also, an important part of the research process is to disseminate the findings to develop, refine, or falsify theories. The purpose of research may be further broken down into exploratory, descriptive, and explanatory research. Recognize the need to develop sound arguments The purpose of excellent research is to find the best or at least good arguments to explain how (parts of) the world works (or how we construct the world). Thus, we may conclude that research is rooted in logic. An argument is the conclusion of a research project providing a reason or reasons that are based on empirical and/or theoretical evidence why you support or oppose an idea, theory, or suggestion. As you can see from this, developing good arguments is a crucial part of the research process. Understand the relevance of conclusion validity Conclusion validity is the degree to which conclusions we reach about relationships of elements and conditions in our data are reasonable. To put it differently, the quality of excellent research is determined by the researcher’s ability to draw accurate, credible, and believable conclusions from the research process. Importantly, conclusion validity is relevant in quantitative and qualitative research. For example, never claim that there is a causal relationship if you lack arguments. Or increase conclusion validity by using larger samples. Embrace reflexivity in your research Reflexive research means interpretation and reflection of the researcher’s role in the research process. First, interpretation is an important part of your research. Yet is not only based on a simple analysis of data. Rather, you must understand that interpretation is influenced by, e.g., your personal values, the use of language, and your political position. Second, reflection means where you turn attention to yourself and your research community. Researchers reflect on how their perceptual, theoretical, ideological, cultural, and cognitive assumptions inform the interpretation. If you will, reflection becomes a sort of interpretation of the interpretation. This makes the research so called reflexive. ◄

16

1  Introducing Research Designs

Critical Thinking Questions

1. Why is it important to differentiate between research design and methodology? 2. What role do arguments play in good research? 3. How can researchers increase conclusion validity? 4. What are the two most relevant challenges to implementation and execution? 5. Why is it not useful to differentiate between qualitative and quantitative research?

Recommendations for further readings on research in general

Hancké, B. (2009). Intelligent Research Design: A Guide for Beginning Researchers in the Social Sciences (Kindle-Position9). Kindle-Version. de Vaus, D. A. (2001). Research design in social research. Reprinted. Los Angeles: SAGE Publications, Inc. Recommendations for further readings on reflexivity Berger, R. (2015). Now I see it, now I don’t: researcher’s position and reflexivity in qualitative research. In Qualitative Research, 15 (2), pp. 219–234. Engward, H. & Davis, G. (2015). Being reflexive in qualitative grounded theory: discussion and application of a model of reflexivity. In Journal of Advanced Nursing 71 (7), pp. 1530–1538. Gabriel, Y. (2018). Interpretation, Reflexivity and Imagination in Qualitative Research. In, pp. 137–157. Haynes, K. (2012). Reflexivity in qualitative research. In Qualitative Organizational Research: Core Methods and Current Challenges, pp. 72–89.

References Alvesson, M., & Skoldburg, K. (2000). Reflexive methodology. SAGE. Alvesson, M. (2004). Reflexive methodology: New vistas for qualitative research. SAGE. Attia, M., & Edge, J. (2017). Be(com)ing a reflexive researcher: A developmental approach to research methodology. Open Review of Educational Research, 4(1), 33–45. Brahler, C. (2018). Chapter  9 “Validity in Experimental Design”. University of Dayton. Retrieved May 27, 2021, from https://www.coursehero.com/file/30778216/ CHAPTER-9-VALIDITY-IN-EXPERIMENTAL-DESIGN-KEYdocx/. Brown, J. D. (1996). Testing in language programs. Prentice Hall Regents. Cambridge University Press. (n.d.a). Design. In Cambridge dictionary. Retrieved May 19, 2021, from https://dictionary.cambridge.org/dictionary/english/design. Cambridge University Press. (n.d.b). Method. In Cambridge dictionary. Retrieved May 19, 2021, from https://dictionary.cambridge.org/dictionary/english/method. Cambridge University Press. (n.d.c). Methodology. In Cambridge dictionary. Retrieved June 8, 2021, from https://dictionary.cambridge.org/dictionary/english/methodology.

References

17

Charmaz, K. (2017). The power of constructivist grounded theory for critical inquiry. Qualitative Inquiry, 23(1), 34–45. Cohen, D. J., & Crabtree, B. F. (2008). Evaluative criteria for qualitative research in health care: Controversies and recommendations. Annals of Family Medicine, 6(4), 331–339. de Vaus, D. A. (2001). Research design in social research. Reprinted. SAGE. Hall, W. A., & Callery, P. (2001). Enhancing the rigor of grounded theory: Incorporating reflexivity and relationality. Qualitative Health Research, 11(2), 257–272. Haynes, K. (2012). Reflexivity in qualitative research. In Qualitative organizational research: Core methods and current challenges (pp. 72–89). Koch, T., & Harrington, A. (1998). Reconceptualizing rigour: The case for reflexivity. Journal of Advanced Nursing., 28(4), 882–890. Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic Inquiry. Sage. Malterud, K. (2001). Qualitative research: Standards, challenges and guidelines. The Lancet, 358, 483–488. Orr, K., & Bennett, M. (2009). Reflexivity in the co-production of academic-practitioner research. Qual Research in Orgs & Mgmt, 4, 85–102. Trochim, W. (2005). Research methods: The concise knowledge base. Atomic Dog Pub. Subramani, S. (2019). Practising reflexivity: Ethics, methodology and theory construction. Methodological Innovations, 12(2). Sue, V., & Ritter, L. (Eds.). (2007). Conducting online surveys. SAGE. Yin, R. K. (1994). Discovering the future of the case study. method in evaluation research. American Journal of Evaluation, 15(3), 283–290. Yin, R. K. (2014). Case study research. Design and methods (5th ed.). SAGE.

2

Understanding Intellectual Contributions

Learning Objectives

When you have finished studying this chapter, you will be able to: • • • • •

define the term intellectual contribution and its key attributes contrast normative with empirical research explain how intellectual contribution can be evaluated understand why research is important to better understand real-world problems explain why we understand theories as a “theory continuum”

2.1 Introduction “If you want to start a physical business, like a café, there are three success components: location, location, location. Similarly, literature suggests that there are three success components for research: contribution, contribution, contribution” (Te’eni et al., 2015, cited in Presthus & Munkvold, 2016). Before elaborating on how to conduct excellent research (i.e., the research process), we first need to address the fundamental question of why we should conduct research at all. Why is it important to conduct research? Why is it part of your degree program? Would “good practice” not be enough? What is the meaning behind all this? The answer to all these questions lies in the existing body of knowledge and in your area of expertise. If we already knew everything about the (present and future) world, we would not need any further research. We may replace the term “the world” with “the economy”, “the industry”, “the company”, “the department” or any other area of interest. Yet, this is not the case. Many cause-effect relationships are not understood well, or even not yet detected at all. And sometimes knowledge that © The Author(s), under exclusive license to Springer Fachmedien Wiesbaden GmbH, part of Springer Nature 2021 S. Hunziker and M. Blankenagel, Research Design in Business and Management, https://doi.org/10.1007/978-3-658-34357-6_2

19

20

2  Understanding Intellectual Contributions

seemed to be certain is not anymore. For example, imagine the relationship between inflation and growth of the GNP. For a long time, it has been consensual knowledge that inflation is a requirement for growth. In times of low-interest rates, this no longer seems to hold true. So, relying on the existing body of knowledge, including so-called best practices, is enticing. Yet, this reliance will not be sufficient for dealing with today's and tomorrow's challenges. Imagine another example: how confident are you whether more diversity on your company’s board of directors would be better for company performance? Also, how can this relationship be proven? Can you convince others about this relationship? Is it a fact or at least consensually validated (Weick, 1989, 1995)? Another mind game: what is the optimal way to enter the French market? Setting up a subsidiary or hiring sales agents or using already established third party sales organizations? Especially designed products for the local market and heavy advertising? And how much is “heavy”? Or, as a last example, is it beneficial to introduce incentives for your middle management in form of variable payments between 20 and 25% of the base salary? You probably have an opinion about some or even all these questions but how sure are you that your opinion is correct? How would you argue and convince somebody else in a discussion about these questions? If you look for new arguments, how would you tackle this? At this stage, research comes into play. The existing body of knowledge has gaps. We do not understand sufficiently how the world works (positivism) and how we construct meaning and make sense from individually perceived phenomena (constructivism). When we use the term “world”, it comprises phenomena that exist independently from the observer and the construction of phenomena. We must decide for our companies or areas of responsibility. That is exactly what management is about. Without research, we could only reuse existing decisions and solutions. We would like to call this the consultant approach. Consultants excel in selling a solution as the only correct one. Yet, is it also the best solution for your specific problem in your specific organization? Especially in times of “fake news” it is sometimes rather hard to rely on others’ opinions about how the world works. Research enables us to add to the existing body of knowledge. This increase is called intellectual contribution. It means understanding a little more about the world and how we make sense out of our perceptions. Keeping this in mind, be aware of the following: 

The distinction between so-called theory and practice does not fit at all. A theory may be defined as “something suggested as a reasonable explanation for facts, a condition, or an event, especially a systematic or scientific explanation” (Cambridge n.d.). It deals with understanding the world (if you follow the constructivism philosophy, these facts, conditions, and events might all be constructed. However, this does not really make up a major difference about the theory). Practice deals with (habitual) action, with changing, dealing with, or living within the (real or constructed) world. Dividing between theory and practice implies that they are independent from each other. Specifically, the frequent use of sentences

2.2 Normative and Empirical Research

21

like “…in theory, but in practice…” seems to confirm this. Yet, they complement and involve each other. The existing body of knowledge is only sufficient if everything in the world works as intended or everything is already known. Thus, research in the sense of adding to the body of knowledge becomes necessary when we face problems in the world (our constructed world) that we cannot solve. The question about how applied research is, is in fact more a question of degree. One extreme may be research that only deals with “understanding (or construction of) the world”. On the other side of the continuum, research may focus on “solving a specific problem”. Often, research is located somewhere between these extremes. This also translates into the different research designs we suggest for business and management. A crucial question remains: how do we gain knowledge? We strive to understand the world by generating a model of the world. Our ontological and epistemological positions affect the relationship between the model and the real world. Yet, independent from these positions, a model is only a representation of the real world. It contains the (allegedly) relevant elements and their interdependencies, and it behaves in the relevant aspects like the object it represents, independent of whether this object is real or constructed. Thus, gaining knowledge means to handle this model. We establish, test, adapt, improve, and apply the model. Any research project focuses on certain aspects of dealing with this model. The research goal describes whether the researcher wants to establish, test, adapt, improve, or apply his or her current model. This aspect of handling the model is the knowledge he or she looks for. Thus, the research question and the research design concentrate on a specific type of knowledge.

2.2 Normative and Empirical Research We characterize business administration as a social science. Intuitively, this seems reasonable as in business the principal actors (until today) are people. Yet, this does not mean that business and management must apply the same approach as social science. Research approaches in business and management are (or should be) comparable to the research principles applied in medical science and other design sciences (Gregor, 2006). This discipline aims at finding solutions for people’s problems as well, with medical science for their health problems. Table 2.1 serves to formulate this comparison a bit more clearly. It shows what challenges management research faces and how medical research deals with them. In many degree programs, business and management research is confused with empirical research. Yet, this is only one side of the coin. Of course, empirical research is well suited for better understanding the world, specifically if we count the construction and sense-making processes also as empirical evidence (of these processes). Yet, business and management also are normative and prescriptive sciences. We look at business and

22

2  Understanding Intellectual Contributions

management science to get advice about what we should do or how we should behave. Only by envisioning, implementing, and testing what we should do, can we solve problems and improve (or not) our organizations and contribute to the body of knowledge. Otherwise, we only reiterate what has been done in the past. Imagine people in the Stone Age making fire by rubbing two sticks together. One has found a prism to focus on the sunlight and start the fire with it. The other has found a lighter left behind by aliens. If quantitative sampling methods were used, these two tools might have been missed, or if they were included in the sample, they would be discarded as outliers. If the survey were conducted on a cloudy day, the one with the prism would even be counted as a failure because he cannot make a fire at all. In business and management, we rarely want to know what all the others do, but what the best in class, or what the best solution for us is. That is the main reason benchmarking is so popular in practice. We commonly believe this realization is rather important because it bridges the already briefly addressed gap between theory and practice. Coming up with a normative theory that this is the best solution, or at least the best or an appropriate solution for your company, is the pinnacle of practical relevance. This view replaces the predominant philosophical discussions about positivism and constructivism (Saunders et al., 1996) and other epistemological and ontological discussions with a very pragmatic approach. Any research that helps us to better understand the world and deal with real or constructed problems and issues in business and management generates an intellectual contribution by adding to the body of knowledge.

2.3 Types of Intellectual Contributions The term theory still lacks a common, agreed upon definition (Lee, 2014). Many graduate students often confuse theory with extant research. Thus, in student’s literature reviews and discussion of empirical results, they use the term ‘theory’ to refer to earlier published research, even if this literature does not relate or contribute to any specific theory (Presthus & Munkvold, 2016). Unless students do deductive theory-testing research, reporting possible theoretical contributions may not be so easy as this may be in different forms: models, frameworks, concepts, propositions, and more. Developing practical recommendations and lessons learned from a well-executed study may not be that challenging. Yet, presenting the theoretical implications of the research process is often more demanding. In the textbooks on research designs and methods, only little advice on how this can be done is available (Presthus & Munkvold, 2016).

2.3.1 Theories and the Body of Knowledge Different views on theory depend to a certain degree on philosophical and disciplinary orientations. Yet, we also see some commonalities. Basically, our body of knowledge

2.3  Types of Intellectual Contributions

23

Table 2.1  Differences between management research and medical research (adapted from Tranfield et al., 2003) Criterion

Management research

Medical research

General research culture

Different perspectives, e.g., positivism and interpretivism

Rigorous scientific evaluation

Research questions

Low consensus

High consensus

Interventions

Experiments possible, depend- Via experiments ing on situation

Aims

Multiples, competing and changing aims

Improving health, reducing illness and death

Inputs to policies

Many, not only scientific evidence

Scientific evidence

Literature reviews

Often narrative, not systematic

Systematic

Literature review protocol

Level of formality and standardization in designing and adopting protocols is commonly low

A plan before the review states the criterion for including and excluding studies, the search strategy, description of the methods used, coding strategies and the statistical procedures. Protocols are made available by international bodies to enhance exchange of knowledge

Selection of studies in literature review

Based on studies that appear relevant. Researcher’s bias is present. Raw data is usually not available. Precise inclusion and exclusion criteria are often not applied, recorded, or monitored

Inclusion and exclusion criteria are expressed in the protocol to ensure a review of the best available evidence. Raw data is usually available for further analysis

Quality assessment of cited literature

Often poor assessment of the Studies are evaluated against fit between research methpredetermined criteria. Internal ods and research questions. validity is explicitly assessed Researchers tend to rely on the ranking of the journal to judge overall quality

Data synthesis from literature review

Usually narrative and qualita- Often, meta-analysis pools the tive. High levels of subjectivity data across studies to increase the power of statistical analysis

Reporting and dissemination

Non-standardized reporting structures. Interpretive long scripts. Explanatory power improved by use of analogy, metaphor, and homology

Standardized reporting structures and protocols. Nonexplanatory style adopted

(continued)

24

2  Understanding Intellectual Contributions

Table 2.1   (continued) Criterion

Management research

Medical research

Practical relevance

Implementation of evidence is usually an afterthought (depending on the research design)

Collaborative process and practice-oriented

Research methods

Usually, quantitative. Methods are based upon a hierarchy of evidence

Both quantitative and qualitative. Subject to interpretation depending on research method. Often, triangulation of methods recommended

comprises theories. In everyday use, the term theory usually means an untested claim, or a guess without supporting evidence. Yet, for scientists, a theory has a different meaning. A theory is a well-substantiated explanation of an aspect of the world that can encompass hypotheses, facts, and laws. Let us look at the Cambridge definition first:  A theory is a formal statement of the rules on which a subject of study is based or of ideas that are suggested to explain a fact or event or, more generally, an opinion or explanation (Cambridge n.d.). As a starting point, we stick with this definition, but also consider less formally stated, implicit theories. We base our thought primarily on the summary of the ongoing debate about theories by Ridder, 2017. He states that a theory comprises three basic elements, drawing on Whetten, 1989; Kaplan, 1998; Suddaby, 2010; and Alvesson, 2004. • elements or components (concepts and constructs) used to identify the necessary elements of the phenomenon under investigation. • relationships between elements (concepts and constructs), explaining how and why the relationship exists and works. • conditions or boundaries (temporal and contextual), limiting the applicability and generalizability of the theory. Overview

The following useful quotes help to better characterize the term theory: “It is a collection of assertions, both verbal and symbolic, that identifies what variables are important for what reasons, specifies how they are interrelated and why, and identifies the conditions under which they should be related or not related” (Campbell, 1990, p. 65).

2.3  Types of Intellectual Contributions

“… a system of constructs and variables in which the constructs are related to each other by propositions and the variables are related to each other by hypotheses” (Bacharach, 1989, p. 498). “A theory is a set of interrelated constructs (concepts), definitions, and propositions that presents a systematic view of phenomena by specifying relations and variables, with the purpose of explaining and predicting the phenomena” (Kerlinger, 1986, p. 7). “Theory is about the connections among phenomena, a story about why acts, events, structure, and thoughts occur. Theory emphasizes the nature of causal relationships, identifying what comes first as well as the timing of such events” (Sutton & Staw, 1995, p. 378). “… theory is a statement of concepts and their interrelationships that shows how and/ or why a phenomenon occurs” (Corley & Gioia, 2011, p. 12).

As you see, theories can be defined differently. Broadly, we can further characterize theories on the “theory continuum” with the following traits (Gregor, 2006, pp. 616): • generalization. Theories can be categorized by their level of generalization. Meta-theory is at a top level of abstraction and provides a way of thinking about other theories, possibly across disciplines. Theories with sweeping generalizations that are relatively unbound in space and time are referred to as grand theories (Bacharach, 1989). Definition of the level of generality or scope of a theory covers specifying the boundaries within which it is expected to hold and providing the qualifying words that are used in theoretical statements (for example, words like always, some, every, all). • causality. The purpose of causality (i.e., the relation between cause and event) is crucial to many theory conceptions. We can think of many ways of reasoning about causality. To a certain degree, different theories reflect different ways of ascribing causality in the phenomena we observe around us and the different explanations that arise. Yet, an important point is that many arguments for causality are not mutually exclusive and at different times and in different circumstances we will rely on different reasons for ascribing causality. • explanation and prediction. Crucial for many theories are both goals of explanation and prediction. For example, Nagel, 1979 sees the distinctive aim of research in theories that offer systematic and responsibly supported explanations. Explanation is closely referred to human understanding. Any explanation may be provided intending to induce a subjective state of understanding. Some theories focus on one goal, either explanation or prediction, at the expense of the other. Thus, it is possible to achieve precise predictions without understanding the reasons outcomes occur (this equals a black box explanation).

25

26

2  Understanding Intellectual Contributions

We use the terms “constructs” and “concepts” interchangeably, being aware that some authors use them with different meanings, for example Gioia et al., 2013: by ‘concept,’ we mean a more general, less well-specified notion capturing qualities that describe or explain a phenomenon of theoretical interest. In our way of thinking, concepts are precursors to constructs in making sense of organizational worlds—whether as practitioners living in those worlds, researchers trying to investigate them, or theorists working to model them (p. 16). Following Weick, 1995, we understand theories as a continuum. They are not a product that reaches “market readiness”. Thus, we consider every perception of elements in relationships under conditions a theory. This is in line with our pragmatic approach that every perception and every action is based on an implicit or explicit theory. Theories differ from each other in degrees of how clearly elements, relationships and conditions are described and how well they are corresponding to the (real or constructed) world. Thus, distinguishing between “constructs” and “concepts” on this level at this point would integrate such a degree into the definition without adding additional qualities. In the process of your research, you might integrate such differentiation to enhance understandability. The discussion of different perspectives on theory at a general level suggests theories as abstract things that help to describe, explain, and enhance the understanding of the real world and, sometimes, to provide predictions of what will happen in the future. Also, theories may give researchers a basis for intervention and action (Gregor, 2006). To sum up, we understand theories as any combination of elements and their relationships under certain conditions. We see no fixed delineation under what conditions such combinations become “true” theories. In fact, we understand theories as a continuum with the key characteristics of “well described” and “corresponds to the (real) world”.

2.3.2 Adding to the Body of Knowledge Theory creation, elaboration, and testing As our (individual and social) body of knowledge consists of theories, we add to the body of knowledge by improving or refining the underlying theories. Generally, we may improve the theories in three different ways, generating three different types of intellectual contributions: • theory creation. The aim is to develop a new theory. Theory creation or development is a well-established phase of the research process, while theory elaboration has received much less attention in the academic community (Keating 1995). The research process of theory development is mainly geared towards formulating

2.3  Types of Intellectual Contributions

27

theoretical constructs (elements), their relationships and conditions. This can be done, for example – by searching for and categorizing of elements, relationships, and conditions as impartially as possible (e.g., grounded theory), – by intuitive, sense-making leaps (Ridder, 2017), and – by establishing a new theory based on the falsification of an existing one or by developing alternative explanations. • theory elaboration. This stage represents the middle ground between theory creation and theory confirmation or falsification. Compared to theory creation, theory refinement or elaboration research entails a more definitive theoretical background and more focused research objectives. Yet, as with theory creation, researchers who conduct theory elaboration research need to be open to the discovery of phenomena, constructs, relationships, and conditions that complement, change, or replace the theory (Keating, 1995). We can elaborate theory by – describing phenomena, elements, relationships, or conditions more clearly, precisely, and comprehensively, – categorizing, grouping, and differentiating elements, relationships, and conditions, – adding, changing, replacing elements, relationships, and conditions, and – refining elements, relationships, and conditions. • theory confirmation. Whenever theory meets reality, we may confirm or falsify a theory. This stage represents the beginning of a new research cycle or the slightly more confident sticking with the existing theory. It is important to realize that theory confirmation can ever be only preliminary, as theories can ultimately never be proven. Hence, being unable to falsify a theory counts as confirmation in the sense of (slightly) increasing the probability that the theory is correct. Theory confirmation or falsification can be achieved by: – hypothesis testing. Here the constructs and relationships are existing in a welldeveloped and mature state. Such theories may be reformulated as hypothesis and empirically tested, often with quantitative methods. This is the most popular approach to test theories. Yet, researchers criticized that theory testing has become the major focus of scientists today even if theories have not yet reached a mature state (Delbridge & Fiss, 2013; Ridder, 2017). – applying the theory and evaluating if the observed phenomena are explained. The level of specification here is more comprehensive than in the above hypothesis testing, but the corresponding phenomena might only be looked for in fewer units of analysis. – creating and implementing solutions based on the theory and assessing the outcome and phenomena. Whereas in the application of the theory the observed phenomena are explained by the theory, here solutions are designed and/or implemented to confirm that the theory suffices for developing a solution and that this solution affects phenomena that are in line with the theory.

28

2  Understanding Intellectual Contributions

We argue that theory creation hardly takes place as researchers (human beings) cannot collect data with no kind of preconception. Remember that this also represents a theory in our terminology. If previous research is substantial, we can, for example, identify four ways to contribute to an existing theory (regarding theory elaboration and theory confirmation above). Drawing on Eisenhardt & Graebner, 2007 and Yin, 2014, we elaborate on four major ways to build theory: confirmation and replication, extension, contradiction, and elimination. We present examples in Table 2.2 by looking at Lewin’s classical theory of change management: unfreeze-change-refreeze (Lewin, 1958, cited in Levasseur, 2001 and Presthus & Munkvold, 2016). Let us consider an example theory where we assume a relationship (cause-effect chain) between three constructs in the following way: 1 ▷ 2 ▷ 3. Example

Here are some different views on how we can understand theoretical contributions. We borrowed them from the area of information systems (Gregor, 2006, p. 613). Yet, they also apply to business and management: • theory as statements that say how something should be done in practice: an early textbook by Davis and Olson (1985) articulates how a Management Information System (MIS) should be designed, implemented, and managed. This theory provides prescriptions to be followed in practice, with the implicit expectation that the prescribed methods will in some sense be “better” than alternatives (Cushing, 1990). • theory as statements providing a lens for viewing or explaining the world: Orlikowski and Robey (1991) drew on structuration theory and empirical work to construct a theory in which the organizational consequences of IT are viewed as the products of both material and social dimensions. We may see such a theory as a desirable product. The formal testing of such a theory is not envisaged (Walsham 1995). • theory as statements of relationships among constructs that can be tested: the technology acceptance model (Davis 1986) posits that, e.g., perceived usefulness and perceived ease-of-use, are of primary relevance for computer acceptance behaviors. This theory leads to testable propositions that can be investigated empirically (Davis et al., 1989). ◄ Also, we may argue that incorporating rejected theories into a new or modified theory could be understood as theory elaboration. Often, in research you do not focus only on one type of intellectual contribution. Defining what your intellectual contribution should or will be is a very crucial step in planning the overall research design (see also Sect. 3.2).

2.3  Types of Intellectual Contributions

29

Table 2.2  Types of contributions to an existing theory (adapted from Presthus & Munkvold, 2016) with amendments by the authors Type

Description and examples

Confirmation and replication (theory confirmation in our terms)

Indicates that a chosen theory is still valid, or that it will work in different setting 1⊲2⊲3

Extension (theory elaboration)

Adds to an existing theory, for example with an extra construct 1⊲2⊲3◮4

Contradiction (theory confirmation and falsification plus theory elaboration)

Contradicts the whole, or parts of the theory, such as providing evidence of more interplay between the constructs 1 ◭⊲ 2 ◭⊲ 3

Elimination (theory confirmation and falsification)

Indicates that parts of the theory are obsolete in the chosen setting 1 ⊲ 2 –⊲ 3

Different research philosophies usually prefer different intellectual contributions (for example, the conditions or context in qualitative studies), but neglect others (for example, cause-effect relationships). Eventually, this hinders the scientific progress as all types of contributions are necessary to further advance the body of knowledge as they are–or at least should be–stimulating each other. A taxonomy of theory types Let us start with the primary goals of the theory. Gregor, 2006 suggests four primary goals of theory that we describe as follows: • analysis and description. The theory describes the phenomena of interest (in our terms elements), analysis of relationships among those phenomena and the constructs used to describe them (in our terms relationships between elements). Also, it addresses generalizability of elements and relationships, and the context or boundaries within which relationships between elements hold (in our terms conditions). • explanation. The theory explains how, why, and when things happen, relying on different views of causality and methods for argumentation. This explanation is usually used to further understanding or insights by others into the phenomena of interest. • prediction. The theory states what happens in the future if specific preconditions are met. The degree of certainty in the prediction is expected to be only approximate or probabilistic. It is important to realize that predictions do not require explanations or the ensuing understanding but are also workable if the cause effect relation is a black box.

30

2  Understanding Intellectual Contributions

• prescription. A special case of prediction exists where the theory provides a description of the method or structure or both for the construction or design of an artifact (akin to a recipe). Putting the recipe to use will cause an artifact of a certain type and with certain properties to come into being (Gregor, 2006, p. 619). Gregor (2006) discusses “what constitutes theory […] and what form contributions to knowledge can take.” (p. 620) and distinguishes five interrelated types of theory in line with the described four goals (as one meets the combination of two goals): • theory for analyzing state what is. The theory provides description and categorization. • theory for explaining state what is why, when, where, and how. The theory provides explanations. • theory for predicting state what will be. The theory provides predictions and has testable propositions. • theory for explaining and predicting state what is why, when, where, how, and what will be. The theory provides predictions and has both testable propositions and causal explanations. • theory for design and action states how to do something. The theory gives explicit prescriptions for constructing an artifact. This normative approach requires specific research designs. These designs are frequently referred to as design science or design science research (see Chap. 6). Strictly, they also encompass action research (see Chap. 7), an often-neglected research design in business and management. Gregor, 2006 states that theories can support junior researchers as follows: “Novice researchers should benefit from the depiction of the basic components of theory, helping with their question of “What is theory?” The approach recommended for theory development is to begin with the research problem and research questions and then determine which type of theory is appropriate for the problem, given the current state of knowledge in the area and using the classes depicted here as a guide” (Gregor, 2006, p. 634). We would agree that thinking in these terms helps to define the intellectual contribution and type of conclusion to be drawn in your research. While established as a foundational resource in our discipline (approaching 2000 Google Scholar citations), there is no universal agreement on Gregor’s taxonomy (Presthus & Munkvold, 2016; Weber, 2012).

2.3  Types of Intellectual Contributions

31

Concrete examples of creating intellectual contributions To give a more application-oriented categorization of intellectual contribution you might want to start from the definition of theory: elements having relationships under conditions, as depicted in Fig. 2.1: Based on this generic definition of a theory and the three types of intellectual contributions, they can, for example, make these intellectual contributions by: • finding new or additional elements, relationships, • characterizing, measuring, and categorizing the elements, conditions, and relationships (the latter for example could be positively or negatively directed, bidirectional or causal), • confirming and eliminating elements, conditions, and relationships, • confirming cause/effect relationships, • finding gaps in the existing body of knowledge, and • confirming the sufficiency of the existing body of knowledge for normative prescriptions. In the following, we exemplify some of these possibilities further. We might aim intellectual contributions at confirming or falsifying relationships as already depicted in Fig. 2.1. We specify the elements as constructs to stress that the elements might be construed in the sense of existing in the observer’s perception or are not directly observable concepts (that we also label constructs). As already mentioned, one type of intellectual contribution is to identify (additional) constructs that have a relationship with other constructs. These relationships might be directed or even causal. That is a major step towards establishing theory, as this allows us to cope with complexity. We are not limited to examine a 1:1 relationship but can also research n:1 or even m:n relationships. It is important to differentiate between the type of intellectual contribution, the research design, and the methods used. You might find similar illustrations in statistics books. But we strictly focus on the intellectual contribution

Fig. 2.1   Elements having relationships under conditions

element X

relationship

condition

element Y

32

2  Understanding Intellectual Contributions

(i.e., identifying constructs and relationships) without implying designs or methods. Fig. 2.2 symbolizes the identification of relationships between a combination of constructs. We cannot define a clear cut between elements and conditions. Regarding theory, the possibility that conditions affect the (form) of relationships between elements seems to be important. However, what counts as an element and what as a condition? Often, we consider conditions to be elements that do not change for the sake of convenience (socalled “context”). The context then is often irrelevant for the research, so that conditions are also treated as constructs. Hence, we do not show the conditions separately in the following illustrations but subsume them in the depicted constructs. Excluding constructs from a specific theory as they have no relationship is also an intellectual contribution. This reduces the complexity and increases our ability to use the theory. Thus, we consider not only finding constructs and relationships an intellectual contribution, but also excluding them if they have no relationship, as we show in Fig. 2.3. As we have discussed the types of intellectual contributions and depicted some examples, we can proceed with the value of intellectual contributions in the next chapter.

Fig. 2.2   Combining constructs and their relationships to other constructs construct a

construct b

construct x

construct n

condition

2.4  Value of Intellectual Contributions Fig. 2.3   Excluding constructs with no relationship

33

construct i

construct a

construct b

construct x

construct n

construct j

2.4 Value of Intellectual Contributions The value of any information equals the amount of uncertainty removed. Removing (some) uncertainty contributes to the body of knowledge. Thus, the value of research about a specific research question that removes a bit of uncertainty is higher than the value of research that removes none. What does this mean regarding research and writing research papers and dissertations? It is preferable to take small but methodologically very sound steps than to make “massive” breakthroughs that do not accomplish to eliminate uncertainty (because of the poor execution). If you strive for A publication in a high-impact academic journal, in most cases, you need to achieve both. Yet, it is perfectly fine to start small, for example, replicating an established methodology for a different sample. Tackling a more challenging problem with a subpar method (a method that is not the best suited to produce results that answer the question, but still provides valuable information) is also acceptable to start with. This also means that there are no “bad” or “disappointing” results. If our goal is to understand the world and give advice on how to act in it, then even a result that does, e.g., not confirm our initial assumptions or does not confirm significant correlations adds information. We may confirm or falsify a theory. Maybe we have refined an existing theory based on the results or even created an alternative theory to explain the observed

2  Understanding Intellectual Contributions

34

data. This research may even add the most valuable information. For example, not being able to confirm existing knowledge or a theory can also mean that we do not yet sufficiently understand how the world works. As we have now established the basic concepts of knowledge stock, theories, and intellectual contributions, we can proceed with the set-up of the research process. Key Aspects to Remember

Get familiar with term theory and its nuances A theory is a formal or implicit statement of the rules on which a subject of study is based or of ideas that are suggested to explain a fact or event or an opinion or explanation. We understand theories as any combination of elements and their relationships under certain conditions. Also, theories can further be characterized on the theory continuum. Explain how researchers can create intellectual contributions As the body of knowledge comprises theories, we add to the body of knowledge by improving or refining the underlying theories. We can differentiate between theory development, theory elaboration, and theory testing. An intellectual contribution is any improvement of a theory, whether this is creating, elaborating, or confirming a theory. Research enables us to add to the existing body of knowledge. This increase is called intellectual contribution. It means understanding a bit more about the real world and how we make sense out of our perceptions. Choosing a proper research design is a prerequisite to create intellectual contributions. Recognize the value of intellectual contributions The value of any information gained by research equals the amount of uncertainty removed. Thus, removing uncertainty contributes to the body of knowledge. The value of research that removes a tiny amount of uncertainty might be still relevant. It is better to take small but methodically solid research steps than to make breakthroughs that fail to eliminate uncertainty (due to poor execution). Also, researchers often believe it is a poor result if they do not find any significant correlations or if their theoretical assumptions are not empirically confirmed. Yet, if our goal is to understand the world and give advice on how to act in it, then even a “bad” result contributes to the body of knowledge. Differentiate between empirical and normative research Empirical research tries to improve theories by finding empirical evidence to create, elaborate and confirm theories. In contrast, normative research adds to empirical research by not only researching existing or socially constructed phenomena but also what phenomena should exist and how their existence can be achieved. Normative research introduces preferences, prescriptions, and solutions to the research process. As empirical research can only find what is there (or constructed) (new) solutions can

References

35

only be empirically researched once they exist, so only after their invention and realization. Normative research is focused on finding and prescribing solutions for problems. These prescriptions are based on the existing body of knowledge and need to be confirmed, like any other theory. Thus, normative, and empirical research are not mutually exclusive but benefit from each other. Critical Thinking Questions

1. What are the main differences between empirical and normative research? 2. Why is it crucial to differentiate between theory elaboration and confirmation? 3. What role does constructivism and positivism play in the research process? 4. What is the main purpose of a theory? 5. How can researchers add to the body of knowledge? 6. Why is it considered difficult to establish a universally accepted body of knowledge?

Recommendations for further Readings

de Vaus, D. A. (2001). Research design in social research. Reprinted. Los Angeles: SAGE Publications, Inc. Godfrey-Smith, P. (2009). Theory and Reality (Science and Its Conceptual Foundations series). University of Chicago Press. Kindle-Version. Gregor, S. (2006). The Nature of Theory in Information Systems. In Management Information Systems Quarterly 30 (3). Saunders, M., Lewis, P., & Thornhill, A. (2019). Research Methods for Business Students. Pearson.

References Alvesson, M. (2004). Reflexive methodology: New vistas for qualitative research. Bacharach, S. B. (1989). Organizational theories: Some criteria for evaluation. AMR, 14(4), 496–515. Cambridge University Press. (n.d.). Theory. In Cambridge dictionary. Retrieved May 19, 2021, from https://dictionary.cambridge.org/dictionary/english/theory. Campbell, J. P. (1990). The role of theory in industrial and organizational psychology. In M. D. Dunnette, L. M. Hough, & H. C. Triandis (Eds.), Handbook of industrial and organizational psychology (2nd ed., pp. 39–73). Consulting Psychologists Press. Corley, K. G., & Gioia, D. A. (2011). Building theory about theory building: What constitutes a theoretical contribution? Academy of Management Review, 36(1), 12–32. Cushing, B. E. (1990). Frameworks, paradigms and scientific research in management information systems. Journal of Information Systems, (4)2, 38–59. Davis, G. B., & Olson, M. H. (1985). Management information systems: Conceptual foundations, structure and development (2nd ed.). McGraw-Hill.

36

2  Understanding Intellectual Contributions

Davis, F. D., Bagozzi, R., & Warshaw, P. (1989). User Acceptance of Computer Technology: A Comparison of Two Theoretical Models. Management Science, 35(8), 982–1003. Delbridge, R., & Fiss, P. C. (2013). Editors’ comments: Styles of theorizing and the social organization of knowledge. Academy of Management Review, 38, 325–331. Eisenhardt, K. M., & Graebner, M. E. (2007). Theory building from cases: Opportunities and challenges. Academy of Management Journal, 50(1), 25–32. Elliot, S., & Avison, D. (2005). Discipline of informations systems. In D. Avison & J. PriesHeje (Eds.), Research in IS: A handbook for research supervisors and their students. Elsevier Science. Gioia, D. A., Corley, K. G., & Hamilton, A. L. (2013). Seeking qualitative rigor in inductive research. Organizational Research Methods, 16(1), 15–31. Gregor, S. (2006). The nature of theory in information systems. Management Information Systems Quarterly, 30(3). Kaplan, A. (1998). The conduct of inquiry. Methodology for behavioral science. In A. Kaplan (Eds.), The conduct of inquiry. Methodology for behavioral science. Transaction Publishers. Keating, P. J. (1995). A framework for classifying and evaluating the theoretical contributions of case research in management accounting. Journal of Management Accounting Research 7 Fall, 66–87. Kerlinger, F. P. (1986). Foundations of behavioral research. Holt, Rinehart and Winston. Lee, A. S. (2014). Theory is king? But first, what is theory? Journal of Information Technology, 29(4), 350–352. Levasseur, R. E. (2001). People skills: Change Management tools—Lewin's change model. Interfaces; Jul/Aug 2001; ABI/INFORM Research, 31(4), 71–73. Nagel, E. (1979). The structure of science. Hackett Publishing Co. Orlikowski, W. J., & Baroudi, J. J. (1991). Studying information technology in organizations: Research approaches and assumptions. Information Systems Research, 2(1), 1–28. Presthus, W., & Munkvold, B. E. (2016). How to frame your contribution to knowledge? A guide for junior researchers in information systems. NOKOBIT, 24(1). Ridder, H.-G. (2017). The theory contribution of case study research designs. Business Research, 10(2), 281–305. Saunders, M., Lewis, P., & Thornhill, A. (1996). Research methods for business students. https:// www.semanticscholar.org/paper/bef028d7d7fcb4705c24451304b089f4912920d4. Suddaby, R. (2010). Editor’s comment: Construct clarities in theories of management and organizations. The Academy of Management Review, 35(3), 346–357. Sutton, R. I., & Staw, B. M. (1995). What theory is not. Administrative Science Quarterly, 40(3), 371. Te’eni, D., Rowe, R., Ågerfalk, P. J., & Lee, J. S. (2015). Publishing and getting published in EJIS: Marshaling contributions for a diversity of genres. European Journal of Information Systems, 24, 559–568. Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14(3), 207–222. Weber, R. (2012). Evaluating and developing theories in the information systems discipline. Journal of the Association for Information Systems, 13(1), 2–30. Weick, K. E. (1989). Theory construction as disciplined imagination. Academy of Management Review, 14(4), 516–531. Weick, K. E. (1995). What theory is not, theorizing is. Administrative Science Quarterly, 40(3), 385. Whetten, D. A. (1989). What constitutes a theoretical contribution? The Academy of Management Review, 14(4), 490. Yin, R. K. (2014). Case study research. Design and methods (5th ed.). SAGE.

3

Setting-Up the Research Process

Learning Objectives

When you have finished studying this chapter, you will be able to: • understand why it is useful to start with the conclusion, not with the research question • produce results for good arguments • develop a preliminary research question • assess the consistency of your research design • set-up the distinct steps for your next research project

3.1 Introduction to the Research Process Scientific research is as a formal, rational, and systematic process that offers answers to the study of a phenomenon applying scientific procedures. Or different, we can define research as a systematic investigation that aims to develop theories, establish evidence, and solve problems. To achieve such a goal, there is a path from identifying a problem to present reliable results (Dresch et al., 2015). We often observe that our students present their work describing what they do or plan to do about data collection and data analysis. However, they often struggle to state concrete research questions, hypotheses, and propositions behind their work and the (type of) conclusion they like to draw. Students often phrase research questions too vaguely and too broadly to be answered within one research project (Blessing & Chakrabarti, 2009). Researchers rarely start with sitting down and stating a research question. Usually, they have an area or topic that they think might be worthwhile to do research in. These areas or topics might stem from: © The Author(s), under exclusive license to Springer Fachmedien Wiesbaden GmbH, part of Springer Nature 2021 S. Hunziker and M. Blankenagel, Research Design in Business and Management, https://doi.org/10.1007/978-3-658-34357-6_3

37

38

• • • • • • •

3  Setting-Up the Research Process

current research papers (and their suggested further research areas) interests of professors (research interests, publications), lectures, seminars, or exercises, personal questions (own interests), which can be addressed within a dissertation, existing dissertations, especially doctoral dissertations, problems, issues, and questions within an organization, and topic pool lists that might be available for research courses.

We summarize the entire research process below. It may surprise you that the process does not start with the development of a research question (as usually suggested in lectures and textbooks on research). The following is one of the key lessons from many research projects that the authors of this textbook have been involved in. 

Never start with the research question. Think first about what (type of) conclusions you would like to draw from your research. What kind of statement would you like to present in the conclusion section at the end of your research paper? For example, such a type of conclusion could be: “in companies with a very customer oriented corporate culture, a cash-based incentive system does or does not have a large impact on personnel’s actions”. This fictitious answer equals your research goal. So, we put the cart before the horse. We first think about what arguments we need to make these desired conclusions. What results do we need to arrive at the desired conclusions? What data and methods are relevant and capable to get to our conclusions?

Having made up our mind about these crucial questions, we can now give thoughts to what research question we answer with our research project. After checking for consistency with our goal and overall project feasibility, we proceed with the execution of our research project. Maybe, we must adjust some decisions taken in earlier steps. In Fig. 3.1, we illustrate a summary of the required steps to plan and execute a research project. In the following, we describe each step. Please note that these steps do not represent a linear path but might require loops and iterations.

3.2 Step 1: Start with the Conclusion you Would Like to Draw Opposed to many recommendations, start at the very end of the research process. Envision the conclusion of your research report. What conclusion would you like to draw from your research? Or similarly, what kind of statement would you like to make? If you are not yet sure what we mean by a conclusion, glance at Sect. 4.4.8 to get a first impression. The answer to these questions leads to your research aim(s). The more detailed you

3.2  Step 1: Start with the Conclusion you Would Like to Draw

Conclusion

Argument

Results

Method

Data

39

•What type of conclusion would we like to draw? •This is our research aim.

•What are good arguments that enable us to draw this conclusion? •What is / how to achieve the desired conclusion validity?

•What are the results we need to achieve to make these arguments? •Internal validity, external validity

•What data analysis method will generate this result? •What are the preconditions that we need to meet?

•What data do we need to use the method? •What constructs do we use? •How do we operationalize and transform our data into constructs? •How and from which source do we get the data?

•What question does our research really answer? Question

Fig. 3.1  Steps in the research process

imagine this (fictitious) statement, the more clues you get about selecting your research design. This does not imply that your research is already biased. Although our conclusion may be affirmative (e.g., A causes B) or negative (e.g., A does not cause B) or both, depending on the context, the specific type of conclusion you will make after having conducted your research is still completely open. If researchers were biased (that is, assuming a particular directed conclusion before conducting research), their research would not produce an intellectual contribution. This is crucial to understand because the type of the conclusion or statement affects or even determines the research design. Example

For example, different hypothetical conclusions you would like to draw could be: • women on the board of directors do (or do not) affect corporate performance. • marketing concept A (or B or C) promises the biggest success for company XY to enter the French market.

40

3  Setting-Up the Research Process

• to make a detailed cost calculation for customer offers that integrates effects of product complexity, we propose an activity-based calculation scheme that apart from direct cost also includes activity cost rates and cost driver for the following activities a, b, c, etc. which results in an average difference to the standard calculation of the product range currently used of X% per product. (or there is no difference in the calculated cost that justifies activity-based costing). ◄ The more specific your fictitious conclusion is, the better and stricter the resulting guidance for your research design. If you look at the first example conclusion in the box above, you recognize the following alternative fictitious conclusions: • women on the board of directors increase the decision quality of the board, • there should be more women on the board of directors, and • the more women on the board of directors the higher the corporate performance. These conclusions look all rather similar. However, they are not and would require different research designs. If you prepare a statement like “I plan to do research on asset turnover (or employee motivation, brand awareness, etc.)”, this is not specific enough to develop a research design. Thus, you cannot write a satisfactory research report. After this crucial initial step, you must think about the kind of intellectual contribution your research project aims to generate. The envisioned contribution to the body of knowledge guides the refinement of your research’s fictitious conclusion (for the different types of intellectual contributions, see Chap. 2). Luckily, there is more than one way to state a hypothetical conclusion more precisely. For example, you can brainstorm on your own or in a group, or you can start with the literature review to find research gaps that could be addressed with your research. This first step seems completely contrary to all research projects you may have been engaged in. Usually, most researchers in undergraduate and graduate programs are asked to start with the research question(s). Strictly, this is still true, as (fictitious) conclusions and research question are two sides of the same coin. If you can phrase your research question precisely and comprehensively, you can do the same with the fictitious conclusion. For example, the broad fictitious conclusion would be: “the proportion of women on the board has a significant or non-significant positive or negative correlation or no correlation at all with firm performance. This is measured by return on equity during the years when women are on the board compared to companies without women.” Obviously, the fictitious conclusion should not be “in our research we show that women on the board have a positive impact on firm performance”. With the first formulation of the hypothetical conclusion, the research is still neutral in the sense that it is not designed to find evidence for a specific conclusion.

3.2  Step 1: Start with the Conclusion you Would Like to Draw

41

The problem is that both students and researchers have immense problems phrasing equally specific research questions at the very beginning of the research project, as shown in Sect. 3.7. Thinking in terms of arguments, answers, or conclusions eases this problem to a certain extent. Yet, this issue is often not entirely solved on the first pass, as the fictitious conclusion in the example above shows. Or could you have come up with such a hypothetical conclusion right at the beginning of your research? However, the improvement process gets easier if you think about the hypothetical conclusion as a statement instead in the form of a research question. The research question, once we can formulate it precisely and comprehensively, is still a very important (later) part of the research design (see Sect. 3.7). An important quality criterion for your research needs to get introduced here as it has a huge impact on the choice of your research design. This quality criterion is called conclusion validity. Conclusion validity may be referred to as the degree to which conclusions researchers reach about relationships in their data are reasonable (Trochim, 2005). This definition focuses on the relationships in the data and thus covers only some types of research designs. We like to define conclusion validity a bit more broadly and practically.  Conclusion validity is the degree to which the conclusions drawn in a study are logical and sound. This means that we should • not draw unwarranted conclusions, and • not miss conclusions that we could have drawn. For example, we may find relations in theories (theories comprise relations between elements under certain conditions) that do not exist. However, we deliberately extend the meaningfulness to all our conclusions and do not limit it to (statistically significant) relationships. As such, conclusion validity is affected by many factors that can be grouped into two categories: • research design related. Is the research design able to generate answers and arguments that let you draw this conclusion (using sound logic)? • research implementation related. Could you conduct your research according to your research design? Where did you deviate (intentionally or unintentionally) from your research design and which consequences did this have on your conclusions and their validity?

42

3  Setting-Up the Research Process

In the first step of the research process, we are primarily concerned with the research design related issues, as these issues guide you to the next step. At this point, conclusion validity and its related concepts internal and external validity are most relevant. If you like to draw conclusions that are valid for a whole population meeting certain criteria (e.g., all companies in industry X with a size above Y) your research design needs to differ from the one for conclusions that refers to some instances in the population (e.g., company A, B and C). These considerations are referred to as external validity issues.  External validity refers to the extent to which conclusions from your research can be applied (generalized) to a whole population (limited by the conditions) (Streefkerk, 2021).

3.3 Step 2: Defend your Conclusion Once you have developed a sufficiently detailed conclusion, the next step is to think about how you can defend this conclusion. So, think about what arguments you could use to make such a statement and what counterarguments you will encounter. The better the arguments you can make and the weaker the counterarguments that are still valid, the stronger your statement will be. Since you do not yet know the direction of your conclusions, this seems far-fetched, but for the moment it suffices to collect pro and con arguments without already taking a side. If you lack arguments, you can again use the brainstorming technique or dig deeper into the existing literature. The more specific your fictitious conclusion is, the more arguments a literature review usually provides (do not reinvent the wheel). However, it then is less likely that you come across original and new arguments. Ultimately, this is an exercise in logic. You create (consciously or unconsciously) a model that explains your fictitious conclusion. In this model, you collect everything you know or suspect about conditions, elements, and relationships in your field of interest. If you realize that you know barely anything about a topic, this is still an important step in developing a research design. After this, you need to create a “wish list”: what arguments do you want to make to direct the discussion in your favor? This list should include facts, empirical evidence you want to refer to in your argument, and the exclusion of alternative explanations. Particularly good arguments are those that are internally valid.  Internal validity refers to the degree of confidence that the causal relationships used in your argument are trustworthy and real. Further, they should not be influenced by other factors or variables. For example, looking at the examples of fictitious conclusions above, you might develop an argument, that:

3.4  Step 3: Produce Results for Good Arguments

43

• there is evidence that the companies with women on the board of directors have a higher or the same or lower corporate performance than the companies without, • there is evidence that women on the board of directors increase corporate performance, • based on the commonly accepted market penetration rates, sales prices, and motivation of involved personnel to execute, marketing concept A or B or C generate the highest return on investment of the market entry in France after five years, and • etc. As a next step, you need to check whether these arguments are good and decisive. Hardly any argument is perfect, but the better ones are difficult to argue against (counterarguments are alternative explanations, see Sect. 2.3.2). E.g., the argument with women on the board is only a good argument if the companies are comparable, so that differences in corporate performance can really be linked to women on the board (assuming causality is difficult and often wrong). Also, the term corporate performance needs to be defined meaningfully. If you were to define corporate performance as “compliance with gender equality principles”, the “evidence” provided would be worthless as an argument because it only repeats self-evident facts without contributing additional facts to the discussion. Regarding the example of the marketing concept, the argument given is only a good argument if return on investment is a goal of market entry. Maybe the goal is market share or something entirely else or a combination of several goals. The same applies to the other criteria, namely the motivation for implementation and that the market penetration rate is really “generally accepted”. In terms of the marketing concept example, the above argument is only a good argument if return on investment is a goal of market entry. Perhaps the goal is market share or something else entirely or a combination of these. The same applies to the other criteria, namely the motivation for implementation and that the market penetration rate is really “generally accepted”. This check and the refinements of the arguments lead to the next step.

3.4 Step 3: Produce Results for Good Arguments At this point, as you know the arguments that you like to present in your discussion, you can derive the facts or results (i.e., outcomes) to make those arguments. So, what would be an outcome that would allow the statement “there is evidence that companies with women on the board have higher or equal or lower company performance than companies without women” to be used as an argument? If you know all arguments you would like to make in your discussion, you can derive the facts or results that enable you to make these arguments. Obviously, the results that make good arguments for any kind

44

3  Setting-Up the Research Process

of discussion to achieve any kind of conclusion vary and depend on the argument. This makes it hard to talk about them in a general way but also strongly hints at potential results. There are no such things as inherently bad or good results, only results contributing validly to a discussion or not. The value of information equals the amount of uncertainty reduced (Shannon, 1948). This corresponds with our statement above: a valuable contribution to a discussion reduces the amount of uncertainty. This will be our guideline for defining valuable results. Hence, what would be a result, that would allow as to use “there is evidence, that the companies with women on the board of directors have a higher or the same or lower corporate performance than the companies without” as an argument? • We would need a result that is generalizable. Just to look at one company that has women on its board of directors and that is performing exceedingly well would not be acceptable evidence (for this argument). There would always be doubt (i.e., the potential counterargument or alternative explanation) that the company would also perform exceedingly well without women on the board of directors. • So, we need a sample of companies that we can compare and then deduce, that the results also hold true to all companies. This means we will not only need some companies to analyze, but quite a lot, to use inferential statistical methods that yield generalizable results. • We need a proxy to measure business performance. This proxy should be widely accepted or at least widely used to avoid the counterargument that our proxy lacks validity. Also, we need to think about the construct validity (see Sect. 1.4) of our proxy: if we were to use Tobin's q (market capitalization divided by book value of assets), we would at least partially include in our measurement how the stock market or investors would value women on the board. This is certainly an argument as well, but the line of reasoning would not be: women on the board make for better decisions and thus increase company performance (or not), but in the line that investors expect the inclusion of women on the board and value companies that do so more highly than companies that do not. Another proxy could be RONA (return on net assets). This would provide an argument in terms of improved decision quality, but only about strategy and operations, not finance, since financial decisions (e.g., financial leverage) are not included in RONA. If this is the argument you want to make, fine. If you'd rather make an argument that includes financial decisions, you need to look for another proxy, such as ROE (return on equity). But more inclusion is not always good because you potentially confuse the effects: perhaps (hypothetically) women on the board improve strategic and operational decision quality but reduce financial decision quality. Then you might find no difference in returns on equity between companies with women on the board and companies without women. We have already briefly touched on the decisions you have to make in your research project. This is one of them. Using neither of the proxies would be wrong per se, but each has implications for your research. The three proxies mentioned are just examples. There are many

3.5 Step 4: Choose the Proper Analysis Method(s)

45

more that are available, at least in theory. Also, we have not yet verified the acceptability of the proxies. We do this in the literature review. • Last, we must think critically about achieving “comparability” of the companies examined. Otherwise, the counterargument would again prevail, that the difference in performance might result from other confounding factors. Again, we have a lot of different options with advantages and disadvantages and implication for our research: – to achieve a very high comparability between the companies, we could analyze the same company: when there have been no women on the board of directors and when there have been women on the board of directors. The election of women to the board of directors would then basically be an incident and we would analyze the impact of this incident. Obviously, the election of women to the board of directors would not be the only change happening to the company and its environment, so we would have to control for these additional changes. Also, we would still need a lot of companies to generalize the results. – another option would be to collect performance data of groups of comparable companies that are similar about several characteristics, e.g., industry, size, country, and economic environment. Then we could compare the performance of companies with and without women on the board of directors of the same group. If the groups are big enough, this would generate a good argument, if we assume that the characteristics chosen are really the key differentiating factors. Usually, this approach requires selecting a limited number of groups for data collection and analysis. Thus, generalizability is limited, as for example, only companies in one geographic region have been analyzed. – a third option would be to gather this differentiating information about a lot of companies and then try to esslish what the predominant factors are that determine differences in Corporate Performance, women on the board of directors being one of them (or not). – there are certainly other, ingenious ways to produce a result that makes a good argument. Just come up with one or several. You are now ready to go a step further and think about the research methods that allow to produce the desired results. We will cover this in the next step.

3.5 Step 4: Choose the Proper Analysis Method(s) Based on the desired result, you now look for a suitable data analysis method that provides such a result. While step 3 was also based on logical considerations (what do we need to do in principle to make a powerful argument?), step 4 is the technical side of the same coin: what methods exist that model such an approach and allow us to generate the results? What are the requirements to use these methods? Are you sufficiently familiar

46

3  Setting-Up the Research Process

with this method? What is the time frame of the project? For a student research project, you will probably rely on the methods you already know. For a doctoral dissertation, you will probably need to learn the method(s) that will produce what you envision as the best result or argument. In this step, it is important not to get carried away by the sheer cleverness of the result achieved by the brilliant method being used, but to consider what kind of results the method will really deliver. For example, it would be great if we could make an argument that the number of women on boards affects business performance. However, most of the statistical methods alluded to in Step 3 do not provide that kind of result. They produce results like “there is a difference in firm performance between the two groups that is unlikely if women on the board were not? a distinguishing characteristic between the groups” or “women on the board are correlated with firm performance.” Neither method leads to the argument “women on the board improve corporate performance.” Thus, in this step, we need to balance the arguments we want to make with the arguments we can make. Precision of language plays an important role here in terms of expectation management. Be aware not to promise arguments you cannot deliver. So, the wording of the desired outcome is of crucial importance.

3.6 Step 5: Collect Proper Data that Fit the Analysis Method(s) Based on the desired result and the analysis method to be employed, we can think about the data that we are going to collect, including data collection methods and operationalization of data. In step 3, we already thought a lot about data. Yet, this was more to produce a good argument. At this point, we must be more stringent. This means that for every single information we want to include in our argument (result and data analyzes method) we must find and establish: • proxies (with a high construct validity), • measurement methods (as accurate as possible), • data sets (not just isolated data) that belong to specific entities or examined objects (e.g., data for company A and B in year Y and not data for A in year Y and data for B in year Z), and • data sources (as truthful as possible). Let us briefly go through these requirements: We briefly touched upon proxies when we talked about how to measure corporate performance in step 3. The economic environment mentioned there is a good example, as it is extremely difficult to find a proxy that could represent the concept “economic environment”. Possible partial proxies for the overall concept of economic environment might be corporate tax rate, public spending ratio, GNP, growth of GNP and so on. This is unnecessary in our example, as we do not want

3.6  Step 5: Collect Proper Data that Fit the Analysis Method(s)

47

to describe the economic environment, we only like to make sure that it is the same for all the companies. Also, we may assume that in any year the economic environment is more or less the same for all companies. Thus, the year might be a viable proxy to use in our research. It is one proxy, and it is important to remember that it represents only “similar economic environment” and thus entails inaccuracies to a certain extent. (e.g., apparently some industries lead or lag on the economic cycle). Also, a proxy might not comprise one data but might be not be directly observable and result from combined and transformed data. This would mean that we would need to find data to establish a set of proxies to describe or define a concept or construct. (see Sect. 4.4.5 for more details on proxies and constructs). What we mean by this is construct validity, defined as follows.  Construct validity refers to the degree to which inferences can be made from the operationalizations in a study to the theoretical constructs on which these operationalizations were based” (Trochim, 2015). Construct validity is the overarching validity that includes that the name of our construct is correct (“job satisfaction” in our understanding might not be the same as “job satisfaction” in your understanding), the proxies used describe the construct accurately, and the operationalization of data collection and transformation is appropriate. The measurement method describes how the data are retrieved, transformed, and presented. Referring to “women on the board,” this means deciding whether to assign, for example, “1” for women on the board and “0” for none, or whether we count the number of women on the board, or the percentage of all board members who are women. Do we distinguish between executive and non-executive board members (if such a distinction exists at all in each country)? Establishing a dataset means deciding which data belong together for which proxy, forming a set (mathematically, a n-tuple). In terms of women on the board, for example, this means that new board members are elected at the shareholders’ general assembly most times at the end of the second quarter. So, is this a year with women on the board or not, especially in view of the developments this year? Would a newly elected board member even influence this year's performance? The same question arises for the year in which the board member was dismissed or resigned. The data source only describes where the data come from. Verifying that the required data is available is an important part of your research project’s risk management. It is very unfortunate and avoidable if the suggested data in your preliminary study is simply not available for your research implementation (e.g., interviews with subject matter experts or archival data). Consider the potential bias in your data. For example, few respondents would likely answer truthfully whether they have committed fraud within the last year. Again, there could be methods to get unbiased data even under these circumstances. If your research relies heavily on the truthfulness of this data, you need to make sure you use such methods. Only now can we address the research question. Let us have a look at the next step for this.

48

3  Setting-Up the Research Process

3.7 Step 6: Specify a Proper Preliminary Research Question Only now we can develop the preliminary research question(s). You might have recognized that this step is far away from the starting point of your research project. We have already taken many decisions and considerations. We believe this to be the reason too many students struggle to phrase an excellent research question: they lack the deliberations of steps 1–6. The research question is the question you really answer. So, it must be phrased as comprehensively and precisely as possible. The answer to the research question is the argument that you use to accomplish your research goal.

3.8 Step 7: Check the Consistency of Your Research Design You might wonder how to check something that you have not yet established. But in fact, you have! You have made various decisions in the previous steps: • you have defined the data you want to collect and the sources from where you are going to retrieve them, • you have specified the operationalization of your data, the transformation, the constructs, and their proxies, • you have chosen the data analysis methods that you are going to apply to your data and the type of results you will get out of them, • you have stated your research question, and • in fact, yes, you have established your research design. As you have an overview of your specific research design now, a good start for checking the consistency of your research design is to label it: which type of research design have you selected (see Chap. 5 for an overview)? Is it for example a single case research design or a cross-sectional research design? Does none of the research designs fully match your research decisions? In this case, check whether you have designed a staggered research design (i.e., a sequence of two or even more designs). If you still face problems selecting the definite research design, ask yourself: have you forgotten to make relevant decisions? Have you taken additional decisions that are not necessary for a specific type of research design? Do you have good reasons for them? Here, the goal is not that you follow the brief description of each research design word for word. Just identify potential deviations from the research designs and reason your choice well. We arrive at the most important step in the consistency check. Based on your increased knowledge about what your type of research design can and cannot do, is your specific research design able to deliver? Can you generate results that support your

3.9 Step 8: Check the Feasibility of Your Research Project

49

arguments to draw the desired type of conclusion? If so, well done. If not, choose one of the four alternatives: • adjust and change the decisions and derivatives in steps 2–6 to develop a research design that lets you draw the desired hypothetical conclusion • adjust the desired hypothetical conclusion towards a conclusion your research design can deliver. Make sure that you can still generate an intellectual contribution with your research and that the result of a downward spiral in following these steps of a research process is not a meaningless conclusion. (E.g., adjusting, and readjusting research design and hypothetical conclusion in several cycles and then coming up with the fictitious conclusion “companies paying lower salaries have or do not have lower personnel cost”.) • do both, adjusting the desired hypothetical conclusion and research design to better match each other. • abolish your research idea. If you cannot match a desired hypothetical conclusion that interests you with a research design that you feel comfortable with, you need to look for other research areas. At this point, you have checked that your research design is (as far as you can anticipate it) able to achieve your research aim. Next, you need to check whether your research design is realizable.

3.9 Step 8: Check the Feasibility of Your Research Project This check does not refer to the validity and logical structure of your research but relates to the ability and willingness to execute your research. Consider the following to check for feasibility: • constraints in the execution: will you be able to conduct the research described in your research design in the time allotted? Will you have access to the data you need? Are you able to finance your research? • risk management considerations: ultimately, you hopefully hand in your research paper on time. How big is the risk that you cannot do so or not in a satisfying manner? What risks can you mitigate and how? (this might relate back to your research design). These risks might refer, for example, to find interviewees, to low response rates, or to answering behavior. Other risks might involve getting unusable data (think of the worst data or response you could get. For example, your interviewee simply answers no to your questions with no further explanation.), data not meeting the requirements of your method of analysis, and so on. • ethical behavior: can you ensure ethical behavior in your research? Can you protect privacy, integrity, anonymization, and confidentiality of participants and results? Most

3  Setting-Up the Research Process

50

of the ethical considerations refer to the data collection and analysis methods, and the reporting of the research. In a nutshell, nobody should be compromised or feel compromised in their personal integrity by your research. Often universities have established ethical boards to ensure that all research adheres to high ethical standards. Despite its overall importance, we only address ethics if the type of research design itself causes ethical issues. Unethical research has a lower feasibility, and rightly so. If your research design is consistent (step 7), has a high probability of feasibility, and does not violate ethical standards, you can start executing it. If your research design cannot or should not be executed, you must change the decisions made in any or all the above steps.

3.10 Step 9: Conduct Your Research Project Finally, you conduct your research along with the decisions you made about your research design. As a rule, a good research design is easier to execute than a poor one. However, you inevitably encounter additional decisions to take. For example, you may think of alternative explanations that you did not consider during the planning phase. Or, while conducting the interviews, you got the distinct impression that interviewee XYZ lied to you. Despite running a pretest, you get the impression that the recipients of your survey understood “question ABC” differently than expected. Or your data does not meet the planned test. How do you tackle these challenges? Ideally, have already considered all these eventualities while developing your research design. Reality shows that you cannot foresee everything. If such things happen, you need to go back to step 7 and check for consistency again, given the modified situation. If the deviation from the research design prevents you from producing the results, making the arguments, and drawing the conclusions, then you must adapt accordingly. What we do not recommend is to ignore the deviations just to conduct your research as planned. In this case, you might violate the conclusion validity in the worst case. Supervisors then usually conclude that a researcher is not capable to conduct scientific research. Key Aspects to Remember

Start with the conclusions, not the research question Surprisingly, we urge you to start at the very end of the research process. This is opposed to what most research literature recommends. You can do so by envisioning the conclusion to your research report. So, the most important question is: what type of conclusion would you like to draw from your research? Or similarly: what kind

3.10  Step 9: Conduct Your Research Project

51

of statement would you like to make? The answer to these questions leads to your research aims. Challenge your conclusions thoroughly If you have envisioned a reasonable conclusion of your research project, think about how you would defend your conclusion in a challenging discussion. Thus, decide about what arguments you could use to make such a statement and what counterarguments you will probably face during the defence. The better the arguments you can develop and the weaker the potential counterarguments, the stronger your conclusion will be. Since you do not know yet in which direction your conclusion should go, this seems far-fetched. Yet, it suffices to collect pro and con arguments without already taking a side. Choose your research methods wisely Based on the desired research results, search for suitable research methods that can generate these results. This is the technical side of the same coin: what methods exist that model such an approach and allow us to generate the appropriate results? What are the requirements to use these methods? Are you sufficiently familiar with this method? What is the time frame of the project? For a student research project, students often rely on the methods they already know. Yet, for a PhD dissertation, researchers may become familiar with the method(s) that will produce the most reliable and valid results. Develop a sound, preliminary research question Develop (a) preliminary research question(s). Please be aware that this step is far away from the starting point of your research project. You have already taken many decisions and considerations about your project. Therefore, too many students have difficulty formulating an excellent research question: they lack the considerations of the previous steps, specifically the very first step about starting with a reasonable conclusion. The research question is the question you like to answer, and you want to answer it comprehensively and precisely.

Critical Thinking Questions

1. Why is it important to start with the conclusion rather than with the research question? 2. What role does reflexive research play in the research process? 3. What do we mean by “the value of information is the degree of reduced uncertainty”? 4. What is the main purpose of checking the consistency of your research project? 5. Why is it difficult to establish internal validity in research?

52

3  Setting-Up the Research Process

References Blessing, L. T. M., & Chakrabarti, A. (2009). DRM, a design research methodology. Springer. Dresch, A., Pacheco, L. D., Cauchick, M., & Paulo, A. (2015). A distinctive analysis of case study, action research and design science research. RBGN, 1116–1133. Streefkerk, R. (2021). Internal vs. external validity. Retrieved May 28, 2021, from https://www. scribbr.com/methodology/internal-vs-external-validity. Trochim, W. (2005). Research methods: The concise knowledge base.

4

Writing up a Research Report

Learning Objectives

When you have finished studying this chapter, you will be able to: • • • • •

write up a state-of-the-art research report understand how to use scientific language in research reports develop a structure of your research report that comprises all relevant sections assess the consistency of your research design avoid dumbfounding your reader with surprising information

4.1 Introduction to Writing a Research Report In the previous sections, we stressed the importance of increasing the body of knowledge. This body refers to society, i.e., what is known by all of us about a topic. This requires dissemination of the research project’s conclusion. To avoid a chasm of believers and non-believers in this conclusion, its validity must be shown by explaining how and why it has been drawn. You disclose all the decisions, reasons, and impacts along the entire research process, divulging your specific research design and its execution. Thus, you must write up a research report to add your research to the body of knowledge. To make sure readers find the information regarding conclusion validity and to facilitate the writing process, a commonly used structure for research reports has emerged. The standard research report comprises the following 7 sections:

© The Author(s), under exclusive license to Springer Fachmedien Wiesbaden GmbH, part of Springer Nature 2021 S. Hunziker and M. Blankenagel, Research Design in Business and Management, https://doi.org/10.1007/978-3-658-34357-6_4

53

54

4  Writing up a Research Report

• introduction, • theoretical background, • literature review, • methods, • results, • discussion, and • conclusion. This may vary according to your institution’s demand and your research project. So, the structure is more of a guideline, but the issues covered in the sections are universal: substance over form. The does not change with the length of your research paper. An article in a scientific journal has the same basic structure (and issues to address) as bachelor’s thesis, master thesis, and a doctoral dissertation. In the following, we describe the structure and the issues to be tackled in each section.

4.2 General Remarks on Scientific Writing Before describing the purpose and content of the various sections of a research report, we address several misconceptions about academic or scientific writing: • no surprises: You do not write a drama or a crime story. There is no building of suspense required. This does not mean that academic texts have inherently to be boring, but the interest stems from the research questions and how it gets answered. The reader is interested in your conclusion and process of argumentation. So basically, nothing should come as a surprise to the reader, but everything should be explained: why you are doing what you are doing, where you are in your report, and what comes next. “No surprises” also holds true to your discussion of arguments. Do not explain to the reader that your findings are in line with the findings of researcher Jane or John Doe if this study drops out of the blue. You need to mention it in your literature review first. • repetitions are fine: Together with “no surprises” comes the necessity to reiterate some issues. That is OK. Better to repeat something than to leave the reader looking for information that he/she expected at this point of the report. Referencing to more details to other parts of your paper might help to prevent too tedious repetitions. • using first person pronouns is not a taboo: Referring to yourself or the author(s) as “I” or “we” is fine. Often it is still taught that scientific texts should not entail personal opinions (we will address this in the next general rule) and to appear as objective as possible. The author(s) should refer to themselves as “the author”. This is not inherently wrong. So, you can refer to yourself as “the author”. But this is rather outdated for several reasons: First, within a research project a lot of decisions must be made. There is no algorithm that you feed with a research questions that results in a

4.2  General Remarks on Scientific Writing

•

•

•

•

55

research report and a conclusion. Writing about these decisions from a first-person perspective shows that these decisions could have been made differently. There are (hopefully) good reasons you did it this way, but it has been your decision and you are excepting the responsibility that goes along with this decision. Second, referring to yourself as the first person makes distinguishing between your opinion and other author’s opinions much easier and formulations much less cumbersome. This is valid especially in the literature review and in the discussion section. Just imagine criticizing another author’s study: “The author of this report thinks that the author of that report is wrong because that author jumped to their conclusion without making this step which this author deems absolutely necessary.” This has the potential to turn your report into a linguistic nightmare for your readers. your own opinion is important and valued: You are the researcher. It is your research project. You are conducting a research project and are deciding along the way. It is not only perfectly legit to state your (well-reasoned) opinions, but also a necessity and a major part of your intellectual contribution. The only thing to remember is that it needs to be abundantly clear what somebody else did, found out, thought, etc. and what you did, found out, thought, decided, etc. The critical reflection of other’s research studies and your own findings in the light of what other researchers found out is an important and integral part of your report. Without it, your intellectual contribution would be critically diminished. user guidance is important: This goes together with “no surprises”. Do not force the reader to draw his/her own conclusions and find out what is happening but explain to him/her what you think you established so far and what you will do next to achieve what. This still leaves the reader with all the information to draw his or her own conclusions but makes it much easier for them to follow your line of thinking. relevance of content: Everything you write must be relevant for your argumentation. So, you do not write everything you know but you need to curtail it to those parts that impact your arguments and conclusion. For example, ff the reader reads about a theory in your theoretical background they expect you to do something with this theory: that you will elaborate on this theory or collect evidence that confirms or rejects this theory or something else. What they do not expect is that this theory is never referred to in your research again. This makes final revision of your report both important and taxing: nothing should come as a surprise but everything you state needs to apply to your argumentation. This requires you to check that there are no loose ends nor beginnings. past tense: Write your complete report in the past tense, describing what you did and the decisions you made. This seems obvious at a first glance but during the research project usually two issues arise. First, how can you deal with the additions to the body of knowledge? Should they not be in the present tense as arguments and conclusions are (still) valid now? Here writing in English makes for easier writing as the simple past denotes action that is ongoing: you drew this conclusion (and still do). Second, the same question arises about the preliminary study. Often a preliminary study

4  Writing up a Research Report

56

detailing the plan for your research needs to be handed in and approved. As this happens before the actual executions it would seem natural to write the preliminary study in the future tense. This would require rewriting everything in the past tense when the research has been conducted. Save the effort and write immediately in the past tense. Your supervisor should be able to infer the timely sequence. With these general remarks in mind, we facilitate the writing process and make the underlying line of thought of the report’s section more obvious. We start with a summary of all sections to get a better overview of the reasoning in your report before providing a more detailed description.

4.3 Overview of the Sections In this chapter, we present an overview of the section in a research report. It helps to know what part of your argument your research report represents can be found where before detailing how to argue in each section. Management summary This section contains a summary of the research. The principal arguments of each chapter are condensed into a standalone one-pager. Introduction In the introduction section you should briefly explain why your research is relevant. You should state your research aim and research question (no surprises). Lastly, you should briefly describe how you structure your report. Theoretical background In this section you state or reference theories that underlie your research or the definitions of concepts that you employ in your study. It could also contain a description of the circumstances of your study, for example, a description of the company and its products you are evaluating the marketing concept for the entry into the French market for. Literature review What is the current body of knowledge about your research aim? What models and empirical evidence already exist? What alternative explanations have been mentioned about the results? This leads to establishing the research gap that your research will (partially) fill. Literature review and theoretical background encompass the existing body of knowledge that is relevant for your argument.

4.3  Overview of the Sections

57

Research design In this section, start with (re-)stating your research aim and research question that should logically follow out of the detected research gap. Then you explain your research design, i.e., how you collect data, what data you collect and how you analyze this data. A preliminary study usually encompasses all the sections up to this point. You have planned what you are going to do. The remainder is the execution of this plan. In most universities, supervisors give you feedback about your preliminary study. If not, but your institution’s process allows for it, it is a good point in time to ask for such a feedback. While executing your research design, it is usually not any longer possible to introduce major changes without infringing the consistency of your argumentation. Results In this section, you present your results of your data collection and analysis. You only report the facts without interpreting and discussing them. If your methods of analysis required any validation or checking of assumptions, show the results of these checks, too. Discussion Here, you interpret the results that you have found. What do the results mean? Do they answer your research question? What is the contribution of these results to the existing body of knowledge? Can you confirm previous findings? Are there any contradicting findings? Do your results change or put into perspective previous findings? What are the limitations of your research? Which of your decisions might have affected the results? Can future research address aspects differently? What alternative explanations would also be possible based on your results? What research question should or might future research address to tackle the remaining research gaps, eliminate alternative explanations, or otherwise generate intellectual contributions? What are the practical implications of the results and your interpretation? This might warrant a separate chapter, depending on the research question. Also, a part of the discussion section is devoted to a critical reflection of your research. What went well, what did not? Where have to deviated from the planned research design? There is no clear distinction between these reflections and the limitations of your research apart from potential “lessons learned”. Conclusion Here you sum up your research. Did you answer your research question? This answer allowed you to come up with arguments and to draw the conclusion. This is the last point of your report. You can sum up what future research should investigate (to round up your argumentation and conclusion). You can also list practical implications. In the conclusion section, no new ideas, thoughts, etc. should appear.

58

4  Writing up a Research Report

4.4 Sections of the Research Report in Detail In the following chapter, we present the purpose of each report section and provide some guidance how to write them.

4.4.1 Management Summary The management summary is the beginning of your report. It condenses your research design, results, arguments, and conclusions into a brief stand-alone one (or at most two) pager. Purpose The management summary, also called executive summary or abstract, summarizes the entire report. It enables any reader to read this summary alone, without reading through the complete research report, thesis, or dissertation. It allows the reader to understand the most important information of the project report. It aims at persuading the reader that the document is worth being read. This abstract is the part that is often published in internet databases. So, it should provide concise, complete, specific, and self-sufficient. How to write The management summary is a condensed version of the complete project report. Obviously, it can only be written when you have finished your report. So, it is the first chapter in your report, but the last one you can write. It contains the essence of your introduction, theoretical background, literature review, research questions, methods, results, and discussion. You can use the following statements as a guideline. It might be a good idea to read through the following chapters about writing a research report, looking at the purpose of each chapter, and then come back to this section. • the introduction will be condensed to the establishing of relevance of the topic: why it is important that you are doing your research? This should be conveyed in one or two sentences. • the theoretical background is essentially reduced to mentioning the (main) theory or theories that you use. For example, you do not explain the principal agent theory, but you state that your research is based on the principal agent theory. • the literature review is only referred to about the research gap that you detected through your literature review. So, you will not cite your sources included in the review, but you will summarize the research gap. If your research mainly aims to enlarge or contradict other research, naming this research is mandatory, but should then basically be part of the introduction. Phrases you could use are (for a more

4.4  Sections of the Research Report in Detail

• •

•

• •

59

elaborated overview see Barros, 2016). According to (or based on or summarizing) the literature review, – the topic (or problem, question, phenomenon, element, etc.) xyz deserves more research attention, – most studies have focused mainly on topic (or problem, question, phenomenon, element, etc.) xyz, – most of the work in this area has focused on topic (or problem, question, phenomenon, element, etc.) xyz, – there is limited research investigating topic (or problem, question, phenomenon, element, relationship etc.) xyz, or – there is scant evidence that (elements, conditions, relationship etc.) xyz exist (or are understood, behave that way, are relevant, etc.) the research question should be stated in full. the research design is described including sampling, data collection and data analysis methods applied to answer the research question: – the basic type of research design should be named. For example, cross-sectional research. – the data sources, for example sample and sampling method, should be briefly described, for example 50 companies in the financial industry sector in Western Europe. – the data collection should be reduced to the method’s name, e.g., closed survey. – the data analysis method should also be reduced to the method’s name, e.g., regression analysis with element X as the dependent variable. All testing of requirements and preliminary analysis should be excluded unless they yielded extraordinary results that affected your research. the results are summarized to the results of your ultimate data analysis method. For example, the multiple regression analysis showed a significant impact for the independent variables xyz. For quantitative data analysis methods, also mention impact size and explanatory power. the discussion should be summarized in your most important arguments and interpretations of the results. if your research project includes recommendations, include them as well (depending on your research design this recommendation might be your conclusion).

The management summary is neither an introduction nor a verbal summary of the table of contents. It summarizes the project report (not the steps of the project). Usually, you provide the management summary on a separate page at the very beginning of the research report. Your main report starts after the management summary. You do not have to guide the reader from the summary to the introduction, as the summary is supposed to stand alone.

60

4  Writing up a Research Report

4.4.2 Introduction The introduction is the first section of the actual report. Gaining the readers’ attention and keeping it by expectation management and proper user guidance is key. Purpose The purpose of the introduction is, first, to catch the reader's interest. Explain why this topic is relevant to you and to the reader. Why is it important? How did you come up with this research? It also shows the scope and direction of the paper, acting as a user's guide for the reader. So, it delineates the entire story you plan to convey in the body of your research. Describe how you interpret and how you approach the topic at hand: • what do you consider a problem? • what is the research aim? and • what is the research question? As your report is a research paper, it is perfectly fine to state your research question and research aim in the introduction, so that everything that follows can be understood in the light of those research questions. It should also indicate your type of conclusion and point of view. Again, in line with the directive of “no surprises”, try to make your conclusions and research understandable to the reader. How to write The introduction serves as a roadmap for the reader and helps them understand what you do in your research report, where you go, what you do to get there, and what the reader will see along the way. Everything must logically flow from a starting point; nothing should come as a surprise. The entire story is outlined in the introduction. Provide details only in the body of your report. So, this is the foundation on which you build the logical next step to reach a conclusion that answers your research question. Try to keep the structure of the introduction simple. An effective way is to start with a rather general statement about the topic. And then gradually narrow down to the specific thesis in the form of the research question or the type of conclusion to be drawn. The length of the introduction depends on the overall length of the research paper. A useful rule of thumb is that the introduction should be no shorter than one-twelfth and not much longer than one-tenth of the total length. If the assignment is a 2000-word essay, the introduction should be between 160 and 200 words, while for a 3500-word report it should be between 290 and 350 words. There is no absolute rule for the length. Be as reasonable about it as you can. The introduction contains the relevant background of the problem. You introduce relevant concepts of the research topic and significant research. You offer user guidance or an essay map, a thesis statement, or a research question. Brief and relevant background

4.4  Sections of the Research Report in Detail

61

information shows how your topic fits into a larger framework and what approach you take. This can also guide your readers in the direction you like them to go. You can show your audience why the topic matters and create a map that includes the scope and direction of the research. The entire introduction is an essay map. Because the introduction serves as a guidance for navigating the research paper, the statements usually begin with phrases such as “this report examines …” or “this essay will …” and “this article shows …”. We recommend that you adopt this strategy. So, you do not forget to put the thesis statement in your introduction, and your reader has a clear idea of what it will be about. Also include the research questions or, depending on your research design, the hypothesis. Delineating the scope is also part of the introduction. Since you do not explain how the entire world works, you usually need to delimit the research. So, decide on what you intentionally leave out. Remember, a poorly written and poorly constructed paragraph does not tell your audience what how to move forward and navigate through your written report. Do not risk losing your reader's goodwill from the very beginning, no matter how well the rest of the paper is constructed. Thus, capture the reader's interest and satisfy their needs in terms of what they can expect from your research. After the introduction where you have plotted the report, you guide the readers to the next section with a statement like “after having introduced the problem area and the research aim, I will now detail the theories underlying the further deliberations and the research design.”

4.4.3 Theoretical Background In the theoretical background, you depict the foundations of your research: the theories you build your research on, the concepts and definitions you employ and other background information (like a short description of client, sponsor, or company being used as case) necessary to understand and position your research. Purpose The theoretical background lays the theoretical foundation for your research. So, all theories that your project is based on as well as all the definitions of concepts you use should be known by the reader afterwards. With management and business administration as applied sciences, there might (yet) be rather few elaborated theoretical concepts that apply to your research. Usually, you start with a theory that the reader should know. Also, there surely are concepts and definitions to present. How to write In the theoretical background section, you describe theories that you base your project on. By necessity, this description needs to be brief. Focus on the aspects of these theories that are relevant to your research. You can assume that the reader is aware of the

62

4  Writing up a Research Report

pertaining theories. So just highlighting the key aspects regarding the relevance to your research project is sufficient. Do not tell the audience that this is something that they should already know. Describe the theories based on the major papers that made up or established these theories as sources. If possible, consider the original sources establishing these theories, major adjustments, or refinements. Sometimes, it might be useful to establish or to introduce a section “background”, where you basically provide a more detailed background or additional information about your research than you could do in the introduction. For example, describe the company that represents your unit of analysis, or introduce the problem that your design science research addresses. So, build the foundation for your thoughts that will follow. This foundation might be very valuable to the reader. For example, to understand a case research design that aims for a comprehensive description of the case, this “background” information also contributes to the comprehensive description. Introducing this information already in this chapter rather than later in the results section, makes the understanding your thoughts easier for the reader. It may enhance your argumentation and explanations. Specifically, if the unit of analysis is not the entire company, but only a part of it (e.g., a department or a strategic business unit). After having introduced the theoretical background of your research, you guide the readers to the literature review. You could use phrases like “after having described the theoretical foundations for this research, I will now present and critically annotate the current state of research relevant my research area in the literature review.”

4.4.4 Literature Review The literature review collects, comments, and criticizes existing research in the relevant field to make an argument for the relevance of your research and its design. Purpose The purpose of the literature review is twofold. First, you need to be aware of the body of knowledge to think about what your intellectual contribution could be. You need to be familiar with the current state of research in your area of interest. So, collect information about previous research questions, methods, findings, and conclusions. This is the primary purpose of conducting the literature review. Second, the literature review justifies your research. Basically, you can show what has not yet been covered in the current research. This leads to the research gap that you plan to close by your research. Convince your reader that you substantially contribute to the body of knowledge. How to gather the information Sources  Standard sources for searching for literature are databases. These databases offer access to many scientific and professional journals. Which databases you might have access depends on your organization. If your databases do not offer enough relevant

4.4  Sections of the Research Report in Detail

63

research results, you may visit university libraries that provide access to those missing databases. Promising databases for research in business and management are, for example: • • • • • • • •

Academic Search Premier (EBSCO) ABI/Inform Complete Business Source Complete (EBSCO) EconLit (EBSCO) Factiva (Dow Jones) Google Scholar JSTOR (ITHAKA) Nexis Uni (LexisNexis).

What to look for  You usually start looking for keywords and authors in the field. The aim is to find all the studies that comprise the relevant body of knowledge. The number of articles in your literature review depends on your research and the length of your project. Obviously, the literature review for a doctoral dissertation needs to be more thorough than for a bachelor’s thesis. The often-heard question “how many articles do I have to include in my literature review?” is unanswerable by your supervisor and strongly hints at a completely faulty understanding of the purpose of the literature review. You need to review sufficient articles to grasp the body of knowledge and to convince your reader that your research is justified. You can only exactly define a gap in the body of knowledge by the knowledge bordering this gap, same as a vacuum is defined by the absence of anything else. An important tool to approach a literature review is a keyword list with the important terms in your field. Here it is beneficial to be specific and to add synonyms to your list. “Accounting” is a generic term whereas “management accounting” is more focused. However, “managerial accounting”, “cost accounting” or “costing” might be used instead of “management accounting”. A potentially very rewarding addition might be “state-of-the-art”, “meta-analysis” or “literature review” to get meta-studies about other studies in the field. Another approach–usually used in combination with the keywords–is to look for known authors in your field of research. You will find these authors in textbooks and the literature reviews of articles you have already found (there you might find additional keywords and synonyms). To look for known authors should not replace a keyword search to prevent you overlooking relevant research by other authors, but it might be a good check for your keyword search: if the results of your keyword search do not include the already known authors (or articles) your keyword search is still lacking. Establishing a search string The literature search with keywords rarely results in a manageable or a relevant list of literature. Therefore, you must refine your research using search strings. The details might slightly differ between databases, but the search strings

64

4  Writing up a Research Report

are based on Boolean operations, especially AND, OR and NOT. Any data base query yields a list of data entries (articles) that meet the search string (for example the keyword “costing” the field “title”), this means that the condition “keyword is contained in title” is TRUE. With the help of Boolean operators this condition can be stated increasingly precisely (see Table 4.1). For example, if you look for “management accounting” and want to include the synonyms from above your search string would look like this: (“management accounting” OR “managerial accounting” OR “cost accounting” OR “costing”). How to compile the information Many research reports lack an in-depth literature review and have problems to remedy this even after criticism of their supervisors or readers because of an unsystematic or not existing information collection. After gathering all the relevant articles, you need to keep the pertinent information, for example, in a table. Things to look for (and to keep) are: • reference number (for internal referencing), • author(s) • year of publication • journal • ranking of journal • number of citations in other research reports/articles • empirical evidence (yes/no) • theoretical deduction (yes/no) • research objective • research question(s) (Hypothesis, if applicable) • research design • sample – size – characterization (country, industry, stock exchange, etc.) • constructs and proxies – constructs included – proxies used for constructs • data collection method

Table 4.1  Specifying research strings using Boolean operators

A

B

A AND B

A OR B

NOT A

TRUE

TRUE

TRUE

TRUE

FALSE

TRUE

FALSE

FALSE

TRUE

FALSE

FALSE

TRUE

FALSE

TRUE

TRUE

FALSE

FALSE

FALSE

FALSE

TRUE

A

B

A AND B

A OR B

NOT A

4.4  Sections of the Research Report in Detail

65

– primary/secondary data – source(s) – operationalization (calculation, characterization, and retrieval specification) – handling of outliers – year(s) of data • Data analysis method – quantitative or qualitative – method – testing of requirements executed and successful – auxiliary means or tools • Results – complete results – significance – impact size – power of explanation – categories used – qualitative results • discussion – interpretation of the results – alternative Explanations – limitations of the study (according to author) – recommended future research • conclusion – major argument made • grouping of literature according to (list of reference numbers in the same group) – argument – sample characteristics – research design – data collection – data analysis – constructs included – proxies used – recommended future research – additional ones depending on the preliminary research gap and the argument for your research • your annotation (your opinion) (as part of literature review) – academically sound argument (yes/no) – (potentially) to be used in your presentation of literature review in detail (yes or no) – (potentially) to be used in your presentation of literature review as reference (yes or no) – reason for your evaluation in bullet points

4  Writing up a Research Report

66

– – – – –

interesting literature referenced (to include in your literature review) (potential) Constructs to include in your research (potential) Proxies to use in your research (potential) Research Gap detected/ideas for your research (potentially) to be used in your discussion (because it makes a good argument) (for or against).

How to write As already mentioned, the literature review section in your report must fulfill two purposes: • convey the impression that your knowledge about the topic is up to date, and • make a convincing argument for the intellectual contribution of your research. You can achieve the first purpose by summarizing the state-of-the-art. Either by lumping all research together (existing empirical evidence is unanimous) or dividing them using the argument being made (existing empirical evidence is unconvincing or conflicting). By structuring the presentation of the state-of-the-art, you can already prepare the argument for your research. Depending on the length of your paper, the presentation of the state-of-the-art will very much vary in length. In a journal article you usually only present the reference, the results (often unfortunately only as significance levels), and the argument being made (conclusion). In bachelor or master thesis, you should usually present exemplary research studies providing a lot of details and then grouping the other research based on their similarity to the exemplarily presented one (same argument, same research question, same research design). Suitable candidates for a full representation in your literature review are studies that: • • • • •

have been published in a high ranked journal, have been cited many times, provide a lot of information about the research itself, you understood completely, and made a good argument

The above grouping makes sure that you do not bore the reader but still use most or even all the articles you have red. Based on the summary of the state-of-the-art you must argue for your research. Your primary argument for your research is your detected research gap. The research gap summarizes the literature review about the absence of knowledge.

4.4  Sections of the Research Report in Detail

67

Research gap Usually, the research gap is the last section of the literature review. To make a strong argument for your research, it is important to realize that the literature review, the research gap, the research aim, the research question, and research design form one linear string with one step flowing logically from another. Thus, in the presentation of the knowledge, you already prepare the delineation of the gap, i.e., the missing knowledge. The presentation of the research gap allows you to argue for the intellectual contribution you plan to provide by your research. We can conclude that the larger the intellectual contribution, the better the argument. Referring to the section about intellectual contribution, there are several possibilities in (roughly) ascending order (first options producing smaller intellectual contributions): • replication of an already conducted study to verify the achieved results, – without changes – using a different sample – using different proxies and/or operationalizations • excluding alternative explanations, • including different constructs, • resolving conflicting results, and • refining (creation, elaboration, and testing of theories). Depending on the aim of your module or course, the level, or your program (undergraduate, graduate) and your institutions, the requirements for the intellectual contribution may differ. This adds to the necessity to argue for your research. For example, using a sample from a country that has not been included in prior studies is a rather weak argument. However, hypothesizing that so far conflicting results might be caused by so far neglected characteristics may be meaningful. So, you use this sample from this country because this characteristic is clearly included in a manner in that enables a much stronger argument. For example, do not argue that until now the impact of women on the Board of Directors on company performance has not yet been researched in Switzerland. Rather, argue that ambiguous results for this question might exist because of different cultures (i.e., women’s opinions voiced and heard in public). A cultural trait in Switzerland might be that women speak up and are heard in public. Thus, if an impact on company performance really exists, in a country like Switzerland, it should be detectable. Based on your literature review, use the recommended future research, or the limitations of previous research as a starting point. Depending on the quality of those studies, these might already be quite extensive. Hopefully, in your annotations, you found additional alternative explanations for the presented results. Those alternative explanations offer the potential for a significant intellectual contribution. For example, studies about diversity might show lower average salaries based on gender (women) and ethnicity. An alternative explanation might be education and experience that often is lower for

68

4  Writing up a Research Report

non-white persons (education) and smaller in years for women because they are often tasked with raising a family. The description of the research gap is the first part where you contribute to the body of knowledge, as you state what this body lacks. Here you can show creativity and academic acumen. This is the foundation of your research, so it is important to make it a strong one. After having reviewed the relevant studies in the literature review and depicted the research gap, you guide the readers to your research design and how you plan to close that gap: “after having presented the research gap, I will now depict the design of my research to close this gap.”

4.4.5 Research Design The research design details all decisions that specify the execution of the research, including the reasons for them. This results in said “blueprint” that would enable other researchers to replicate your research.

4.4.5.1 Purpose This section aims to describe your plan for your research. It contains your research design. Hence, it should be labeled “research design” to reflect its content. However, the label “method section” is well established. Here you describe what like to achieve and what you do in your research. You establish replicability as one quality criterion of scientific in this section. So, using the information provided here, anybody else should be able to replicate your research. 4.4.5.2 How to Write Research aim The research design section usually starts with the research aim. The research aim is the intellectual contribution you want to achieve (i.e., the argument you like to make). The research aim is based on the research gap stated at the end of the literature review. Your goal does not have to be to close the entire gap, as this may be impossible to accomplish. In the broadest sense, your goal is to contribute closing parts of the gap. The research aim is the general description of what you want to achieve with your research and is substantiated on the research question. The research aim states the desired intellectual contribution of your research. (see Sect. 2.3 for types of intellectual contribution). The conclusion that you like to draw at the end of your research refers to this aim, drawing on the argument your research generated. (for an overview of the intellectual contribution of each research design, see section Sect. 2.1.).

4.4  Sections of the Research Report in Detail

69

At a first glance, the difference between research aims and the research questions does seem to be small or even neglectable. However, it is an important step in the logical sequence. Research question The research question is the specific, detailed and fully developed question that you answer. You will accomplish the research aim by answering the research question. The formulation of the research question is a powerful sign of the research design that you need for answering the research question. As we introduced in Chap. 3, the statement of the research questions results from an iterative process. It always should be a “working” research question. Depending on the further decisions in this method section, you might adapt or sharpen your research question. In some research designs, the research question is a hypothesis that will be tested. Sometimes, students turn the relationship of research question and research aim upside down. They start with the research question and detail-based on this questiongoals they want to achieve. First, this leads to a logical leap between research gap and the research question, as the question does not immediately follow from the gap. Second, the research question tends not to be focused enough to diminish this leap. Third, by trying to find goals substantiating the question, they either state the distinct steps of the research process as objectives or they invent additional objectives that either dilute the focus of the research or are even not attainable, raising wrong expectations. (Type of) research design The (type of) research design briefly describes the basic design your study will follow, the general direction of your decisions about your research. Your specific research design, the specific decisions you made, will be described in the next sections. But in line with the “no surprises” and “user guidance” directive, it enhances the understandability of your specific decisions if you briefly present the type of your research design. Your type of research design in business and management is usually one out of the list below: • • • • • • • • •

design science research design, action research design, single case research design, multiple case research design, cross-sectional research design, longitudinal research design, experimental research design, literature review research design, and staggered research design.

70

4  Writing up a Research Report

We describe these research designs in Chap. 5 ff. Most of the types of research designs involve different methodologies. If you use one of these methodologies, present it here (e.g., an event study as a particular type of longitudinal study). If you did not find your intended (type of) research design in the list above, refer to the research designs in Chap. 5 to see if we have labeled it as a methodology within a research design. The choice of the research design results from your research question. It should be immediately apparent that the research question and the (type of) research design fit together. For example, “does mental fatigue impair decision quality? Conclusion from experimental research” fits, whereas “does Corporate Social Responsibility improve corporate performance? A case study” does not. If they do not, reconsider the wording of your research question or choose another research design. This “fit” is the most important determinant of the validity of your research. If you cannot achieve this “fit” regarding this fundamental question, no subsequent decision will compensate for it. Consequently, your conclusion validity will be severely compromised, at the very least, if you can achieve any at all. Sample and sources As part of the research design section, describe the choice of the sample and data source(s). Argue why this source or that sample meets the aim of your research project. Specifically, if collecting qualitative data from non-random sources, the selection and access to the sources (for example, interviewees) is very important. For example, if you are conducting a case study about “How environmental considerations impact management decisions at xyz” getting the C-suite managers to participate is an absolute must. If you cannot use them as sources, you must abolish your research as you will not get the pertinent information. To phrase it positively, here you argue that and why you can collect the information from the sources you target. This also holds true for quantitative data, even retrieved from databases. Because of the choice of the source or the sample, your data might be inherently biased. You might even use a cleverly chosen sample to prevent or exclude a lot of counterarguments and alternative explanations. For example, in the impact’s discussion of the balanced scorecard on firm’s performance, a prominent counter argument for the absence of a relationship has been that the BSC might not have been correctly implemented. For this reason, Früh et al. (2019) examined the companies in the BSC hall of fame of the Palladium Group, the consulting group of Kaplan and Norton. Here, the companies are listed that the BSC’s inventors assessed as having excelled in implementing the BSC. So, for this sample the alternative explanation “not correctly implemented” should not apply. Hence, the absence of a relationship between BSC and corporate performance cannot be explained away using this counterargument.

4.4  Sections of the Research Report in Detail

71

Data collection What data or information do you collect? This requires a description of the concepts and constructs used as well as the proxies used. For all data, be it qualitative or quantitative, you need to operationalize the data collection. First, think about what kind of information you need for your research and gather ideas where (see sample and sources) and how to retrieve them. This information refers to the constructs or elements you use in your research. Basically, you establish an information guideline (in analogy to an interview guide) where you list the information you want to collect and how you plan to get the data (for example by using proxies or asking specific questions.) Let us take company size, for example. If, based on the theoretical background and literature review, company size is an important or at least relevant construct for your research, you must gather size-related information. The next question is, what information do you need about company size? Do you look for absolute or relative values, a categorization (e.g., big, medium, small), observable facts or opinions (whose)? What proxies do you use for non-observable constructs? These might include market value, revenue, net operating assets, balance sheet total, and employees. How do you operationalize the proxies? Do you equal employees to full-time-equivalents? Do you look at the beginning of the period, the end of the period, or an average? Do you consider worldwide revenue or only the revenue in the respective country? How do you transform this data into an information about size? Do you only take one proxy or a combination of several? What is your combination rule? Two out of three (one proxy to be omitted for example for companies with high revenue and NOAs, but a small number of employees). How do you combine proxies (revenue and NOA and market capitalization)? This process might also help to define the constructs more clearly. We might look at size from an economic perspective, a costumer perspective and from a self-image perspective among others. You need to define what you exactly mean and need for your research or if you are looking for information on all different aspects of the general construct. As the example about “size” might give the impression that quantitative data might have an edge on opinion-based information, let us look at different examples: “digitalization” is a multifaceted construct for which only a few accepted quantitative proxies exist, if any. Thus, information is gathered for all the different facets, usually based on opinions, and then combined. This might change in the future as the construct “digitalization” becomes clearer delineated (which would be an intellectual contribution or scientific progress). On a more abstract level, disclose the operationalization of your data retrieval (e.g., which data, from what sources, what periods, etc.). Then, show how you transform these data into proxies of a single construct and which proxies you used to represent the construct (and perhaps even which not). If you have more than one proxy for a construct, explain how you transform the proxies and how you assess the construct on the corresponding measurement scale. During the entire process, you might be prone to biases

72

4  Writing up a Research Report

that you should reflect on and evaluate their potential impact on the choices you made. Figure 4.1 summarizes this process and stresses that the constructs we use are most often not directly observable. Rather, they flow from a rather complex process requiring many decisions by the researcher. Innovation could act as a more balanced example of a construct in the sense that the options for decisions are very diverse. Information about innovation could include • • • • • • • • •

the number of patents, R&D expenditures, sales of products and services less than three years old, number of internal improvement suggestions made or implemented, time to market of products, cycle time of projects, “attitude” of employees toward change, self-assessment of R&D staff or marketing staff, and assessment of customers.

With these examples, we raise awareness that this information guideline is not a trivial matter. We provide more detailed examples in the next section. Because of the different aspects and interpretations of the constructs, it is more than a list of used constructs, but a specification of what we want to know from our sources. Depending on the research design, this information guideline may focus more on the choice of proxies and operationalization, or more on the different aspects of the constructs. The distinction between qualitative and quantitative data is very misleading because it can change during the process of data transformation. For example, one could count the use of the term “strategic” and thus turn qualitative data into quantitative data, or one could divide income into levels of above and below $40,000 and thus turn quantitative data into qualitative (categorical) data. Let us provide some more insight into the

operationalization data da

data db

data d…

data dz

transformation proxy p

i

proxy p

j

proxy p…

proxy px

proxy py

transformation assessment construct C measurement scale

Fig. 4.1  Constructs as the result of operationalization, transformation, and assessment

4.4  Sections of the Research Report in Detail

73

idiosyncrasies of collecting quantitative and qualitative data. Of course, if you convert the data type, all the above issues may become relevant. Proxies used can always be argued based on the literature review: you decide to follow examples from literature, or you change something as part of your intellectual contribution. You need to operationalize the data collection by clearly stating retrieval, allocation, transformation, and calculation procedures. What data did you retrieve from where in which format or dimension (e.g., number of employees, at the end of the period, collected as headcount)? However, is the end of a period for all elements the same (i.e., end of the year) or is it the end of the reporting period? Which data did you allocate or group together into data sets? For example, do you group headcount at the end of period 202X into the data set of 202X or of 202Y? Figure 4.2 illustrates this (rather simple) example. This example focused on the decision what data to select as the single proxy for the construct. It gets a bit more complex if you have not only to choose the data, but you also need to transform it to develop the proxy. How do you transform data? For example, if a woman is elected to the Board of Directors at the general assembly in May and starts in July, is she on the board for this period or not? Do you collect data on whether women are on the board (yes or no), or number of women on the board (does newly elected count as 0.5?), or months on the board divided by twelve? What happens with resignations and elections that also change the total number of board members? Unexperienced researchers may assume that these challenges do not happen with (pure) quantitative data. But this is not true. You often must calculate proxies based on

Fig. 4.2   From data to proxy to construct (example)

data da

proxy pi

construct a

74

4  Writing up a Research Report

retrieved data (or if you retrieve it as data ask yourself how it has been calculated). For example, how do you calculate return on equity? Is it profit divided by average equity, initial equity, final equity (including period profit)? Or how do you determine (i.e., calculate) economic value added (EVA) or Tobin’s q? How do you handle EVA conversions and which? Figure 4.3 stresses that proxies are often derived from retrieved data. You must explain this transformation process. It makes sense to distinguish between directly retrieved data used as proxies and calculated and transformed ones. The latter might also be included in the data analysis part as it is not clearly defined whether the transformation process is part of data collection or data analysis. Going one step further, constructs can be represented by a combination of proxies. Often, they gain substance and comprehensiveness if they are not represented by just one proxy. But then we must disclose and reason the transformation process achieving the combination. We must consider the measurement scale, especially if the proxies themselves use different scales. For example, company size in period x is depicted with three proxies: • market capitalization by end of June, • revenue of period, and • average number of full-time-equivalents (FTE’s) in period. The construct company size then is assessed by the following assessment rule. • company counts as big, if 2 out of three proxies surpass these thresholds [to be specified] for proxies 1, 2 and 3, • company counts as small, if 2 out of three proxies do not surpass these above thresholds for proxies 1, 2 and 3 (see also Fig. 4.4). Fig. 4.3   Operationalized data is transformed to generate a proxy

data da

data db

proxy pi

data dc

4.4  Sections of the Research Report in Detail

75

Please note that the definition of establishing the threshold is based on an assessment of counts (“big” and “small”). This addresses the same issue as in the example above about the newly elected board members: are they counted in the period they got elected? Another issue is that we indeed gained a better specified construct. However, we lost information because of the change in the measurement scale and our inability to combine the proxies. Here, we face a trade-off between construct validity and the precision of measurement (of a single proxy). Researchers do often not realize that these issues refer also to quantitative data, proxies, and constructs. We like to stress that the same issues are equally relevant for qualitative data. Table 4.2 shows an example of such an operationalization. The example above does not depict the content of the interview but represents the operationalization of its retrieval. The content plus the operationalized retrieval results in a data set. The transformation of this data set, the biases in the collected data, and the transformation process needs also to be disclosed. Referring to the example in Table 4.2, this might look like: • facial expressions recorded but not included in transcription nor assessed, • body language not observable due to Zoom session and camera perspective, • speech patterns and idiosyncrasies (“ah, …”, pauses, etc.) recorded but not included in transcription nor assessed, • no objective data on success collected, and • Mr. and Ms. Miller oversaw the market introduction: potential self-confirmation bias.

Fig. 4.4   Proxies are transformed and assessed to represent the construct

proxy pi

proxy pj

construct C on measurement scale

proxy pk

76

4  Writing up a Research Report

Table 4.2  Example of operationalization of data retrieval Operationalization dimensions (examples)

Example

Source

Mr. or Ms. H. Miller, Head of German subsidiary of ABC plc

Retrieval date

15 May 202X

Retrieval method

Unstructured interview via Zoom, camera running

Archiving and documentation method

Recording of session, transcription of oral statements

Reference of data: • time • geography or location • organization or organizational unit • etc

Refers to: • past or present or future • Germany • German subsidiary • etc

The transcript of the interview is the proxy of the interview itself. In the above example, you further analyze your transcript, not the interview. As with quantitative data, you need to describe which constructs you like to use or what kind of information you like to retrieve. We like to term this procedure as the information guideline corresponding to the well-established interview guideline (e.g., Yin, 2013). In the Information guideline, you describe what kind of information you look for and you gather ideas, where and how to collect them. For a primary data collection, the actual questions asked are an important operationalization of the information guideline. The more structured the data collection is, the more important become the actual questions asked. For unstructured data collection, these questions are only the starting point for the researcher that will be further elaborated on. This information needs to be included in the main report, not in the appendix. Data analysis Method  Specify the method you use to analyze the data. Do not explain this method to the reader, as you can assume that the reader is familiar with these methods. Make sure to reason that the data analysis method is consistent with the research aim, research question, and data collection. This is an especially important test for students. Experienced researchers usually aim for the maximum intellectual contribution they can accomplish. However, students often struggle with the data analysis method is often. This may lead to subpar choices of data analysis methods for the planned research. Particularly as an undergraduate, you face some limitations regarding the availability of methods. Your supervisors are aware of these limitations (e.g., a specific method is not taught until a later semester). We recommend discussing with your supervisors how you can tackle these discrepancies. An adequate solution is usually to state clearly that you are aware that another method of data analysis would be more appropriate, but that you

4.4  Sections of the Research Report in Detail

77

cannot use that method because of your current limitations. Of course, it would be even better if you familiarize yourself with the idiosyncrasies of the relevant method, as there exists guidance on this. You can find guidelines for beginners on how to select the appropriate method, specifically for quantitative or categorical data. You probably know this from your statistics courses. Figure 4.5 illustrates analytical methods for different types and numbers of variables. As you can see in the picture, the choice of the data type has a tremendous impact on the method of data analysis in statistics. You should also include decision criteria, if there are decisions to be made (for example the rejection of a hypothesis.) This holds especially true for qualitative data that you categorize (how to allocate data to a category?), evaluate, interpret (is this data affirmative or not?), and assess in your research project. Sample adjustments  Data collection and the chosen data analysis method often require a change of the sample. Usual reasons are insufficient quality of the data, for example incomplete data sets (missing values or answers) and outliers (data that would bias the result, if included). You must describe the reasons and the methods of sample adjustments (for example, if data is to be considered an outlier and how you handle outliers). The reasons need to be consistent with your method of analysis. For example, if you retrieve data for five periods, but you do not use those periods as control or independent

# of outcome variables

type of outcome

# of predictor variables

type of predictor

# predictor categories

same entities in each predict. category

continuous

1 categorical

2

continuous

2+

Mann-Whitney test, …

different

independent t-test

Mann-Whitney test, …

same

one-way repeated measures ANOVA

Bootstrapped ANOVA, …

different

one-way independent ANOVA

Kruskal-Wallis test, …

same

…

…

…

…

both … …

different

…

…

both

…

…

… … …

… … …

…

2+ 1

categorical

MANOVA

categorical

factorial MANOVA

both

MANCOVA

1 categorical

2+

Spearman correlation, …

paired-samples t-test

continuous categorical

Pearson correlation or regression

assumptions of linear model not met

same

3+ 1

assumptions of linear model met

continuous

2+

…

Fig. 4.5  Analytical methods for different types and numbers of variables (adopted from Field, 2020)

78

4  Writing up a Research Report

variable, it makes no sense to exclude companies for which you do not have the data for all the five years. Whereas in quantitative methods sample adjustments are quite common at a first glance, this seems not to be relevant for qualitative methods. But what do you do with data from an interview that you consider not the truth? We conclude that the same reasoning as with quantitative outliers applies. Requirements check Many data analysis methods require the data to meet certain requirements (e.g., normally distributed data, linearity, autocorrelation, homoscedasticity). You must list all the requirements and how you check them. Mixed data analysis  The method sections become rather intricate if you apply several methods. Basically, you go through all the issues mentioned for each method and source. Also, you need to describe how you combine the results of the different data analysis. After having shown the blueprint for the research, it is time to describe the results that have been produced by executing the plan. “The results of the execution of the here describe research design are depicted in the next section.”

4.4.6 Results In the result section, (all) the findings of executing the research design are depicted. If the blueprint described a wall, the results section contains a picture of the wall but with no interpretation of the wall. Purpose This section writes down the replicable results of your study with no kind of interpretation, alliterations, comments, or exclusions. You applied the method described in the method section and this is the result that you produced. If any other researcher applied the same methods, he or she should come up with the same result. How to write You can use basically the same structure as in the methods section and present the results piece by piece. The better you prepared your method section, the easier it is to write this section. If, for any reason, you deviated from or adjusted the method you had planned in the method section, you need to list the reasons and the adjustments here in the result section. Sample and data collection  You retrieved data on [dates] from [sources] by (survey, download, interview, observation, participation and got [number] of data sets. You calculated and transformed these data to generate additional data/category allocation, etc. Also, you adjusted the sample using the described methods and decision rules. This led

4.4  Sections of the Research Report in Detail

79

to the following adjustments: number of elimination of sources/data sets; number of adjustment of data from to. This resulted in a sample of [number] of data sets. You might use different adjustments to achieve different data sets for different analysis. You must explain this here, too. If you deem it appropriate, you can present a description of the data (descriptive statistics) or an example of the data sets used. Data analysis  If appropriate, you describe the results of the requirement testing for the chosen data analysis method requirement by requirement: what is the result, what is the decision criteria if the requirement has been met and the result: requirement met /not met. If requirements are not met, you can try to adjust the data set, for example using a logarithmic scale or the analysis method with less severe requirements. Of course, you must also reason this well. If all the requirements are met, you present the results of the data analysis. After the result section where you presented the bare findings, you guide the readers to the discussion section where you interpret and comment on them. For example, you may write the following transition: “having depicted the results produced by applying the research design, I will discuss the results in the following section.”

4.4.7 Discussion In the discussion section, you interpret and review the results. Here you deliberate and develop your arguments. Also, you comment on the “wall” (referring to the metaphor introduced in Sect. 4.4.6), whether you think it is stable, nice and at the right place. Purpose In this section you interpret and comment the results of the last section: what do the achieved results mean for the existing body of knowledge? In this section, you develop and formulate your argument. Here you demonstrate your scientific acumen in presenting the intellectual contribution achieved by your research. How to write Many students face problems writing this section and cannot show that they can generate an intellectual contribution. Research is not primarily about convincing your supervisor that you are familiar with so-called scientific methods (e.g., statistical methods). The discussion section provides your critical interpretation of the research results. Explain your reasoning and state your opinion. So, the use of the first person singular (if you write on your own) or plural (if you writer as a team) is perfectly fine. Questions you should address in your discussion are:

80

4  Writing up a Research Report

• what do your results mean? Explain and asses your results. For example, an adjusted R2 of 5% does mean what? First, you can explain 5% of the variation of the dependent variable by the independent variables in your model. This also means you cannot explain 95% variation. Second, you can conclude the following about your research question: despite being significant and having to reject the H1 Hypothesis, the explanatory power of the model is very low. • have your decisions paid off? You have included/excluded constructs, proxies, operationalization, used sources, and so on. Reflect critically whether any of those decisions added to the body of knowledge and seems reasonable with hindsight. You specifically looked for artifacts to understand the company culture. Did this yield insight that you would have missed if you had foregone this source? You use another proxy for capturing “size”. What was the impact of this change? • what went well and what did not? Did you detect a game changing information in the last interview and you had no possibility to re-interview the 15 former interviewees? Does this affect the validity of the data gathered? Did the sequencing of the questions in your survey cause an anchoring bias? Did some participants misunderstand question “5”? Did you get in touch with person X from whom you looked for confirmation about Y’s statement? Did you run into problems while transforming the data? Did you categorize your data into rather arbitrary groups? (E.g., you divided into below average and above average data, but 30% of your data is between ±5%, leading to data sets that are very close to your categorization criterion but are allocated to different categories). • what would you do differently next time? This can also be grouped with other ideas about future research. • are the results in line with the empirical evidence that you have reviewed in your literature review? Have you expected them or not? If your literature review showed a diverse picture with inconclusive findings, you can compare the literature findings with your results. You might have some ideas why your findings corroborate some aspects of the previous research, but conflict with other one. This might be in form a theory (refinement) or reasonable speculation. Both might be a good input for future research. • are the results in line with existing theories that you addressed in your theoretical background? Can you refine the theory based on your results? Can you resolve conflicting findings? Basically, you can apply the same comparison that you did in your literature review. • are there alternative explanations for your results? Have you found unaddressed biases responsible for your result? Have you used constructs that are not unambiguously defined? Are the proxies you used good representations of the constructs or do they (additionally) represent other constructs? Have you overlooked other constructs affecting the included constructs directly or indirectly? Ideally, you would have thought of and resolved all these issues by designing your study, but that is just unrealistic. There is a huge grey shaded area between the two extremes “blatant design

4.4  Sections of the Research Report in Detail

• • • • • •

81

error” and “strike of creative genius”. Even if you made a “blatant design error”, those are much easier to detect in hindsight. It is preferable to realize and concede that your design had some flaws or room for improvement than to hope that other researchers or your supervisors do not detect these issues. Not everything can be realized ex ante. Creative alternative approaches and insights might be generated by the research process. Here you can present those insights. does your research explain empirical evidence detected in other studies? what are the limitations of your study? Which are based on your design decisions and which are based on the results or the empirical evidence? what is the answer to your research question? have you accomplished your research aim? What is the argument that you can make? And what argument are you not able to make? what parts of the research gap did you close, partially close, and which are still open? Has your research revealed new and additional gaps? what do you recommend for future research? Based on the discussion of your results, you conclude what arguments your research produced. Every interpretation and every comment you judged as “this might be”, is potentially relevant for future research.

Here is a structure for your discussion that might also help you organize your thoughts: • • • •

the discussion draws on your results: the results have shown…, your interpretation of the results: this means…, alternative explanations: these results generated might be the effect of…, discussion and problems of the choices about methods and processes used: I encountered the following problems…, and • future research: future research topics might be… After having discussed your findings and arguments, you can draw your conclusion from these arguments and literally conclude your report. Guide your readers to this conclusion in the conclusion section.

4.4.8 Conclusion In the conclusion section, you draw your conclusion based on the arguments you produced. The conclusion section finishes your report. Purpose The purpose of the conclusion is to present the bottom line of your research: the argument you concluded based on your research design, the results, and their discussion. You started your research to make an argument. The argument that you can make based

4  Writing up a Research Report

82

on your research is the conclusion of your research. This represents your intellectual contribution. How to write Repeat the research aim of your research and the argument you planned to develop. Contrast this argument with the actual argument you could accomplish based on your research. Briefly delineate the major deviations and what lacks to make the desired argument. Remember that in scientific publications nothing should come as a surprise. So, do not include new information or thoughts in your conclusion. Everything needs to be already presented and argued for in the discussion section. In this sense, the conclusion must be repetitive (remember, the argument is also part of the management summary). This is the conclusion of your research in the literal meaning of the word. Without stating a conclusion, the argument underpinned with many thoughts that you contribute to the academic discussion misses an ultimate point. Now, you have (almost) finished your report. It provides a comprehensive account of your research that encompasses the discussion of the results and the conclusions drawn. Your report only lacks the required indexes (e.g., bibliography and appendices). We do not further expand on the formal requirements for a research report and refer to the specialized literature (see “recommended literature”). Key Aspects to Remember

Nothing should come as a surprise to the reader You do not write a drama or a crime story. There is no building of suspense. This does not mean that academic texts need to be boring, but the interest stems from the research question and how it gets answered. The reader is interested in your conclusion and process of argumentation. So basically, nothing should come as a surprise to the reader, but everything should be explained: why you are doing what you are doing, where you are in your report, and what comes next. “No surprises” also holds true to your discussion of arguments. Do not explain to the reader that your findings are in line with the findings of researcher Jane or John Doe if this study drops out of the blue. You need to mention it in your literature review first. Your opinion is a crucial part of the intellectual contribution You are the researcher. It is your research project. You conduct a research project and decide along the way. It is not only perfectly legit to state your opinions but also a necessity and a major part of your intellectual contribution. The only thing to remember is that it needs to be abundantly clear what somebody else did, found out, thought, etc. and what you did, found out, thought, decided, etc. The critical reflection of other’s research studies and your own findings in the light of what other researchers found out is

4.4  Sections of the Research Report in Detail

83

an important and integral part of your report. Without that, your intellectual contribution would be critically diminished. Understand the importance of an excellent introduction The purpose of the introduction is, first, to catch the reader's interest! You must explain why this topic is relevant to you, as well as to the reader. Why is it important? How did you come up with this research? It also shows the scope and direction of the paper, acting as a user's guide for the reader. So, it should delineate the entire story you are going to tell in the body of your research. You should describe how you interpret and approach the topic at hand. As your report is a research paper, it is perfectly alright, even very helpful to state your research question and research objective in the introduction, so that everything that follows can be understood in the light of that research question. It should also indicate your type of conclusion and point of view. Put enough emphasis on the discussion chapter In this very important section you interpret and comment the results of the results’ section: what do the achieved results mean for the existing body of knowledge? In this section, you develop and formulate your argument. Here you demonstrate your scientific acumen in presenting the intellectual contribution achieved by your research. For supervisors, this chapter is very important, as they can quickly assess your scientific skills. Our recommendation is to spend enough time on this chapter!

Critical Thinking Questions

1. Why is it important that nothing comes as a surprise to the reader in a research project? 2. What is the difference between a plain summary of your project and a management summary? 3. What are the major differences between the sections “results” and “discussion”? 4. What are the main purposes of a literature review? 5. What would you consider the most important aspects of “scientific writing”?

Recommendations for further Readings

Field, A. (2016). An adventure in statistics. The reality enigma. Los Angeles: SAGE. Field, A. (2020). Discovering statistics using IBM SPSS statistics. 5th ed. London: SAGE. Howe, S. & Henriksson, K. (2007). Phrasebook for writing papers and research in English. Over 5000 words and phrases to help you write at university and research level in English. 4th edition. Cambridge: EnglishforResearch.com.

84

4  Writing up a Research Report

Miles, M. B., Huberman, A. M. & Saldana, J. (2014). Qualitative data analysis – a methods sourcebook. Los Angeles: SAGE. Yin, R. K. (2013). Case study research: Design and methods (5th ed.). Thousand Oaks: SAGE Publications. Barros, L. O. (2016). The only academic phrasebook you’ll ever need. Createspace Independent Publishing Platform. Morley, J. (2020). Academic Phrasebank: An academic writing resource for students and researchers. The university of Manchester.

References Barros, L. O. (2016). The only academic phrasebook you’ll ever need. Createspace Independent Publishing Platform. Field, A. (2016). An adventure in statistics. The reality enigma. SAGE. Field, A. (2020). Discovering statistics using IBM SPSS statistics (5th ed.). SAGE. Früh, M., Keimer, I., & Blankenagel, M. (2019). The impact of Balanced Scorecard excellence on shareholder returns. IFZ Working Paper No. 0003/2019. Retrieved June 09, 2021, from https:// zenodo.org/record/2571603#.YMDUafkzZaQ. Yin, R. K. (2013). Case study research: Design and methods (5th ed.). SAGE.

5

Comparing Types of Research Designs

Learning Objectives

When you have finished studying this chapter, you will be able to: • differentiate between research design and research method • understand the role of mixed methods and staggered approaches in the research process • differentiate between research and consulting projects • understand the general purpose of research design • compare different research designs commonly used in business and management

In Chap. 3, the starting point of our thoughts has been the conclusion we would like to draw from our research. We then followed up with the specific steps to produce this conclusion. We believe this stepwise research process is helpful to better understand and develop research designs, but it lacks guidance for the (unexperienced) researcher. Many researchers may immediately ask “what are my options for each decision” and “what options match my decisions best”? In this chapter, we start the other way round. We provide an overview of commonly used research designs in business and management. As you see, in business and management, the range of research designs is broader than initially assumed. This holds true especially if you consider research designs in adjacent fields like the social sciences. Most textbooks on research in business and management neglect research designs that are primarily used in other disciplines. We first summarize our thoughts regarding the comparison of each research design. Then, we provide questions that guide you to the type of research design that meets your research interests. We conclude this chapter with © The Author(s), under exclusive license to Springer Fachmedien Wiesbaden GmbH, part of Springer Nature 2021 S. Hunziker and M. Blankenagel, Research Design in Business and Management, https://doi.org/10.1007/978-3-658-34357-6_5

85

86

5  Comparing Types of Research Designs

a synopsis of the characteristics of each research design covered in this textbook: design science research, action research, single case research, multiple case research, crosssectional research, longitudinal research, experimental research, and literature review research.

5.1 Recapitulation of Purpose and Content of Research Designs You need to choose the research design that is suitable to generate the envisioned conclusions you like to draw. Remember that is crucial to think of the research question and figure out what data or evidence you would like to present from your research. Also, it is important to think about the resources that you have at your disposal (e.g., time, access to data, costs). As you have learned, research is a systematic, careful, and rigorous way of solving actual problems and creating new intellectual contributions. Remember that we defined research as a systematic process of discovery and advancement of human knowledge. This knowledge should be capable of solving problems and thus contribute to the existing body of knowledge (Gratton & Jones, 2010). What does research distinguish from consulting? Research must be systematic, methodically correct, and needs to meet relevant requirements and standards for internal (sometimes also external) validity and conclusion validity. Or research design deals with the aims, uses, purposes, intentions and plans within the constraints of time, money, location, and the researcher’s availability (Hakim, 2000). So, these aspects must also be considered when choosing your research design. The following chapters cover different research designs that offer implementation plans for your research project. Note that the choice of the suggested research designs has been made in the domain of business and management research. Also, the authors of this textbook have selected and developed the designs based on their experience of supervising many undergraduate and graduate research projects. Thus, you will find other research designs in the literature. As the term research design has not yet reached any consensus in the literature, it may be the case that other authors define it differently. For example, some authors equal research methods with research designs. For our textbook, a research design serves as the major plan of how to reach the desired (type of) conclusion by answering your research question(s). Thus, a research design refers to the strategy that you develop to integrate the different components of a research project in a coherent way (Trochim et al., 2015). In this sense, research design determines what data is required, what methods must be used to collect and analyze the data, and how your research answers the research questions. As Jongbo (2014) correctly suggests, a researcher collecting data before thinking deeply through the different research design aspects, the conclusions drawn from the research will be most likely false and unconvincing.

5.2  Mixed Methods and Staggered Research Designs

87

Use the resources effectively and efficiently. A well-developed research design also serves this purpose. It ensures that the research process produces the answers that you look for, and the research process focuses on the important aspects to achieve an intellectual contribution. We observe that our students focus on describing what they plan to do about data collection and data analysis. Yet, they often fail to phrase proper research questions and the (type of) conclusion they like to draw from their research. For some type of conclusions, more than just one research design might qualify. The choices about the design you make depend also on your priorities in the research and usually involve tradeoffs. A research design that is ideal in one area (e.g., internal validity) might be weaker in another (e.g., representativeness) and vice versa. Based on the content and purpose of research designs, we characterize each type of research design about these criteria: • • • • • • •

type of conclusion drawn, adding intellectual contribution to the body of knowledge, achieved by arguments made, based on results generated, by which data analysis methods, applied to which data, and answering the research question.

Before we use these criteria to characterize the different research designs, we need to address a misconception about research designs and methods, especially regarding the so-called mixed methods.

5.2 Mixed Methods and Staggered Research Designs In the literature, research designs are (unfortunately) often divided into three major categories: qualitative, quantitative, and so-called mixed methods design. Often, there is a consensus in the literature that the researcher must choose the most appropriate “design” that best supports the type of research project (Asenahabin, 2019). However, as already explained in Chap. 1, we do not follow this method-based classification of research designs, as that classification caused a lot of confusion, as every research design according to our definition can comprise a combination of qualitative and quantitative methods. Because this trichotomy is so popular in the research literature, we briefly discuss this in the following. Specifically, we believe that a mixed method research project is not a separate research design (based on our definition of research design). It only means that researchers integrate qualitative and quantitative research methods into one research project. For example, Burke-Johnson et al. (2007) state that mixed methods are an empirical research approach in which a researcher combines quantitative and qualitative methods to increase the breadth and depth of understanding and corroboration. Indeed,

88

5  Comparing Types of Research Designs

researchers are asked to combine different methods and data if this adds to the reliability and validity of their research. Yet, we consider this view rather as a basic principle of excellent research than a separate research design. Thus, mixed method research is not a design, rather, it supports the idea that qualitative and quantitative designs have some drawbacks Thus, combining both may reduce the weakness of each other. In mixed methods approaches, we need to introduce another very popular term in the research literature called triangulation. Basically, a triangulation is a methodology that focuses on collecting complementary, yet distinctly different, data on the same research topic. And then, you can combine this data for further analysis and interpretation. Triangulation offers opportunities for better convergence and corroboration of research results gained from more than one research method. Thus, triangulation basically belongs to the mixed methods, which are strictly methodologies, as they are composed of a set of methods. We can further categorize mixed methods as convergent parallel mixed methods, explanatory sequential mixed methods, and exploratory sequential mixed methods (Asenahabin, 2019; Creswell, 2014). Convergent parallel mixed methods are a kind of mixed methods approach where the researcher collects and converges both quantitative and qualitative data to offer a comprehensive investigation of the research question(s). Choosing this approach, you collect different data, followed by a comparison and analysis of the data. If you face contradictory findings, you are asked to explain them (Creswell, 2014). In contrast, explanatory sequential mixed methods are a form of mixed methods approaches that first focus quantitative methods. After having analyzed the (quantitative) results, researchers may build on these results to explain them in more detail by the support of qualitative research. The term explanatory is used here because the focus is on quantitative methods that are often used for explanatory research. Yet, the results are further challenged by qualitative data. This approach is called sequential because the first quantitative stage is followed by the qualitative stage. Clearly, this approach is quite popular in research domains that follow a dominant quantitative orientation (Creswell, 2014). Exploratory sequential mixed methods are basically nothing else than the reverse sequence from the explanatory sequential design. If you follow this approach, you first begin with qualitative research methods and analyze this data. The data are then analyzed, and the results gained from the qualitative stage can be used for the subsequent quantitative stage. The qualitative stage may develop a (quantitative) survey to identify instruments, constructs, or variables that are used in the subsequent quantitative study (Asenahabin, 2019; Creswell, 2014). Looking at the intellectual contribution generated and the different steps of scientific advancement, it is possible to link two or more research designs, either between projects or within the same project. For example, you could consider starting with a design science research project and combine this with an action research design. Or you can think of conducting a multiple case research design and use the conclusions from this design to proceed with a cross-sectional study. Many students struggle with these staggered approaches because of two reasons:

5.3  Overview of the Most Important Research Designs

89

• using two or more research designs makes it more difficult to detect and follow the inherent logic of one research design, as several issues might occur or seem to be mixed up, and • it takes significantly more time. Basically, you conduct two (or more) research projects in sequence, with some overlap about the theoretical background and literature review. If your degree program comprises several research projects, this might offer the opportunity to stagger different research designs over the course of consecutive research projects and modules. To sum up, we consider the mixed methods approach not as a particular research design, rather a methodology combining a set of different research methods but used within one research design. However, we can also go a step further and combine different research designs. Here, we call this a staggered research design.

5.3 Overview of the Most Important Research Designs As you already know, we can group research designs into different types. Each type is well suited to reach specific objectives but not others. To choose the right research design, you must think about what you like to accomplish: do you want to solve a problem, or do you want to understand something (better)? Are you interested in causeeffect relationships? Do you like to generalize your findings? There are eight different research designs that we consider appropriate, depending on your desired aim. Each can be described by its own type of conclusion, intellectual contribution, argument, results, methods, data, and research question. As we already put forward, research designs describe the way to reach an answer to research questions and to conclude with an argument about the research area that contributes to the body of knowledge. We use the term research design for both the research specific research design comprising all decisions made in your research and the different (research specific) research designs that are grouped based on similar decisions about the major choices. Prior to presenting the overview of the research designs, we provide you with some guiding questions to determine which research design might be appropriate (based on your interest in specific types of research questions). 1. do you want to create a solution for something? • are you interested in finding or developing a suitable solution for a problem? → design science research design (Chap. 6) or • are you interested in solving the problem in real life (by changing system behavior by implementing a solution)? → action research design (Chap. 7) or 2. do you want to understand something (better)?

90

5  Comparing Types of Research Designs

• are you interested in consolidating the existing research and integrating the knowledge into refined theories? → literature review design (Chap. 13) or • are you interested in understanding cause-effect relationships? – do you want to proof or at least infer a cause-effect relationship? 1. can you control all elements? → experimental research design (Chap. 12) or 2. are you not able to control all elements? → longitudinal research design (Chap. 11) – do you want to detect and understand specific causes and effects in specific circumstances? → single case research design (Chap. 8) or • are you more interested in detecting and refining types of elements and their relationships? – are you more interested in general statements about all elements and relationships of the same type? → cross-sectional research design (Chap. 10) or – are you more interested in specific elements, their characteristics, and their relationships? → single case research design (Chap. 8) or – are you more interested in grouping elements and their relationships to establish types of elements and relationships? → multiple case research design (Chap. 9) or – are you more interested in changes of types of elements, circumstances, and their relationships over time? → longitudinal research design (Chap. 11) Tables 5.1 and 5.2 summarize these research designs which simultaneously represent the most important research designs in business in management. We cover them in more detail in the subsequent chapters. Keeping our thoughts about mixed methods in mind, it should come as no surprise that you do not find any research designs labeled mixed methods or similar in this overview. Whereas the different types of research designs are suited to draw different types of conclusions, their results can be used to create, elaborate, and confirm theory. The intellectual contribution depends on the already existing theory and the research’s results. To facilitate orientation, we depicted each research design’s most common intellectual contribution in italics. In the subsequent chapters, we describe each of the above presented research designs. Each research design is embedded in the research process (see Chap. 3) and its underlying logic. Thus, we only point out specific characteristics of the research design presented without repeating the general comments made in the previous chapters. For example, a unique and important characteristic is the type of research question that can be addressed with each specific research design. At the end of each research design, we address adjacent research designs to give you directions for refining your choice.

5.3  Overview of the Most Important Research Designs

91

Key Aspects to Remember

Do not confuse research method with research design Research designs are (unfortunately) often divided into three major categories: qualitative, quantitative, and so-called mixed methods design. Often, the term research method is interchangeably used with research design. However, we do not follow this classification of research designs that are primarily based on research methods. This is rather confusing, as every research design can comprise a combination of qualitative and quantitative methods. Do not confuse research with consulting We define research as a systematic process of discovery and advancement of human knowledge. This knowledge should be capable of solving problems and thus contribute to the existing body of knowledge. What research distinguishes from consulting is that is must be systematic and methodical in its approach and needs to meet relevant requirements and standards for internal (sometimes also external) validity and conclusion validity. To put it differently, research design deals with the aims, uses, purposes, intentions and plans within the constraints of time, money, location, and the researcher’s availability. This differs significantly from consulting. Understand the purpose of mixed-methods research We think that a mixed method research project is not a separate research design (based on our definition of research design). It only means that researchers integrate qualitative and quantitative research methods in one research project. Mixed methods are an empirical research approach in which a researcher combines quantitative and qualitative methods to increase the breadth and depth of understanding and corroboration. Indeed, researchers are asked to combine different methods and data if this adds to the reliability and validity of their research. Yet, we consider this view rather as a basic principle of excellent research than a separate research design. Think of your research goal first, then select an appropriate research design Research designs describe the way to reach an answer to research questions and to conclude with an argument about the research area that contributes to the body of knowledge. Each research design is well suited to reach specific objectives, but not others. To select the proper research design, you must think first about what you like to accomplish: do you like to solve a problem, or do you want to understand something better? Are you interested in cause-effect relationships? Do you want to generalize your findings to a broader population? We introduce eight different research designs that will be appropriate depending on your desired aim. Each can be described by its own type of conclusion, intellectual contribution, argument, results, methods, data, and research question.

 uitability of solution based • In collaboration with S on system members a solution • Characterization of probwas (developed), selected, lem and conditions, and implemented, • Criteria for evaluation • this changed the system • Available options for behavior solution, • theory

Arguments that defend the conclusion

Based on the rich, comprehensive data gathered these elements, relationships, and conditions are important to describe and understand the case

• Change that improves • Finding new or refined system behavior relevant (causal) rela• Criteria for change tionships, elements, and evaluation conditions • Solution, problem, and • Comparing with existing condition characteristics theory • Comparing with existing • Creating, elaborating, theory confirming theory • Creating, elaborating, confirming theory

• Finding solution • Developing criteria for evaluation • Finding problem and condition characteristics • Comparing with existing theory •C  reating, elaborating, confirming theory

Intellectual contribution offered by the conclusion

Comprehensive description All elements, (causal) relationships and conditions that • Describe the case comprehensively • Meet existing theory • Are not yet included in current theory

Change • Change in the (behavior of) system • Effected by (development and) implementation of a solution • Factors important to (solution specific) change management

Solution • To a problem under specific conditions, • In line (or not) with existing theory, • Applicable to problems and conditions with these characteristics

Single case

Action research

Type of Conclusion drawn

Design science

Table 5.1  Overview of distinct research designs (part I)

(continued)

Similarities and differences between cases can be explained by these categories of • Elements • Relationships • Conditions

• Finding important categories of relationships, elements, and conditions • Comparing with existing theory • Creating, elaborating, confirming theory • Establishing testable theory

Comparison Similarities and differences between cases in • (Categories of) elements, relationships, and conditions • Grouping of cases • Meet existing theory • Are not yet included in current theory

Multiple case

92 5  Comparing Types of Research Designs

About • Problem • Condition • Criteria • Solutions

What is the solution for this problem under these conditions?

Data to be analyzed

Research question the research answers

Comprehensive data about the case • Multiple data types • Varied data • From multiple sources • Until no new information is retrieved

• Analyzing and triangulating varied data

Data about the cases • As in single case • Focused on the categories already established or assumed • Additional if case cannot be explained by these categories

• Categorizing • Clustering • Comparison

Complete case descriptions using categories of these • Elements • Relationships • Conditions

Multiple case

How can we change the How and why does this sys- What are the similarities behavior of the organization tem behave in this way? and differences of cases about [problem]? that might explain variation in the phenomenon of interest?

About • Problem • Perception of the problem • The conditions • The members as part of the, condition • Preferences of the members • System behavior before and after

• Varying • Participation in system • Integrative (for evaluation) • I nput in cycles (try and error) • Reflection on resulted change

Method of analysis to achieve results

Single case

Processes (how) and reasons Comprehensive, rich case (why) this solution was description • Developed • Selected • Implemented • Changed the system behavior

Action research

• Characterization of problem and condition • Criteria for evaluation • Available options for solution • Evaluation of solution

Design science

Results that allow making the arguments

Table 5.1   (continued)

5.3  Overview of the Most Important Research Designs 93

Correlation generalizable non-random correlations or differences between specific elements under certain conditions

• Inclusion or exclusion or categorization of elements (and their effect sizes), conditions, and (directed) relationships (and their significance) into theory representing models (and their explanatory power) • Comparing with existing theory creating, elaborating, confirming theory

The likelihood of observing the data if the relationships between the elements under the conditions do not exist, is very low

Type of Conclusion drawn

Intellectual contribution offered by the conclusion

Arguments that defend the conclusion

Cross-Sectional

Table 5.2  Overview of distinct research designs (part II)

The likelihood of observing the data if the relationships between the elements under the conditions do not exist, is very low

Like in cross-sectional research with the addition of: • Time • Sequence • (Potential) impact into the theory representing models • Comparing with existing theory • CREATING, elaborating, confirming theory

Development generalizable non-random development of elements under conditions over time relationships and elements (events) effecting this change

Longitudinal

• Summary of the (development of the) body of knowledge • Directing research to gaps • Creating, elaborating, confirming (integral) theory

Integration of body of knowledge into more inclusive theory this is the (development of the) body of knowledge gaps in body of knowledge to be filled with future research

Literature review

(continued)

(Statistically) controlling • These are the theofor all elements and condiries, confirmed by this tion apart from treatment, evidence there is a significant differ- • This cannot be explained ence between the treatment by existing the theories group and the control group (gap) • This integral theory can synthesize the body of knowledge

• Finding causal relationships of elements under conditions • Comparing with existing theory • Creating, elaborating, confirming theory

Cause-effect this (group of) element(s) or causes that effect under these conditions

Experimental

94 5  Comparing Types of Research Designs

Continuous and categorical Continuous and categorical data for one point in time in data for different points in complete data sets time in complete data sets

Which elements under which conditions need to be depicted in a generalizable model of a phenomenon at a single point of time?

Data to be analyzed

Research question the research answers

About • The cause, the treatment • The effect, the effected element • The two groups (descriptive) • The experiment set up

Statistical analysis of differences between treatment group and control group

• Treatment effect size • Significance of relationship of treatment • Explanatory power of treatment

Experimental

Which elements with which What impact has change of time lags under which con- X on Y? ditions need to be depicted in a generalizable model of the development of a phenomenon?

Statistical analysis of a mathematical model that include time (as a marker) or events

Statistical analysis of a mathematical model

Method of analysis to achieve results

• Time marked elements with effect sizes • Conditions • Significance of relationships of individual time marked elements • Explanatory power and significance of model

Longitudinal

• Elements with effect sizes • Conditions • Significance of relationships of individual elements • Explanatory power and significance of model

Cross-Sectional

Results that allow making the arguments

Table 5.2   (continued)

What is the body of knowledge, its gaps, and potential integral theories?

• Studies • Theories • Theories and concepts from other (adjacent) disciplines

• Categorization • Triangulation

• Categories of annotated studies • Categories of annotated existing (qualified) theories

Literature review

5.3  Overview of the Most Important Research Designs 95

96

5  Comparing Types of Research Designs

Critical Thinking Questions

1. Why is it important to differentiate between research design a research method? 2. What role do mixed methods play for the research process? 3. What do we mean by a staggered approach? 4. What challenges do you face when applying a staggered approach in your project? 5. What is the main purpose of triangulation?

Recommendations for further Readings

We refer to the main chapters on the distinct research designs. There, you will find specific literature recommendations.

References Asenahabin, B. M. (2019). Basics of research design: A guide to selecting appropriate research design. International Journal of Contemporary Applied Researches, 6(5), 76–89. Burke-Johnson, R., Onwueegbuzie, A., & Turner, L. (2007). Towards a definition of mixed methods research. Journal of Mixed Methods Research, 1(2), 112–133. Creswell, J. (2014). Research design: Qualitative, quantitative, and mixed methods approaches. SAGE. Dresch, A., Lacerda, D., & Antunes, J. (2014). Design science research: A method for science and technology advancement. Retrieved June 10, 2021, from https://www.semanticscholar.org/ paper/bf2a9807a0d9be8c5c11684786ae3129f3e8003e. Gratton, C., & Jones, I. (2010). Research methods for sports studies (2nd ed.). Routledge. Hakim, C. (2000). Research design: Successful designs in social and economic research. Routledge. Jongbo, O. C. (2014). The role of research design in a purpose driven enquiry. Review of Public Administration and Management, 3(6), 87–94. Trochim, W., Donnelly, J., & Arora, K. (2015). Research methods: The essential knowledge base. CENGAGE Learning.

6

Design Science Research Design

Learning Objectives

When you have finished studying this chapter, you will be able to: • understand that design science research creates an artifact that solves a problem. • develop a research question like “what is the best or most appropriate or sufficient solution to this problem for this organization?” • phrase a fictitious conclusion, as: a concept or product or strategy A is the best or most appropriate (for this specific organization and for this specific purpose) • define the unit of analysis properly, like one company, department, market, or a combination thereof.

Design science research

• creates an artifact that solves a problem • research question: what is the best or most appropriate or sufficient solution to this problem for this organization? • fictitious conclusion: concept, product, strategy A is the best or most appropriate (for this organization for this purpose) • unit of analysis: one company, one department, market, or combination thereof

© The Author(s), under exclusive license to Springer Fachmedien Wiesbaden GmbH, part of Springer Nature 2021 S. Hunziker and M. Blankenagel, Research Design in Business and Management, https://doi.org/10.1007/978-3-658-34357-6_6

97

98

6  Design Science Research Design

6.1 General Description of Design Science Research Design science research aims at developing or creating an artifact that solves a (practical) problem. It is an outstanding example of the normative research mentioned in Sect. 2.2. Thus, you hardly find this (very relevant, but often overlooked) research design in the empirical research oriented scientific literature. Obviously, we cannot observe things that do not exist. This means without research efforts geared toward developing new solutions and systems, only little opportunity for evaluative research would exist (Nunamaker et al., 1991). Design science research (also labeled action science, participatory action research, participatory case study) deals with the construction and evaluation of artifacts to meet organizational needs. It also contributes to theory development (intellectual contribution). Its focus lies primarily on utility. The unit of analysis of design science research can be the society, a profession, inter-organizational issues, organizational issues, project-related, group or team-related, an individual, a concept, any system, and any component of a system. As you can learn from this, the application of this research design is quite flexible about the unit of analysis. We note that management seeks to solve problems or even design and build artifacts that can be used on frequent, even daily basis. In so far, management can be understood as a design discipline (Boland, 2002; March and Storey 2008; Simon, 1996). Managers use information technology, among other resources, to develop work systems to support the achievement of organizational goals (Alter, 2003; March and Storey, 2008). A study that describes or explains a situation is not per se sufficient for (prescriptive) knowledge improvement (Dresch et al., 2015). Often, the question “so what?” remains unanswered. Design science research aims at designing and prescribing solutions to real problems that “traditional” science cannot address (Denyer et al., 2008; Pandza and Thorpe, 2010). Because it presents prescribing features and design, design science research encompasses disciplines such as medicine, engineering, and management (Denyer et al., 2008). Van Aken (2004) advocates research beyond descriptions, explanations, and predictions (cited in Dresch et al., 2015). One criticism that has been made to studies in management is that they are too focused on understanding real-life phenomena. Researchers focus too less on developing knowledge that solves practical problems (Ford et al., 2005). In business and management, design science research may be an adequate research design because it contributes to bridging the gap between theory and practice. This is because it addresses problems both of interest to professionals in organizations and to academia (Hughes et al., 2011, cited in Dresch et al., 2015). Another characteristic of design science research is that it does not seek an optimal solution, but a satisfactory solution to practical problems. This solution might also be achieved in (iterative) steps. So, instead of asking “what is the best business model for value-based healthcare in diagnostics from the point of view of pharmaceutical companies?”, you could first start with assessing viable business models, and then describing

6.1  General Description of Design Science Research

99

the deployment context for important criteria (for example, the organization of the respective health care system). Next, you could develop evaluation criteria from the point of views of the different involved parties. Eventually, you could try to find a solution to the above stated question. Each such step might encompass a sufficient scope for a bachelor’s or master’s thesis. In fact, this stepwise approach perfectly serves as an example for a staggered research design (see Sect. 5.2). Although practical solutions gained by design science research are unique and specific, they must be capable of generalization to a certain class of problems (Sein et al., 2011; Lacerda et al., 2013). This generalization essentially allows other researchers and professionals to make use of knowledge generated by this research design (Dresch et al., 2015). The construction and improvement of theories establish the intellectual contribution of design science research. Theoretical contributions based on design science research might not be the same type of grand theory proposed by traditional sciences, but middlerange theories or substantive theories. Overview

Grand theory is a term coined by the American sociologist C. Wright Mills in The Sociological Imagination (Mills, 1959). It refers to the form of highly abstract theorizing in which the formal organization and arrangement of concepts takes priority over understanding the social reality. In the view of Mills, grand theory is isolated from concrete concerns of everyday life and its variety in space and time. By the 1980s, grand theory was reformulated to include theories such as critical theory, structuralism, structural Marxism, and structuration theory. Barnes and Gregory (1997) confirmed this adaption. They said: “no matter the phenomenon investigated; it could always be slotted into a wider theoretical scheme. Nothing would be left out; everything would be explained.” In contrast, the term middle-range was introduced in sociology by Robert K. Merton in the late 1940s. The purpose is connecting high-level social theory with empirically observable aspects. As such, it stands between high-level social theory (e.g., hermeneutics) and low-level general laws or principles (e.g., stratigraphy) (Oxford, n.d.). Middle-range theory are based on an empirical phenomenon (in contrast to a broad abstract entity as a system) and abstracts from it to generate general statements that can be verified (falsified) by data (Merton, 1968).

The development of theories based on design science research can be divided into four stages (Holmström et al., 2009). These stages entail building a theory from its source to the stage of initial ideas, turning them into more simplified theories, and eventually into formal theories.

100

6  Design Science Research Design

The first stage in the theory development based on design science research is called solution incubation. This first stage creates a framework, properly representing the problem under study. From this framework, researchers can develop workable solutions to the problems under study. These suggestions, if properly formalized, enable their implementation in a pilot level (Holmström et al., 2009). The second stage is called solution refinement. At this stage, solutions previously developed are evaluated in a real environment. This is done to verify if the solution suggested by the researcher meets the criteria for an adequate solution of the problem (Holmström et al., 2009). These two first stages support the construction of a theory by conducting design science research. Also, these two stages often happen within organizations. Thus, professionals commonly contribute only to these first two stages. Yet, this kind of contribution is not considered a scientific contribution by some academics. The third stage deals with the development of theories based on design science research. It is called substantive theory or mid-range theory. This stage seeks relevance also from an academic point of view for the knowledge gained in the first two stages. In this stage, researchers carry out activities such as evaluation of the artifact from the perspective of theory rather than practice. Mid-range theories dependent on the context in which solutions have been developed and may not be general theories. Thus, a mid-range theory does not intend to be generalized to all contexts. The goal is rather to generalize theoretical concepts that can contribute to the areas of interest in specific research programs. Finally, the fourth stage of theory development corresponds to formal theories that are independent of their context. In this last stage, theoretical contribution is more important than practical relevance (Holmström et al., 2009). An important issue to be addressed in design science research is the still low implementation and impact of research results. This has been discussed for many years in journals and at conferences (Upton & Yates, 2001), but until now with little effect. Most results end up in scientific publications only and are rarely implemented in practice. If the goal of design science research is to improve design (i.e., to establish artifacts that solve problems), this research should have some effect on practice, directly or indirectly (Blessing & Chakrabarti, 2009). Literature on how to do research focuses on other disciplines and provides little support (Blessing & Chakrabarti, 2009). The artifact as the deliverable of the research project can be many things. For example, we may create or develop artifacts, as illustrated in Table 6.1. Design science research starts with the identification of a relevant, practical problem. Researchers need to understand the problem in depth to identify all its facets and interrelations (Dresch et al., 2015). Once we understand the problem, we need to identify artifacts to address such problems, and possible classes of problems (Dresch et al., 2015). By doing so, it is possible to make suggestions for future artifacts. Thus, one or more alternatives of artifacts to solve a (practical) problem should be made explicit (Manson, 2006). This step results in a set of artifacts. One of them is selected to advance to the

6.1  General Description of Design Science Research

101

Table 6.1  Deliverables of design science research (adopted from March & Smith, 1995; Rossi & Sein, 2003; Vaishnavi & Kuechler, 2008) Deliverable

Description

Constructs

The conceptual vocabulary of a domain.

Models

A set of propositions or statements expressing relationships between constructs

Methods

A set of steps used to perform a task—how-to knowledge

Instantiations

The operationalization of constructs, models, and methods

Better theories

Artifact construction as analogous to experimental natural science, coupled with reflection and abstraction.

next steps, which focus on the development of such an artifact. The suggestion step is essentially creative and somewhat subjective (Dresch et al., 2015). During the development of the artifact, researchers construct the internal environment of the artifact (Simon, 1996). To design the artifact, different approaches may be used, for example, algorithms, graphic models, models, etc. (Lacerda et al., 2013). The product of the development step will be the artifact itself at a functional state (Manson, 2006, cited in Dresch et al. 2015). Functional state means that the artifact “works” in a specific environment. It does what it is supposed to do. So, a sufficiently detailed concept for a marketing campaign or a software can be an example of a functional state. You do not have to have filmed the advertising spots or programmed the code for software. In design science research, the development of an artifact does not represent the goal of the research process. In fact, the evaluation of the artifact is crucial. Such evaluation aims to analyze how the artifact behaves in the context for which it was designed. Thus, researchers must verify to the extent possible its ability to meet the intended objectives (Dresch et al., 2015). In the ultimate stages of design science research, you need to formalize the entire research process, including its results and learnings gained from it. At this stage, the steps of the research should illustrate the conduction process and substantiating the choices made. The study should generalize the solutions gained by using the artifact for a determined class of problems (Dresch et al., 2015). Finally, dissemination of the research results is fundamental to contribute to the academic and practical communities (Peffers et al., 2007). Example

Let us look at a concrete example in the information systems domain: A design science research approach contribution requires • identification and clear description of a relevant organizational IT problem (like an interactive dashboard using data from multiple corporate and third-party databases to graphically depicted individually configured reports, with drill-down and explanatory functions),

102

6  Design Science Research Design

• demonstration that no adequate solutions exist in the extant IT knowledge base, • development and presentation of a novel IT artifact (concepts, models, methods, or instantiations) that addresses the problem, • comprehensive evaluation of the IT artifact, enabling the assessment of its utility, • articulation of the value added to the IT knowledge-base and to practice, and • declaration of the implications for IT management and practice (March & Storey, 2008). ◄

6.2 Peculiarities of Design Science Research Design In this section, we specifically address the elements that make design science a discrete research design differentiated from others. Next to the characteristics of design science, we address the main issues and decisions to be made within this research design and the major pitfalls as we perceive them from our supervision of students’ projects.

6.2.1 Characteristics of Design Science Research Here, we elaborate on the key characteristics of design science research along the steps of the research process. Conclusion The conclusion of a design science research paper is typically that “this artifact is appropriate for solving this problem under these conditions”. An artifact can be many things and needs to be specified appropriately. The type of problem and conditions should be characterized and generalized and not only be “this specific problem at company XYZ”. The second part of the conclusion is the relation of the found solution to the existing theory. Intellectual Contribution The intellectual contribution of design science research can be in creating, elaborating, or confirming theory. It can be one or (better) a combination of: • • • • • • •

potential solutions, a solution, criteria for solution evaluation, problem characteristics, conditions characteristics, comparison with existing theory, creating a new tentative theory if the problem, the conditions, solutions, and evaluation criteria cannot be (ultimately) deduced from existing theory,

6.2  Peculiarities of Design Science Research Design

103

• elaborating an existing theory by introducing relevant problem and conditions, and • confirming a theory if the solution follows from the existing theory. Argument The argument your research makes is based on the existing theory (if appropriate) and the data collected and analyzed for this (type of) problem. Under these (type of) conditions, this artifact is a satisfying (good or best) solution according to these evaluation criteria. Results To make this argument, develop the following research results: • a characterization of the problem (properties, traits, peculiarities, dimensions, measures, etc.), • a characterization of the conditions (properties, traits, peculiarities, dimensions, measures, etc.) und which the solution is viable, • criteria for solutions (features, properties, etc. a solution needs to have to be a solution), • options for solutions (existing solutions and their properties, artifacts that do not exist but that could be potential solutions), and • evaluation of solutions (assessment by evaluating body based on the above criteria). Methods Methods employed in design science research are multiple and varied. To generate different results, you need different methods. Many of these results are not defined regarding scope and content. An evaluation criterion could be, for example, the net present value (NPV) but another could be the opinion of the solution implementing managers. Eventually, you always need a kind of method to bring all the different results together. As the aim might be a satisfying solution, this does not mean any kind of comparison. Data You will need data about the • problem, • condition, • criteria, and • solutions. These data might be qualitative, quantitative, or both and stem from different sources.

104

6  Design Science Research Design

Research question The typical research question of a design science research is “what is a [qualifier] solution for this problem under these conditions?” For example, the qualifier might be potential, satisfying, appropriate, good, the best, or similar. The problem might be described in categories of problems (e.g., accounting system). Conditions might be described with the name of the system (e.g., company XYZ).

6.2.2 Issues to Address in Design Science Research In detailing the research design, you face many design science specific problems and decisions. We list the most important ones and the main options in the following. Problem definition As design science research solves a problem, the research process needs to start with the definition of the problem to be solved, the hypothetical conclusion being “this is the solution for your problem”. For example, a company is dissatisfied with its planning process. This makes the planning process the problem. This sounds easier than it is as this is not yet an exact definition of what the problem really is. Is the company dissatisfied with the quality of the results, the effort spent, the time consumed, the number of iterations, the participation of knowledgeable personnel, just to name a few possibilities? And who is the company? Is there a consensus within management about the problem, is it based on facts, for example benchmarks with other companies or is it just the impression a single person, often called “client” or “principal”? Thus, the problem definition not only includes the definition but also the process that led to this definition and the limitations the researcher faced while defining the problem. Ideally, the researcher collects qualitative and quantitative data and compares those with relevant benchmarks to determine what the problem is. Often, the principal defines the problem independently of the researcher. Yet, this does not inhibit the researcher from gathering additional data and from reflecting on the problem definition process. Characteristics of the solution After defining the problem, establishing what is the required level of the solution becomes a close second. Basically, this is the definition of the deliverable mentioned in the general description of this research design (see Sect. 6.1). Is the deliverable a construct, a model, a concept, a method, an instantiation, an introduced and tested instantiation, or a mid-range theory? Or to term it differently, whether the aim of the research is solution incubation, solution refinement or a substantive theory. Designing the best, most appropriate, or a sufficient artifact as a solution requires establishing criteria to evaluate the quality of the solution. These criteria might stem from a critically reflected literature review and might be collected during the data collection (e.g., from former evaluations or interviews in the solution definition process).

6.2  Peculiarities of Design Science Research Design

105

Particularly, if the sole source is a few interviews, the collected criteria need to be critically reflected and evaluated among others in the light of existing research. Basically, there are three options available: the researcher comes up with a list of (founded) criteria to evaluate the design; the client or case delivers a set of criteria that needs to be critically discussed; or the researcher establishes a list of criteria based on experience, literature review and collected data and opinions during the research and the understanding of the context. Obviously, establishing these criteria needs to be fully disclosed. Development or application of the artifact Another important clarification of problem and solution refers to the development of an artifact or the application of an artifact. So, is the development of a costing system the solution (definition of cost types, cost centers, cost objects, allocation bases, etc.), or the application and of said costing scheme to the company’s cost (i.e., calculation of the numbers: product A generated last period total cost of €, etc.)? This refers to the type of data collected and analyzed, based on the conclusion you would like to draw. So, do you want to conclude that this artifact meets the criteria in a certain way according to the people you talked to or who answered your survey? Or do you want to show (establish or calculate) the consequences or benefits of choosing this artifact? Let us look at the following example to illustrate this further. Let us assume you must design a pricing model. By doing so, will you ask people how they would do it or do you establish buying behavior, cost, and sales volumes? If you want to conclude that this pricing model generates the highest contribution margin, will you draw this conclusion because the head of sales suggested so or because you collected data to calculate and compare the contribution margins? ◄

Preselection of designs A third issue is the preselection of alternative designs. It is usually not possible or workable to design “the best” artifact, especially if you really need to prove the superiority of your design. Exemptions might be mathematical algorithms and numerical solutions under certain preconditions. Thus, the chosen design is commonly the relatively best out of several alternatives. These alternatives need to be derived from a literature review or factor analysis. Also, it needs to be clear how they became part of the “available alternatives”.

6.2.3 Major Fallacies in Conducting Design Science Research While providing guidance and support for research projects, these are the major pitfalls students encounter in their design science projects.

106

6  Design Science Research Design

Missing evaluation and evaluation criteria One of the most pertinent problems in design science research is muddy evaluation criteria or no such criteria at all, closely followed by evaluation criteria that are not derived from literature or deduction but drop out of the blue. Incomplete data collection Make sure that the data collected is comprehensive and consistent. For example, when asking people about their preferences and experiences, make sure that their evaluation criteria are disclosed as well. Otherwise, you risk misinterpreting the data. This fallacy also refers to the description of the context of problem and solution. Without those, any kind of generalization of the solution to types of problems and conditions is severely hindered. Wrong or ambiguous problem definition and expectations Another issue of this research design is that researchers may not live up to the expectations raised about the delivered solution. This holds especially true for the development versus application of the artifact. For example, if the problem is wrongly calculated prices because of traditional job-order costing schemes, then the solution is not a process costing scheme, but the new prices calculated based on a process costing scheme. Missing critical reflection In design science research you will have to make even more decisions than in other research designs. Just look at all the distinct steps you might go through. Design science research is normative research. This means that you gather not only empirical evidence to make an argument. Recommend a (practical) solution. Thus, the critical reflection of all the decisions you have made, alternatives you could have chosen but rejected and the quality of the solution itself and whether it should be implemented is a must in a design science research project. Missing intellectual contribution Delivering a solution to a problem is an important feature of design science research. But without considering the generalization potential of the solution, this could also result from a consulting project. Apart from the disclosure and discussion of pros and cons, the feedback to the body of knowledge is an important distinguishing factor. So, the question if and where the solution can be applied needs to be addressed additionally. So, generalizing from problem to classes of problems and their defining characteristics and generalizing from a single context (case) to several contexts with specific features is important. Not realizing that a business case is an artefact Often students do not realize that a business case is an artefact, and they can apply the design science research design. A business case depicts the financial consequences of choosing one alternative over another. The typical research question is “what financial

6.2  Peculiarities of Design Science Research Design

107

consequences result if you (entrepreneurial) decide for this option (and no other)?” (Taschner, 2017), translated by the authors). The focus is shifted to the financial consequences of alternatives, limiting the decision criteria. Which methods are used to calculate the financial consequences (e.g., investment analysis and NVP; integrated financial planning for various decision criteria) depends on the decision to be made? Important issues for establishing a business case are defining the decision and the alternatives. The decisions are usually one of the following. • yes or no, • A or B, and • now or later. Now or later requires the selection of discrete points in time or automated calculation (often numerical) algorithms. A particularity of a business case is that one alternative always is to keep the status quo. I.e., the alternatives need to yield financial results that are better than today’s (or projected forward) financial results. A further issue is the period for analysis. It should start with the date of the first impact of the decision. The end theoretically should include the last date affected by the decision. In practice, this is mitigated by data availability and the planning horizons of the involved parties (recipients, deciders etc.). Yet, decision specific milestones should be included if possible. Not realizing that a business plan is an artefact Often students do not realize that a business plan is also an artefact, and that they can apply the design science research design. A business plan is the written summary of a future entrepreneurial plan, including the goals and the implementation plan (Taschner, 2017). It often includes shortly described reasons for the selection of options and choices. This reasoning typically takes the form of short business cases, e.g., to justify why direct sales channels were suggested instead of indirect ones. The resulting business plan is part of the discussion (see also recommendations in Chap. 4). As the business plan with its several chapters is the designed artifact, this might lead to a rather uncommon table of contents. Typically, a business plan comprises the following sections: • executive summary, • product or service, • management team, • marketing, • business system and organization, • schedule of realization or implementation, • risk management, and • financial planning.

108

6  Design Science Research Design

This makes the “discussion chapter” unwieldy, especially if you also include other thoughts. In this case, you might develop a structure such as: • 7. discussion, • 7.1 business plan, – 7.1.1 executive summary of the business plan – 7.1.2 etc. As an advantage, you can take out the entire section “the business plan” as a stand-alone document for further use (like a documentation for potential investors). Obviously, there is a plethora of other artifacts that might be developed in a design science research project. However, with the guidance provided here, you can tackle those equally well.

6.3 Writing a Design Science Research Paper Writing a design science research paper follows the principles and structure detailed in Chap. 4. However, some important aspects regarding design science project reports are (partly) different from other reports. We address these idiosyncrasies based on the standard structure of a scientific paper. Introduction Expectation management is important in every research paper, but even more so in design science research. So, starting from an initial description of the problem, you state clearly what the deliverable (i.e., the solution or the artifact) comprises and what not. Background Substantiative theories based on solutions in specified context would be a great background if they can be found. Even better, derive general theories from mid-range theories. The context of problem and solution should be described in the background section as well. If you add a theoretical background, split this section into two separate ones. Literature review The literature review in a design science research project comprises several parts: • available designs (artefacts like concepts or products), • criteria for choosing the best or appropriate or sufficient design, and • existing studies about the application of designs.

6.3  Writing a Design Science Research Paper

109

Typical research gap The research gap can be one or a combination: • there is no already proven design that meets the company or the market, • there are no designs available (at all), and • there are no decision criteria and contextual factors for decision criteria available. If you have found none of the above stated research gaps, reconsider, whether the intellectual contribution of your envisioned research is sufficient for the research paper you plan. Typical research aim The typical aim of a design science research is solving or contributing to the solution of a (practical) problem. Typical research question Typical research questions of a design science research study are: • what is the (are the) [level of requirement] solution(s) for [object or problem description] of [client or system] based on [criteria sources] using [evaluation criteria], and • what are decision criteria to evaluate solutions for [object or problem description] of [client or system] based on [criteria sources]. We can illustrate this further. By [level of requirement], we mean the following. • possible (results in a list of basically available solutions), • workable (results in a list of solutions that would basically work for the problem or client combination), and • sufficient (results in a list of solutions that meet certain criteria, ranging potentially from zero to many), • best (results in a recommendation of a specific solution). By [object or problem description] we mean the problem to be solved by the artifact. This can basically be basically anything. Some examples might be: • • • • •

better linking employees’ behavior to company goals, allocating cost to cost objects based on causal relations, reducing time to market, focusing product and service development on market requirements, and entering the south-Asian market.

110

6  Design Science Research Design

By [client or system], we mean for example: • company, • department in a company, • industry, • country, • product, or • a combination thereof. By [criteria sources] we mean the following: • theoretical (based on literature), • practical (based on opinions of participants, or • a combination thereof. Example for [evaluation criteria] may be: • performance criteria – profit, – market share, – cycle time, – discounted cash flow, – risk, and – etc. • assessment of several criteria – performance criteria, – time (elapsed, duration until implemented, etc., – resource requirements – longevity, and – acceptance by participants. • combined assessment of several criteria – combining several criteria in a utility value analysis, and – etc. Methods In the method section, you cover, several steps that might require very different methods, depending on the desired solution. • problem definition. Data collection often includes interviews and company documents (or company databases). Data analysis might range from qualitative methods to mixed methods.

6.3  Writing a Design Science Research Paper

111

• establish evaluation criteria. Data collection often includes interviews and documents, especially literature analysis. Data analysis might range from qualitative methods to mixed methods. • preselection of solutions. This might be, for example, criteria based, involves the application of creativity techniques, interviews of organization members, and experts. This step is the most subjective one. Hence, full disclosure of the methods used is a must. The selection process has some similarities to the sampling process described for case studies in Sect. 8.2.2. • Developing one or several potential solutions. Depending on the preselection, one or several solutions can be fully developed. The methods used depend very much on the artifact to be created and the characteristics of the solution. It is important to realize that these methods refer to designing an artifact, not (only) to collecting and analyzing empirical evidence in the scientific process. Examples might be a process description, organization chart, mathematical formula, algorithms, software code, etc. • Evaluating the developed artifact(s). The result of the development needs to be evaluated based on criteria. This can be done by the researcher, involved parties, focus groups or a mix, using qualitative or quantitative data. Depending on the method, the criteria are given, as they have been already established or available for adjustments. If the application is a major part of the solution and the quality of the artifact(s) can only be assessed after applying it (them), this step might be skipped. • If the solution requires the application of the artifact, then this application is simultaneously the method used. • Evaluation of the artifact and its application. The results of the development and the application need to be evaluated based on criteria. This can be done by the researcher, involved parties, focus groups or a mix, using qualitative and (or) quantitative data. Depending on the method the criteria are given, as they have been already established, or available for adjustments. • Generalization of the solution. Results The results comprise (depending on the defined deliverables): • • • • • • • •

definition of the problem, evaluation criteria (pre-solution and post-solution), preselection of possible solutions, developed artifact(s), evaluation of artifact(s), results of application of artifact(s), evaluation of the application, and generalization of the solution: mid-range theory including specification of classes of problems and contexts.

112

6  Design Science Research Design

Discussion The discussion should include (besides the generic issues mentioned in Sect. 4.4.7) a critical reflection of the problem definition, the evaluation criteria, the preselection of the solutions and the created artifact. The critical reflection should also encompass the application and the results of the application (i.e., what does this mean for the organization?) if the application is one deliverable. You need to address limitations and weaknesses of the solution. As the discussion should lead to a recommendation of how to proceed, with one extreme being the recommendation to implement a specific solution. Other recommendations might be • to gather additional information, • to redefine the problem, or • not to implement the solution as disadvantages and downside risk would outweigh advantages an upwards potential. You can place this recommendation in a separate chapter, specifically if the research is conducted for a client. Discuss the intellectual contribution, too. The artifact itself does only add to the existing body of knowledge if at least some thoughts are presented. Can you generalize your solution or artifact? If so, for which classes of problems and which contexts? Thus, also reflect the progress about the development of a mid-range theory. This also offers relevant clues for further research (e.g., testing whether your solution applies to the assumed classes of problems and contexts). Conclusion The conclusion addresses the artifact, its application, the recommendation, and implications for generalization.

6.4 Related Research Designs In this section, we briefly describe or cross-reference research designs that are similar to the design science research design. They might partially overlap with or considered to be adjacent to design science research design or in fact be a design science research design that has its own label in the literature. If the design science research design does not fully meet your intentions and expectations, look here for direction where to continue your search. Action research design In action research (Chap. 7), the focus is on implementation and change management. The development of solutions is less important than changed system behavior. It still allows exploring, describing, explaining, and predicting phenomena in the traditional

6.4  Related Research Designs

113

sense, as opposed to design science that is based on the prescriptive design science paradigm (Romme and Georges, 2003; Van Aken, 2004). Single case research design In single case research (Chap. 8), the focus shifts away from the prescription of an artifact to understanding how and why existing artifacts and their success and failure came to be results in a case study. Such a case study might generate the understanding of the context and the problem for a consecutive design science research. Single case research still allows exploring, describing explaining, and predicting phenomena in the traditional sense as opposed to design science that is based on the design science paradigm (Romme and Georges, 2003; Van Aken, 2004). Action design research Action design research (e.g., March & Smith, 1995; Sein et al., 2011) as a combination of action research and design science research follows through with implementing the designed concepts and evaluates the results of their implementation. Even adjustments to the original design might be included as well as its implementation. Often, the design part of the action design research is less comprehensive and more collaborative than in design science research and less prone to using a stage gate model. However, we basically consider action design a staggered design. This staggered design has especially gained attention in information systems research (e.g., Ahlemann et al., 2013; Zuiderwijk et al., 2014). Key Aspects to Remember

Understand management as a design science discipline Design science research has the aim to develop or create an artifact that solves a problem. It is a splendid example of normative or prescriptive research. Thus, you will hardly find this research design in the empirical research oriented scientific literature. However, management seeks to solve problems or even design and build artifacts that can be used on frequent, even daily basis. In fact, management can be understood as a design discipline. For example, managers within organizational contexts use information technology, among other resources, to define work systems through which organizational goals are accomplished. Design science research creates artifacts Design science research deals with the construction and evaluation of artifacts. The aim is to meet organizational needs. It also contributes to theory development (intellectual contribution). Its focus lies primarily on utility. The unit of analysis of design science research can be the society, a profession, inter-organizational issues, organizational issues, project-related, group/team-related, an individual, a concept, any system, and any

114

6  Design Science Research Design

component of a system. The conclusion of a design science research paper is typically that an artifact is appropriate for solving a specific problem under specific conditions. Critical reflection of your design science research project is key In design science research you will have to make even more decisions than in other research designs. Just look at all the distinct steps you might go through. Design science research is normative research. This means that you gather not only empirical evidence to make an argument. Also, you are asked to recommend a solution. Thus, the critical reflection of all the decisions you have made, alternatives you could have chosen but rejected and the quality of the solution itself and whether it should be implemented is a must in a design science research project. Think about the intellectual contribution you like to generate Delivering a solution to a problem is significant. But without considering the generalization potential of the solution, this could also result from a consulting project. Apart from the disclosure and discussion of pros and cons the feedback to the body of knowledge is an important distinguishing factor. So, the question of if and where the solution can be applied needs to be addressed additionally. This means generalizing from problem to classes of problems and their defining characteristics as well as generalizing from a single context (case) to several contexts with specific features.

Critical Thinking Questions

1. Why is design science research considered a great example of normative research? 2. Does design science research look for optimal solutions? 3. What major challenges do you face when applying a design science research project? 4. What are the differences between action research and design science research? 5. How can design science research projects contribute to theory development? Recommendations for further Readings

If you are still unsure whether design science research design is suitable for your research project, you might find the following literature and readings helpful. • Blessing, L. T. M. & Chakrabarti, A. (2009). DRM, a Design Research Methodology. London: Springer London. Retrieved 10 May 2021 from http:// site.ebrary.com/lib/alltitles/docDetail.action?docID=10310350. • Dresch, A., Pacheco L. D., Cauchick M. & Paulo A. (2015). A Distinctive Analysis of Case Study, Action Research and Design Science Research. In RBGN, pp. 1116–1133.

References

115

• Vaishnavi, V.K. & Kuechler, W. (2008). Design Science Research Methods and Patterns. 1st Edn., Auerbach Publications, Taylor and Francis Group, Boca Raton, FL. • van Aken, J. E. (2004). Management Research Based on the Paradigm of the Design Sciences: The Quest for Field-Tested and Grounded Technological Rules. In Journal of Management Studies 41 (2), pp. 219–246.

References Ahlemann, F., Hesselmann, F., Braun, J., & Mohan, K. (2013). Exploiting IS/IT projects’ potential—Towards a design theory for benefits management. In Proceedings of 21th ECIS, Utrecht, Netherlands. Alter, S. (2003). 18 Reasons Why IT-Reliant Work Systems Should Replace ‘The IT Artifact’ as the Core Subject Matter of the IS Field. Communications of the AIS, 12, 365–394. Barnes, T. J., & Gregory, D. (1997). Grand Theory and geographical practice. In T. Barnes & D. Gregory (Eds.), Reading human geography: The poetics and politics of inquiry (pp. 85–91). Arnold. Blessing, L. T. M., & Chakrabarti, A. (2009). DRM, a design research methodology. Springer London. Retrieved May 10, 2021, from http://site.ebrary.com/lib/alltitles/docDetail. action?docID=10310350. Boland, R. J. (2002). “Design in the Punctuation of Management Action” in Managing as Designing: Creating a Vocabulary for Management Education and Research. In R. Boland (Eds.), Frontiers of Management Workshop, Weatherhead School of Management, June 14–15. Denyer, D., Tranfield, D., & van Aken, J. E. (2008). Developing design propositions through research synthesis. Organization Studies, 29(3), 393–413. Dresch, A., Lacerda, D. P. & Miguel, P. A. (2015). A distinctive analysis of case study, action research and design science research. RBGN, 1116–1133. Ford, E. W., Duncan, W. J., Bedeian, A. G., Ginter, P. M., Rousculp, M. D., & Adams, A. M. (2005). Mitigating risks, visible hands, inevitable disasters, and soft variables: Management research that matters to managers. AMP, 19(4), 24–38. Holmström, J., Ketokivi, M., & Hameri, A.-P. (2009). Bridging practice and theory: A design science approach. Decision Sciences, 40(1), 65–87. Hughes, T., Bence, D., Grisoni, L., & O’regan, N. & Wornham, D. (2011). Scholarship that matters: Academic-practitioner engagement in business and management. Academy of Management Learning & Education, 10(1), 40–57. Lacerda, D. P., Dresch, A., Proença, A., Antunes, J., & José, A. V. (2013). Design science research: Método de pesquisa para a engenharia de produção. Gest. Prod., 20(4), 741–761. Manson, N. J. (2006). Is operations research really research? ORiON, 22(2). March, S. T., & Smith, G. F. (1995). Design and natural science research on information technology. Decision Support Systems, 15(4), 251–266. March, S. T., & Storey, V. C. (2008). Design Science in the Information Systems Discipline: An Introduction to the Special Issue on Design Science Research. MIS Quarterly, 32(4), 725–730. Merton, R, K. (1968). Social theory and social structure (1968 enlarged ed.). Free Press. Mills, C. W. (1959). The sociological imagination. Oxford University Press. Nunamaker, J., Minder, C., & Purdin, T. D. M. (1991). Systems development in information systems research. Journal of Management Information Systems, 7(3), 89–106.

116

6  Design Science Research Design

Oxford Reference (n.d.). Middle range theory. Retrieved June 09, 2021, from https://www. oxfordreference.com/view/https://doi.org/10.1093/oi/authority.20110803100156350. Pandza, K., & Thorpe, R. (2010). Management as design, but what kind of design? An appraisal of the design science analogy for management. British Journal of Management, 21(1), 171–186. Peffers, K., Tuunanen, T., Rothenberger, M. A., & Chatterjee, S. (2007). A design science research methodology for information systems research. Journal of Management Information Systems, 24(3), 45–77. Romme, A., & Georges, L. (2003). Making a difference: Organization as design. Organization Science, 14(5), 558–573. Rossi, M., & Sein, M. K. (2003). Design research workshop: A proactive research approach. In 26th Information Systems Seminar, Haikko, Finland. Sein, M. K., Henfridsson, O., Purao, S., Rossi, M., & Lindgren, R. (2011). Action design research. MIS Quarterly, 35(1), 37–56. Simon, H. A. (1996). The sciences of the artificial (3rd ed.). Cambridge: MIT Press. Taschner, A. (2017). Business cases. Ein anwendungsorientierter Leitfaden (3rd ed.). Springer. Upton, N., & Yates, I. (2001). Putting design research to work. In S. Culley, A. Duffy, C. McMahon, & K. M. Wallace, (Eds.), Proceedings of the international conference on engineering design. IMechE. Vaishnavi, V. K., & Kuechler, W. (2008). Design science research methods and patterns (1st ed.). Auerbach Publications. van Aken, J. E. (2004). Management research based on the paradigm of the design sciences: The quest for field-tested and grounded technological rules. Journal of Management Studies, 41(2), 219–246. Zuiderwijk, A., Janssen, M., Choenni, S., & Meijer, R. (2014). Design principles for improving the process of publishing open data. Transforming Government: People, Process, and Policy, 8(2), 85–204.

7

Action Research Design

Learning Objectives

When you have finished studying this chapter, you will be able to: • understand the aims of action research: implement change and observing its effects • develop a research question, as: what solution to a problem will finally be implemented in a system and will change system behavior? • state a hypothetical conclusion, as: implemented change in or changed behavior of a system • define the unit of analysis properly, as: one company, department, or social system • understand that action research usually requires a strong collaboration with the system to be changed.

Action research

• implementing change and observing its effects • research question: what solution to a problem will be implemented in a system and will change system behavior? • hypothetical conclusion: implemented change in or changed behavior of a system • unit of analysis: one company or department or social system • usually requires the long-time immersion in or strong collaboration with the system to be changed

© The Author(s), under exclusive license to Springer Fachmedien Wiesbaden GmbH, part of Springer Nature 2021 S. Hunziker and M. Blankenagel, Research Design in Business and Management, https://doi.org/10.1007/978-3-658-34357-6_7

117

118

7  Action Research Design

7.1 General Description of Action Research Action research is a change-oriented approach. Its key assumption is that complex social processes can best be researched by introducing change into these processes and observing their effects (Baskerville, 2001). The fundamental basis for action research is taking actions to address organizational problems and their associated unsatisfactory conditions (e.g., Eden & Huxham, 1996; Hult & Lennung, 1980). Action research focuses on practical problems with theoretical relevance. It allows researchers to develop and test theoretical ideas on the efficacy of specific actions through interacting and intervening with practitioners in a naturalistic setting (Baskerville, 1999). Researchers identify a problem, conduct data analyses, plan actions, implement actions and finally present an evaluation of a problem (Coughlan & Coghlan, 2002; Thiollent, 2009; Cauchick, 2011). Studies that do not predominantly aim to create an artifact but to implement change may adopt action research rather than design science research (or action design research). Action research has been the subject of considerable criticism over the years. Some critics have asserted that action research produces either “research with little action or action with little research” (Dickens & Watkins, 1999, p. 131). Action research is an empirical type of work, whose conception and construction should solve a collective problem. Researchers, participants, and representatives of the situation researched, are involved in a cooperative and participatory way (Thiollent, 2009). Action research may be defined as an emergent research process in which applied behavioral science knowledge is integrated with existing organizational knowledge and applied to address real organizational issues (Shani & Coghlan, 2019). It is also concerned with bringing about change in organizations, in developing self-help competencies in organizational members, and in adding to scientific knowledge (Shani & Pasmore, 1982). Action research combines theory generation with researcher intervention to solve immediate organizational problems (Baskerville & Wood-Harper, 1998). Thus, action research aims to link theory with practice, and thinking with doing. It is typically an iterative process based on working hypotheses refined over repeated cycles of inquiry (Davison et al., 2004; Sein et al., 2011). Finally, it is an evolving process that is undertaken in a spirit of collaboration and coresearch (Shani & Coghlan, 2019). Even though action research emerged about 60 years ago, action researchers still face doubt about the scientific value of this research design (Altrichter et al., 2002). Although half a century have passed since Foster (1972) stated that “the literature on action research is not overburdened with attempts to distinguish between it and other forms of applied social research” (p. 533), this is still a problem today (Blichfeldt & Anderson, 2006). Although action research is today recognized as a viable research design (specifically in areas such as health care, information systems, and organizational development), “much of this, of course, happens outside the published literature and therefore outside the public and academic eye” (Dick, 2003, p. 258). Grønhaug and Olsson (1999) note that “there are actually only a few action researchers which have made major contributions to the scientific community.” (p. 13) (Blichfeldt & Anderson, 2006).

7.1  General Description of Action Research

119

Before we delve into the specifics about the execution of action research, we need to establish an initial researcher-organization set-up. It provides the structural foundation for such a project. It includes protocols that determine some of the basic rules of engagement for the researcher, and the roles, responsibilities, and expectations for behavior on both sides. For this set-up to work effectively, it is essential that the client organization and its key o personnel understand what action research is and how it is executed, and what the advantages and disadvantages are for the organization. The set-up should build trust among the various stakeholders. It should also establish a spirit of shared inquiry: the action researcher is not allowed to take a dominant role, nor should the clients be too passive. Everyone needs to be involved (Davison & Martinsons, 2007). Questions that might apply to foster an effective researcher-organization set-up (Davison et al., 2004): • did the researcher and the client agree action research was the approach to apply in the organizational situation? • did the client and the researcher make explicit commitments to the project before it began? • were the scope and boundaries of the project specified before it began? • were the roles and responsibilities of the researcher and client organization members specified before the project began? • were the project objectives and evaluation measures specified explicitly before the project began? • were the data collection methods specified before the project began? Action research projects follow a cyclical design, starting with diagnosis, then action planning, intervention, evaluation and finally reflection. The completion of one step leads sequentially to the next, thus helping to ensure that an action research project is rigorously conducted. As a cycle, there is the suggestion of a linear, unidirectional flow (Davies et al., 2004; Davison & Martinsons, 2007): • was there adherence to the cyclical process model or a rational justification for a deviation from it? • did the project begin with a diagnosis of the organizational situation that resulted in the factor’s identification causing the problem? • how were data collected? You can collect data in different ways depending on the context of the action research project (meeting notes, interviews, reports, statements, emails). Other data sources are informal context such as lunch breaks, coffee breaks, and leisure situations. The direct observation (e.g., leadership, power, roles, communication, and culture) is an important source in action research projects (Coughlan & Coughlan, 2002; Thiollent, 2009)

120

7  Action Research Design

• was there feedback on the data collected? Collected data is provided to all taking part in the action research project to validate the data (Coughlan & Coughlan, 2002; Thiollent, 2009). • how was data analyzed? It is important to review the data by the researcher and other people within the organization involved in the research. Clients of action research projects are assumed to know their organization and what might work for them. • was an explicit action plan formulated based upon the results of the diagnosis? Once data analysis is concluded, new actions can be planned. Researchers must decide on what will be changed in what parts of the organization. Who is supportive of this change and who might be reluctant? • were the planned actions implemented and evaluated? Participants of the action research project implement the planned actions. Evaluation is a very important step to learning from action research. The results of the action research process are reflected and though through, this holds also for unexpected results. • was there reflection on the results of the planned actions and on the value of the theoretically based model? • was this reflection followed by an explicit decision on whether to proceed through an additional process cycle? • was the exit of the researcher and the conclusion of the action research project because of the project objectives being accomplished or some other clearly articulated justification? (Davies et al., 2004). The application of relevant theory is crucial. A basic premise of action research is that action and theory should be interwoven to create results of value to both researchers and clients. Some scholars (e.g., McKay & Marshall, 2001) claim that action research without theory is not research at all. Other researchers suggest it may be counterproductive to apply theory too early (e.g., Bunning, 1995). Cunningham (1993) reinforces this, stating that “it is highly unlikely that the researcher can know definitely and in advance the exact theory that will be used or developed” (p. 61). Drawing on theories too early in the research process may lead to challenges if the data collected does not support that theory. Heller (1993) states that “there are still very many social issues for which no paradigmatic model and no appropriate evidence exists. In those circumstances, a research phase, wherever possible with the people who experience the problems, has to precede action”. This research phase may well be operationalized as a theory-free episode of action learning (Davies et al., 2004; Davison & Martinsons, 2007). The principle of change through action reflects the crucial strengths of action research (Davies et al., 2004; Davison & Martinsons, 2007): • did the researcher have an explicit motivation to effect change and improve the organizational situation? • did the client expect that the project improves the organizational situation? • were the problem and its cause(s) understood and specified because of the diagnosis?

7.2  Particularities of Action Research

121

• were the planned actions designed to address the specified problem? • was the status of the organizational situation measured before and after the intervention? • did the client approve the planned actions before they were implemented? • were the timing and the actions taken documented? This principle of action research is very important and may be a challenge to accomplish. Lau (1997) states that the most critical step in action research is considering the two-fold nature of the action researcher’s responsibilities to the research community and to practitioners: • was the change in the organizational situation assessed by the researcher and communicated to the client? • did the researcher and the client reflect upon the results of the project? • were the results considered regarding their implications for further action in this organizational situation? • were the results considered in terms of their implications for action to be taken in related research areas? • were the results considered in terms of their implications to the academic community (general knowledge, informing/re-informing theory)? • were the results considered concerning their implications for the applicability of action research (Davies et al., 2004)? An important issue in action research is that often too much emphasis lies on the practical outcomes for the companies. Yet, the discussion about the creation of new knowledge seems limited (Shani & Coghlan, 2019). Thus, more systematic rigor needs to be used such that new knowledge creation process is designed into the action research process. For example, Von Kroch et al. (2000) developed five knowledge creation steps: sharing tacit knowledge, creating concept, justifying a concept, building a prototype, and crosslevelling knowledge. Their suggestions can serve as a starting point to enhance new knowledge creation in action research projects. Also, Mohrman and Lawler (2011) suggest that development towards the creation of actionable knowledge as a welcomed outcome supports creating new knowledge (Shani & Coghlan, 2019).

7.2 Particularities of Action Research In this section, we specifically address the elements that make action research a discrete research design differentiated from others. Next to the characteristics of action research, we address the main issues, decisions to be made within this research design, and the major pitfalls.

122

7  Action Research Design

7.2.1 Characteristics of Action Research In this section, we elaborate on the key characteristics of action research along the steps of the research process. Conclusion The conclusion of an action research study typically is the change in the system behavior that has been triggered by the collective development and implementation of a solution. As action research is a change project, the originally intended change in the sense of problem solution and the emerging change might be very different. The change project has started the emerging change. In this regard, it is the trigger of the change, but possibly without intending the realized change. Thus, there is not necessarily a cause-effect relationship between the intended problem solution (if there even is one) and the change in system behavior. Intellectual Contribution The focus of the intellectual contribution of an action research design is on the changed system behavior and on the realizability of a solution. The intellectual contribution itself therefore is finding and improving an implementable solution for this system with the emphasis on one or several or even all the mentioned components. This means focus can be on the collective process of finding a solution for a problem, which includes the perception and redefinition of the so-called problem, the collective improvement of an existing solution is the sense of altering existing system behavior, the change that has been affected and the conditions that are established by the idiosyncrasies of the system, its elements, and their relationships. The last component is especially important as all characteristics of problem perception and definition and criteria for solution evaluation are rooted in the existing (social) system. This system is simultaneously the condition under which all these processes take place. Finding, describing, or even altering the characteristics of these conditions creates often also a significant intellectual contribution. You can enhance the intellectual contribution by reflecting the change process in the light of existing theory (confirmation) and by creating or elaborating on theories about the impact of types of conditions on the processes of (types of) problem perception and definition. Argument The key argument that is derived from action research is how (descriptive or exploratory) and why (explanatory) under these conditions given by the existence of the system (organization and its environment) this solution has been chosen, implemented, and affected changed system behavior.

7.2  Particularities of Action Research

123

Results The results are the description of the processes taking place while (implicitly and explicitly) perceiving and defining problems, developing, choosing, and implementing solutions and affecting changed system behavior. The description of the change processes entails the (perceived) reasons for the processes and their results. Methods The methods used to generate the results vary. They often encompass methods of qualitative data collection and analysis, especially interviews and observation. This requires usually at least partial participation in the change process and data collection over a longer period and in varying settings, for example individually prior to a workshop, observation during the workshop and individually after the workshop. An important and basically a defining meta-method employed in action research is the cyclical approach of triggering change–taking part in change or observing change– reflecting on the changed system behavior–triggering (additional) change. Methods for triggering change range from rising problem awareness to presenting solutions. Methods for reflection are predominantly the (iterative and recurring) application of the research process. Have a look at the following example of such a cycle and its steps: • individual preparation, • discussion with the principal, • workshop, • debriefing with principal, • evaluation of workshop, and • individual preparation of next step (and so on). Data You can collect and analyze qualitative and quantitative data. Data typically refer to many constructs and conditions. This makes the data collection and analysis demanding. The data used encompasses: • the problem, • the perception of the problem, • the system, conditions and participants and their cognitive processes as part of the conditions (we listed these components to make the point clearer not because the terms are mutually exclusive, as they are not.), • preferences of participants, • the perceived solutions, • the decision criteria, • the implemented solution, and • the changed system behavior.

124

7  Action Research Design

The data, especially deviations from past states, needs to be collected at least for each cycle. Research Question The typical research question of an action research design project can be formulated like this: • how to effect change in the organization? • how does change or improvement happen in the organization? • how to improve or change the behavior of the organization about [problem]?

7.2.2 Issues to Address in Action Research In detailing the research design, you face many action research specific problems and decisions. We list the most important ones and the main options you have in the following. Relationship between implementation and solution quality The general supposition of action research is that implementation of a solution takes precedence over the potential quality of the solution. So, when evaluating a solution, the dominating criteria in action research is “adopted by the organization”. Or to phrase it more colloquially: it is better to have an 80% solution that is used by the organization than a 100% solution that is not adopted and does not change its behavior. The measurement becomes actual change. Action research is the ultimate bridge between theory and practice. Thus, change management becomes a central pillar in action research, this is maybe even more important than the content or solution related knowledge. This requires a general shift of focus to the perception of the organization or system members. What they perceive consciously or subconsciously as “the problem” and “a viable solution” becomes more important than what the actual problem and a good or the best solution is. Complex social processes This makes action research a promising research design for dealing with complex social processes. This holds specifically true if it is clear from the beginning on that there is not a single best solution (maybe because there simply is none), but the challenge is to find (and implement) an acceptable one. This seems rather straightforward but is misleading: all social processes are complex. Gauging change resistance prior to introducing change is very difficult. This means that the argument and justification for your research design is the (probable) impact of the complex social processes. Thus, prior knowledge, preferably gained by self-experience, about the organization is very beneficial.

7.2  Particularities of Action Research

125

Cycling research process Action research is based on a cyclical process. Starting with diagnosis (what is the problem), then action planning (what solution should we implement), intervention (implementing the chosen solution), evaluation (how successful is the implementation and how successful is the solution) and finally reflection (of the process, the implemented change, and the potential improvement yet to be gained and the gap between desired behavior and actual behavior). This then serves as the trigger for the next cycle (Davies et al., 2004; Davison & Martinsons, 2007). To stick with the analogy from above: perhaps you only accomplished to implement a 20% solution in the first cycle. But now all the participants are better aware of the problem and potential other (better) solutions. So, in the next cycle you might achieve implementing a 60% solution. The following two issues apply to your research: • how many cycles can you include, or do you have time for in your research? • what makes up enough change and who decides about sufficiency? E.g., your client might decide that after implementing a 20% solution or improvement, the priorities have shifted. The cyclical nature is a very distinct and potentially very demanding feature in action research. If you go for action research, it is probably worthwhile to check the expectations of your supervisor. Collaboration Action research requires a lot of collaboration and interaction. Especially if you run several times through the action research cycle. This is time-consuming: working time but also elapsed time because of availability and priorities of participants. Acceptance is enhanced by participation in problem definition and solution development. Establishing common perceptions is not (only) conducted by individual interviews but also by workshops. So, workshop facilitation becomes a key ability in the research process. This does not mean that you must moderate the workshops yourself, but somebody must do it. Hence, the clear allocation of responsibilities and alignment of expectations is a must. Role of researcher and client Action research simultaneously deals with implementing change in organizations and with the relationship between the researcher and the organization and its members (Coughlan & Coghlan, 2002; Coghlan & Shani, 2005; Thiollent, 2009). Thus, action research is characterized by “the active and deliberate self-involvement of the researcher in the context of his/her investigation” (McKay & Marshall, 2001, p. 49). Baskerville and Lee (1999) suggest that such collaboration deteriorates the action researchers’ ability to control processes and outcomes and their freedom to pick problems. This clearly is a trade-off to getting the possibility for such research at all. It also diminishes the possibility for ending the project on its own (Blichfeldt & Anderson, 2006).

126

7  Action Research Design

Clearly define with the client what the job of the researcher is. This might vary. Either you are asked to find and implement the solution (or the change), or you are only an observer, with many degrees in between. Expectation management is a must for a successful action research project. Please keep also in mind that the more active your role in the change process is, the more your personal traits affect this process. For undergraduate students, their (young) age and (lack of) professional experience might pose a challenge to the change process. Intellectual contribution Action researchers are faced with the problem inherent in doing prescriptive: answering a research question and fulfilling a practical requirement (Rapoport, 1970). Implementing a solution is not a scientific endeavor. Consulting basically offers that as well. To be recognized as an action research project, several conditions about adding to the body of knowledge need to be met: • the solution needs to be derived from theory. This might be more affected by implementability than in a design science research project, but basically there should be a solution that the researcher argues to be the “right” or “planned” solution. Sometimes, there might not be enough knowledge available to come up with this, but this is the exception. • the reflection needs to encompass reasons the implemented solution differs from “planned” one. This might be reasons related to the solution or the circumstances. • the reflection needs to compare the implemented or realized solution (and its outcome) with the planned solution and its outcome). This might trigger an additional cycle where all the above stated conditions need to be met as well. • the collection of all these reflections (deviations and modifications of solutions, adaptation to which circumstances, measuring of the outcome of the solution) needs to be added to the body of knowledge, i.e., reported appropriately. Overall, action research contributes predominantly to theory confirmation. Is our theory detailed enough to explain the observed processes and phenomena? What might be elaborated in the theory to improve (guidance for) future solution finding and implementing processes?

7.3 Major Fallacies in Conducting Action Research While providing guidance and support for research projects, these are the major pitfalls students encounter in their action research projects. Initial problem definition is wrong One of the biggest issues in action research is the tentative nature of the initial problem definition. In a typical process, you will have contact with the client through the

7.3  Major Fallacies in Conducting Action Research

127

department or unit head who describes a problem from his or her perspective. You agree to start the research process based on your assumption based on the problem area that interests you and you feel comfortable to achieve a solution in collaboration with the participants. You meticulously prepare for the research process, for example by collecting an extensive body of knowledge. The collaboration starts, and you realize the problem looks completely different. As is known from coaching experience this is quite common on a personal level and might be multiplied in social settings (e.g., Bamberger, 2015). In the worst case, this leaves you handling a problem in an area you are not interested in and you know relatively little about. You have spent a lot of work in preparation of a process that now can all be abolished and took up much of your available working and elapsed time. This often leads to sticking with the original perception of the problem, and, to poor research quality. Thus, it is best to either think deeply about either cutting your losses and start afresh, basically making this loop the first (unsatisfactory) research cycle or to abandon the project. Elapsed time restrictions Planning the schedule of an action research project in advance is very demanding. Especially if availability of participants is limited and several group meetings are required. Change process results might even require a complete additional cycle. As a bachelor’s and a master’s thesis often have a time constraint, it might pose a challenge and is quite risky to align those timelines. Continued collaboration Commitment of participants might wane during the process, especially if several cycles are required. Also, here expectation management and a clear definition of the researcher’s role are necessary. But even that might not prevent the shifting of priorities on the participant’s personal level. At least motivating the participants should be part of the client’s responsibilities. Area of expertise Many researchers become frustrated within research projects using an action research design because their expertise is to a very high degree (up to almost only) related to change management and social behavior. This makes some researchers who originally stem from other areas like finance, accounting, operations research, production, uneasy. Some also lack sufficient knowledge of change management and social behavior to execute an action research project satisfyingly. They first must acquire (and train applying) a lot of new knowledge. Poor risk management Based on all the problems and fallacies stated so far and the ones to follow, action research is quite risky, especially for undergraduate and graduate research projects. This needs to be considered if you think about selecting this research design.

128

7  Action Research Design

Trial and error and client supremacy With a focus on the change process and implemented solutions, the problem arises that the solution does not get developed collaboratively, but solely by the client. Here, the intellectual contribution diminishes as one part of the intellectual advancement is whether the normative solution can be implemented or not and under which circumstances and which modifications. Then the solutions generation process dwindles down to “if the client thinks, that works, let us try it”. Benchmarking When designing solutions, arguments from client and participants often refer to benchmarks. Benchmarks (“others did this, too”) is a substantial part of their experience. Thus, it affects the process directly or indirectly by shaping participants’ input. For example, in a benchmark you find that all the Fortune 500 companies employ an incentive system. You might even find that the “best in class” has a high percentage of variable payments linked to company performance. But you should not confuse this with empirical evidence about its usefulness for solving the specific problem at hand. In the literature review, you present the (relevant) existing body of knowledge, including the empirical confirmation about the relationships of systems, situations, and problems. However, you must delineate it from these “benchmarks” (unless they are the result of scientific research). Slipping off into consulting In response to criticisms that action research and consulting are quite synonymous, some researchers looked for differences between these two disciplines (e.g., Baskerville & Wood-Harper, 1998). They conclude that, among other aspects, consultants work only for a client, whereas action researchers work for both a client and the broader research community, to which they must report their findings. In case of a confidential bachelor’s and master’s thesis, this broader community might just comprise your supervisors.) Published stories of consulting projects have claimed to be action research only because they were collaborative and followed one or more cycles of action and reflection. In this context, Schein (2010) argues action research has often been diminished by being a glib term for involving clients in research and has lost its relevance as a powerful conceptual tool for inducing truth on which action may be taken (Shani & Coghlan, 2019). Many consultants implement a standardized solution, failing to apply the contextspecific planning. A one-fits-all solution gives the consultant a high profit margin and, in most cases, satisfies some of the client’s basic requirements. Yet, it violates important principles of action research in failing to diagnose the specifics of the organizational situation. As a result, consultants typically neglect to identify all the factor(s) that, for example, cause the performance deficiencies or jeopardizing the future competitiveness of the business. The lack of a diagnosis also means that problems and their causes cannot be clearly understood and specified. This violates another of action research principle, “change through action” (Davison & Martinsons, 2007).

129

7.3  Major Fallacies in Conducting Action Research

Because of these violations, a solution may be confronted with resistance during its implementation and can be far from optimal in contributing to the improvement of organizational performance. Not reflecting on what was done and what could have been done differently is a further violation of the action research principles. Often, consulting projects lack a systematic process. This violates another key criterion of action research design to follow a clear cyclical process model. Also, consultants may neglect to evaluate the change that resulted from the intervention. This violates the principle of specifying learning in action research. Clients of consultants may not change the initial solution, and most times must use a poorly performing solution or hiring another consultant to provide another, better solution. Well-executed evaluation and reflection on the consequences of the implementation, ideally drawing on a theoretical model as introduced by another principle of action research, provides guidance for implementing or adjusting a solution (Davison & Martinsons, 2007). Table 7.1 summarizes potential failings of consulting projects in the past and relates them to specific violations of action research principles (Davison & Martinsons, 2007). The poor recognition of action research as a viable research design stems from poorly executed, poorly reflected, and poorly disseminated research projects. Poor action research and consulting look very similar, and this adds to the unwarranted disparagement of this research design. Table 7.1  Violations of action research principles in consulting projects (adapted from Davison & Martinsons, 2007) Violations

Related action research characteristic

Inadequate preparation by the client, prior to engaging consultant

Researcher-organization set-up

Inadequate involvement of the client during the consulting engagement

Researcher-organization set-up

Insufficient definition of the project, in terms of scope and boundaries

Researcher-organization set-up

Insufficient specification of project objectives and evaluation measures

Researcher-organization set-up

Inadequate diagnosis of the organizational situation to identify the factors causing the problem

Cyclical process model

Lack of theory or model that links the planned actions to specific aspects of performance, and factors causing the problem

Theory

Unclear or under specified understanding of the problem, and it causes

Change through action

Failure to evaluate the change that occurred, and the implications for further action

Cyclical process model and specifying learning

Failure to reflect on the consequences of the engagement, and the implications for further action

Cyclical process model and specifying learning

130

7  Action Research Design

7.4 Writing an Action Research Paper Writing an action research paper follows the principles and structure detailed in Chap. 4. However, some important aspects regarding action research project reports are (partly) different from other reports. We address these idiosyncrasies based on the standard structure of a scientific paper. Structure of the paper Different cycles can either be addressed per chapter or the cycles become the primary structuring criteria. This means you can–after the introduction–use the different cycles to present the specific literature review, methods, results, discussion, reflection (instead of conclusion) and then continue with the second cycle. Or you can use the standard structure. However, within the chapters you must explain the different cycles and what is referred specifically to them. Introduction In the introduction to an action research paper, explain the relationship between client and yourself as a researcher, as well as the distribution of responsibilities between the parties. Theoretical background We should base action research on extant theories. There might be one or several related theories to the problem and its solution. Also, consider rather generic ones about change management and social behavior. These theories also provide a foundation to your approach of collaboration. In specific cases, some argue that action research can or even should be conducted with no theory. Which basically means that there is no planned solution, just coaching or instructing the participants to find a solution for themselves. Without arguing against the merits of such an approach, we think it does not contribute to the body of knowledge and should not be used in scientific research. Literature review The literature review encompasses the state-of-the-art about several areas, especially studies about: • the characterization of system, situation, and problem (that might provide insight to developing the solution and implementing the change). What empirical evidence exists about the relationship of elements in the original perceived problem, and the grouping of elements and situational factors that might be relevant? • solutions of the problem. What empirical evidence exists about proposed and implemented solutions, and the usefulness of the solutions?

7.4  Writing an Action Research Paper

131

• the modification of the solution about implementation and the usefulness of the solution in practice. What empirical evidence exists about characteristics of the solution that made it unsuitable for implementation under certain circumstances, and modifications of the solution that made it better suitable for implementation or generating the desired effects? • the implementation and change management process. What empirical evidence exists about how to support and manage the change management process, how to detect and balance quality of the solution and implementability, and how to evaluate the results and when to trigger the next cycle? Another obstacle in the literature review is the potential change in the relevance of the empirical evidence for different action research cycles. In a worst-case scenario, the next cycle needs to be based on partly or even different empirical evidence (not because your literature search and review has been faulty, but because the problem definition and/or the contextual factors have changed. To disclose your mind-set as the researcher to your audience you clearly delineate which evidence you used (and added for each cycle). Typical research gap The typical research gap is the missing implementation of a curtailed solution to this specific system affecting change. Clear, detailed, and comprehensive guidelines affecting comparable systems with comparable system behavior by implementing a solution tailored to that system and its problem. Typical research aim The typical research aim of an action research project is to affect change in a system towards a (more) desired state. This desired state can be the implementation and application of different systems, tools, processes or more showing a changed system behavior. Typical research question The typical generic research question of an action research project is: • how to improve [client or system] situation by changing system behavior by [implementing] a solution for [object or problem description] We can illustrate this further. By [client or system], we mean, for example, a: • company, • department in a company, • team in a company, • industry, • country,

132

7  Action Research Design

• product, or • combination thereof. Especially departments and teams are quite common as action research systems. By [implementing] we mean the process leading to the implementation and the changed behavior related to this implementation. This process can start at varying points, e.g., with the finding of a potential solution or with a given solution. By [object or problem description] we mean the problem to be solved by the artifact. This can be anything. Some examples might be: • • • • •

better linking employees’ behavior to company goals, allocating cost to cost objects based on causal relations, reducing time to market, focusing product and service development on market requirements, or entering the South-Asian market.

By [client or system], we mean, for example: • company, • department in a company, • industry, • country, • product, or • a combination thereof. Methods The method section in action research specifically refers to scientific methods of sampling, data collection, and data analysis. It usually does not refer to workshop moderation techniques. Unfortunately, there is some overlap, as the collaborations tools employed affect the data collection. For example, you can use specific creativity techniques or discussion rules to reduce biases. Hence, a section on those methods is fine if the purpose of using them is clearly delineated. Apart from this, the methods commonly comprise qualitative data collection and data analysis methods. Because of the collaborative setting, you often have more sources available than just the usual transcripts (e.g., flip charts and pictures). This puts an additional focus on the data analysis, as the sources do not yield the same results. However, this is an important result. One conclusion of the collaborative methods employed in a cycle might be the requirement to change or adjust them. Especially if the results of the data analysis have been too divergent. Next to the usually qualitative methods used for collecting and analyzing opinions and contributions from the participants, quantitative methods might also

7.4  Writing an Action Research Paper

133

be employed. Quantitative methods may measure the effectiveness of the solution, the degree of collaboration about certain topics, or the social behavior of participants (e.g., who speaks or answers to whom, for how long, on which topics?). Results Depending on the research question and the agreement with the client, you may refer your results to three different areas: • the affected change regarding the problem – the developed / agreed on solution: what is the solution? – implementing the solution: which solution has been implemented, when, how, etc.? – the effect of applying the solution: by whom, when and to which effect is the solution being used? • the social interaction patterns and idiosyncrasies of the participants: power structure, group dynamics, idea creation patterns, openness to new ideas, solution determination, solution refinement, (hidden) agendas of participants, and so on. • the impact of the process on the implemented solution and the social behavior: what has been the impact of the different process steps about dividing time and effort between formal (i.e., planned steps) and informal ones, what has been the impact on the social behavior patterns in point two? For example, have arguments been put forward and discussed and resolved? Did participants change their behavior in the formal settings by contributing more or different issues? This result requires a baseline for comparison. If none exists and cannot be determined by participating or observing informal interactions, the first cycle does not yield these kinds of results. For a second cycle, the first cycle might be used as a baseline. Discussion The discussion addresses the interpretation of these topics: • the affected change about the problem – the developed or agreed on solution: why has this solution been chosen, which characteristics appealed to the participants, was the decision based on the merits of the solution or on the opinion of certain participants? How much does the solution deviate from the “planned” solution? Can the deviation be explained by contextual factors? What portion of the contextual factors are based in the behavior of the participants? – the implementation of the solution: to which degree has the agreed-on solution been implemented? What are reasons for deviations about scope, the solution itself, specific parts of the solution, orders, guidelines, or interpretations to or by the users? – the effects of applying the solution: did the implemented solution solve the problem? Which parts of the problem? What did it not solve? What are the most

134

7  Action Research Design

common complaints? Is the organization satisfied with the solution? was the problem addressed really the relevant one or does the problem definition now change? Has a sufficient solution been implemented, or a sufficient change realized? Is another cycle necessary (based on the perception of the problem or the gap to the planned solution?). • the social interaction patterns and idiosyncrasies of the participants like power structure, group dynamics, and so on: what do the detected group dynamics mean for this change project and future change projects? What drives change? The formal steps in the research process or the informal interactions in between? Did real discussions take place or have the steps only been formalizations of already decided on issues? Who (really) decides what? Who contributes to which issues? Does a discussion still take place after certain participants voiced theirs? Is it reduced to specific individuals? Is it a distribution of decision rights or about certain topics? Does it correspond with the formal organization structure? • the impact of the process on the implemented solution and the social behavior: what impact has the process on the solution? Have solutions been rejected based on dominant individual opinions? Did participants feel empowered to voice more or different input? What needs to change to improve the change process? Is the behavior of specific individuals addressed? What about the distinct steps in the process? What does this mean for the next cycle? Conclusion The conclusion needs to refer to the three areas (solution, change process, and social behavior) and might be specific for the different cycles, especially about the decisions to take with the client to start a next cycle.

7.5 Related Research Designs In this section, we briefly describe or cross-reference research designs that are similar to the action research design. They might partially overlap with or considered to be adjacent to action research or in fact be an action research design that has its own label in the literature. If the action research design does not fully meet your intentions and expectations, look here for direction where to continue your search. Design science research If the focus is predominantly on creating a solution (to a specific or even a class of problems) but not on implementing it, then design science research might be more appropriate (March & Smith, 1995; Collatto et al., 2018). This might be the case, when solutions parameters are yet not clearly apparent or defined based on the existing body of knowledge.

7.5  Related Research Designs

135

Both action research and design science research make an intellectual contribution by adding to the body of knowledge and solve current or expected problems of practitioners. They both improve or support existing organizational activities and processes and intervene in reality (Rossi & Sein, 2013). Action research focuses on problem-solving processes or group dynamics in a specific problem situation, whereas design science specifically aims to generate an artifact (Holmström et al., 2009). They both are fundamentally similar (Järvinen, 2007), but have a different focus, and can make use of each other (e.g., Cole et al., 2005; Järvinen, 2007; Loebbecke & Powell, 2009). Iivari and Venable (2009) on the other hand, stress their differences. Design action research Action research and design science research can be used as a staggered design (that sometimes is confused with mixed methods, that can be understood as the use of different research methods concurrently or sequentially (Borrego et al., 2009). The combination of creating a solution and affecting change in what is sometimes called action design research we see here as a sequential combination of research designs. The explanations of the different parts refer then to the steps in this stepwise design. Such a staggered design was used to develop and evaluate an artificial artifact (Hüner et al., 2009; Ractham et al., 2012). In a later step, the researchers implemented the artifact in an action research project in a real context (Pries-Heje & Baskerville, 2008; Collatto et al., 2018). Single case research Shifting the focus away from the development and prescription of an artifact to understanding how and why existing artifacts and their success and failure came to be would cause a single case research. Such a case research might generate the understanding of the context and the problem that can act as a starting point for a consecutive design science research. Case researchers usually rely on the participants to investigate phenomena specified by the researcher before doing the study. Thus, collaboration between the researcher and the participants is more critical to the success of an action research endeavor than it is for case-study research, which relies more on the participants as sources of evidence. Baskerville and Lee (1999) state that collaboration reduces action researchers’ ability to control processes and outcomes and their freedom to choose problems and reduces possibilities for finishing an action research project if focus changes during the process (Blichfeldt & Anderson, 2006). Case study research and action research aim for an in-depth understanding of phenomena in real-world settings. Action researchers sometimes adopt the guidelines that some proponents of single case research (case study) offer, as some of the latter actively argue they should (e.g., Cunningham, 1993).

136

7  Action Research Design

Key Aspects to Remember

Action research is about change Action research is a change-oriented approach. Its central assumption is that complex social processes can best be studied by introducing change into these processes and observing their effects. The fundamental basis for action research is taking actions to address organizational problems and their associated unsatisfactory conditions. Action research combines theory generation with researcher intervention to solve immediate organizational problems. Thus, action research aims to link theory with practice, and thinking with doing. It is typically an iterative process based on working hypotheses refined over repeated cycles of inquiry. Interlace action research and theoretical considerations The basic premise of action research is that action and theory should be interwoven to create results of value to both researchers and clients. The fitting point in time to include relevant theory might be in doubt but only interlacing the action research and the theory makes it research and is a distinguishing factor from consulting. Action research requires a lot of collaboration Action research requires a lot of collaboration and interaction. Specifically, if researchers go several times through the action research cycle. This takes a lot of time: Working time but additionally elapsed time because of availability and priorities of participants. Acceptance is enhanced by participation in problem definition and solution development. Establishing common perceptions is not (only) conducted by individual interviews but by workshops. So, workshop moderation becomes a key ability in the research process. This does not mean that researchers must moderate the workshops themselves, but somebody must do it. Thus, the clear allocation of responsibilities and alignment of expectations is a must. Understand the synergies between action research and design science research Both action research and design science research add to theory, i.e., contribute scientifically, and solve current or anticipated problems of practitioners. Design science research and action research are both intervening in reality to improve or support existing organizational activities and processes. The common goal of both research designs is the same: the researcher is interested in developing “a means to an end,” an artifact to solve a problem. Given the similarities, it is not surprising that the idea of cross-fertilization between action research and design science research has received attention in the literature.

References

137

Critical Thinking Questions

1. Why is collaboration important in action research design projects? 2. What is meant by “change orientation” as a crucial characteristic of action research? 3. What major challenges do you face when applying a design science research project? 4. What is meant by the “tentative nature of the initial problem definition” in action research? 5. How does action research differ from consulting projects?

Recommendations for further Readings

If you are still unsure whether action research design is suitable for your research project, you might find the following literature and readings helpful. • Athanasopoulou, A. & Reuver, M. (2020). How do business model tools facilitate business model exploration? Evidence from action research. In Electronic Markets 30. • Bhatnagar, V. R. (2017). Systemic Development of Leadership: Action Research in an Indian Manufacturing Organization. Syst Pract Action Res 30 (4), pp. 339–376. • Blichfeldt, B. S. & Andersen, J. R. (2006). Creating a Wider Audience for Action Research: Learning from Case-Study Research. In Journal of Research Practice 2 (1). • Bradbury, H. (2015). The SAGE handbook of action research. Third edtition. Los Angeles: Sage. • Davison, R. M. & Martinsons, M. G. (2007). Action Research and Consulting. In Ned Kock (Ed.): Information systems action research. An applied view of emerging concepts and methods, vol. 13. New York: Springer (Integrated Series in Information Systems, vol. 13), pp. 377–394.

References Altrichter, H., Kemmis, S., McTaggart, R., & Zuber-Skerritt, O. (2002). The concept of action research. The Learning Organization, 9(3), 125–131. Bamberger, G. G. (2015). Lösungsorientierte Beratung. 5. Auflage. Weinheim: Beltz. Baskerville, R. (1999). Investigating Information Systems with Action Research. Communications of AIS, Volume 2, Article 19. Available online at https://wise.vub.ac.be/sites/default/files/thesis_ info/action_research.pdf.

138

7  Action Research Design

Baskerville, R. (2001). Conducting action research: high risk and high reward in theory and practice. In E. M. Trauth (Ed.), Qualitative research in is: issues and trends (pp. 192–217). Hershey, PA, USA: IGI Publishing. Baskerville, R. & Lee. A. (1999). Distinctions Among Different Types of Generalizing in Information Systems Research.” In O. Ngwenyama et al., (Ed.), New IT Technologies in Organizational Processes: Field Studies and Theoretical Reflections on the Future of Work. New York: Kluwer Academic Publishers. Baskerville, R., & Wood-Harper, A. T. (1998). Diversity in information systems action research methods. European Journal of Information Systems, 7(2), 90–107. Blichfeldt, B. S., & Andersen, J. R. (2006). Creating a wider audience for action research: Learning from case-study research. Journal of Research Practice, 2(1), Article D2. Retrieved May 10, 2021, from http://jrp.icaap.org/index.php/jrp/article/view/23/69. Borrego, M., Douglas, E. P., & Amelink, C. T. (2009). Quantitative, qualitative, and mixed research methods in engineering education. Journal of Engineering Education, 98(1), 53–66. Bunning, C. (1995). Placing action learning and action research in context. International Management Centre. Cauchick, M. (2011). Metodologia de Pesquisa em Engenharia ee Produção e Gestão de Operações. (2nd ed.). Elsevier. Coghlan, D., & Shani, A. B. (2005). Roles, politics and ethics in action research design. Systemic Practice and Action Research, 18(6), 533–546. Cole, R., Purao, S., Rossi, M., & Sein, M. K. (2005). Being proactive: Where action research meets design research. ICIS. Collatto, D. C., Dresch, A., Lacerda, D. P., & Bentz, I. G. (2018). Is action design research indeed necessary? Analysis and synergies between action research and design science research. Systemic Practice and Action Research, 31(3), 239–267. Coughlan, P., & Coghlan, D. (2002). Action research for operations management. International Journal of Operations & Production Management, 22(2), 220–240. Cunningham, J. B. (1993). Action research and organizational development. Praeger Publishers. Davison, R. M. & Martinsons, M. G. (2007). Action Research and Consulting. In Ned Kock (Ed.), Information systems action research. An applied view of emerging concepts and methods, vol. 13. New York: Springer (Integrated Series in Information Systems, vol. 13), pp. 377–394. Davison, R., Martinsons, M. G., & Kock, N. (2004). Principles of canonical action research. Information Systems Journal, 14(1), 65–86. Dick, B. (2003). Rehabilitating action research: Response to Davydd Greenwood’s and Björn Gustavsen’s papers on action research perspectives. Concepts and Transformation, 7(2), 2002 and 8(1), 2003. Concepts and Transformation, 8(3), 255–263. Dickens, L., & Watkins, K. (1999). Action Research: Rethinking Lewin. Management Learning, 30(2), 127–140. Eden, C., & Huxham, C. (1996). Action research for management research. British Journal of Management, 7(1), 75–86. Foster, M. (1972). An introduction to the theory and practice of action research in work organizations. Human Relations, 25(6), 529–556. Grønhaug, K., & Olsson, O. (1999). Action research and knowledge creation: Merits and challenges. Qualitative Market Research, 2(1), 6–14. Heller, F. (1993). Another look at action research. Human Relations, 46(10), 1235–1242. Holmström, J., Ketokivi, M., & Hameri, A-P. (2009). Bridging Practice and Theory: A Design Science Approach. Decision Sciences 40(1), 65–87. Hult, M., & Lennung, S. -Å. (1980). Towards a definition of action research: A note and bibliography. Journal of Management Studies, 17(2), 241–250.

References

139

Hüner, K. M., Ofner, M., & Otto, B. (2009). Towards a maturity model for corporate data quality management. 24th Annual ACM Symposium on Applied Computing (ACM SAC 2009). Järvinen, P. (2007). Action research is similar to design science. Quality & Quantity, 41(1), 37–54. Lau, F. (1997). A review on the use of action research in information systems studies. In A.S. Lee, J. Liebenau, J. I. DeGross (eds.), Information systems and qualitative research. IFIP — the international federation for information processing. Springer. Loebbecke, C., & Powell, P. (2009). Furthering distributed participative design. Scandinavian Journal of Information Systems, 21, 77–106. Iivari, J., & Venable, J. (2009). Action research and design science research - Seemingly similar but decisively dissimilar. In 17th European Conference in Information Systems. ECIS, Verona, pp. 1–13. March, Salvatore T., & Smith, G. F. (1995). Design and natural science research on information technology. Decision Support Systems, 15(4), 251–266. McKay, J., & Marshall, P. (2001). The dual imperatives of action research. Information Technology & People, 14(1), 46–59. Mohrman, S. A., & Lawler, E. E. (2011). Useful research: Advancing theory and practice. Berrett-Koehler. Pries-Heje, J., & Baskerville, R. 2008. The Design Theory Nexus. MIS Quarterly, 32(4), 731–755. Ractham, P., Kaewkitipong, L., & Firpo, D. (2012). The use of facebook in an introductory MIS course: Social constructivist learning environment*. Decision Sciences Journal of Innovative Education, 10(2), 165–188. Rapoport, R. N. (1970). Three dilemmas in action research. Human Relations, 23(6), 499–513. Rossi, M., & Sein, M. K. (2003). Design Research workshop: A proactive Research Approach. 26th Information Systems Seminar. Haikko, Finland. Schein, E. H. (2010). Organization development: Science, technology or philosophy?. In D. Coghlan, & A. B. (Rami) Shani (Eds.), Fundamentals of organization development (1) (pp. 91–100). London, UK: Sage. Shani, A. B., & Coghlan, D. (2019). Action research in business and management: A reflective review. Action Research, 1–24. Shani, A., & Pasmore, W. (1982). Towards a New Model of the Action Research Process. Academy of Management Proceedings, August. Thiollent, M. (2009). Metodologia da Pesquisa-Ação (17th ed.). Cortez. Von Kroch, G., Ichijo, K., & Nonaka, I. (2000). Enabling knowledge creation. Oxford University Press.

8

Single Case Research Design

Learning Objectives

When you have finished studying this chapter, you will be able to: • understand the purpose of single case research: in-depth analysis of a small sample in its environmental context • embrace the contextual conditions as part of the research process • develop “how” and “why” research questions • understand that single case research lacks a standard methodology • argue why single case research is a subsidiary design that should be used if no other, more specific research design fits

Single case research

• • • • • • •

in-depth analysis of a small sample in its environmental context contextual conditions are part of the research process non-random sampling of cases answers typically “how” and “why” research questions unit of analysis can be everything no prescription of a standard methodology available subsidiary design that should be used if no other, more specific research design fits

© The Author(s), under exclusive license to Springer Fachmedien Wiesbaden GmbH, part of Springer Nature 2021 S. Hunziker and M. Blankenagel, Research Design in Business and Management, https://doi.org/10.1007/978-3-658-34357-6_8

141

142

8  Single Case Research Design

8.1 General Description of Single Case Research Design Case study research is a collective term for an in-depth analysis of a small non-random sample. The focus is on in-depth. This characteristic distinguishes the case study research from other research designs that consider the individual case as a rather insignificant and interchangeable aspect of a population or sample. We define a case study as follows. u A case study depicts a holistic picture of the unit of analysis in its environmental context that contains all relevant aspects, elements, and relationships. It shows (all) the workings of the unit of analysis that are relevant for answering research questions about understanding the case. Such a case can be an individual, a group, an organization, a process, a problem, an event, or an anomaly. Thus, the focus of case study research is to investigate into real-life phenomena and to consider the environmental context of the case. Here we can identify a significant difference to experimental research designs (see Chap. 12), where we can fully control the environmental context. With the case study research design, you can accomplish diverse research aims. These aims range from comprehensively describing a case out of curiosity to test extant theories by single cases. Yet, typically, most research questions revolve around “how” or “why” questions (Yin, 2014). Such research questions can be identified and developed by inspecting any previous studies and identifying any suggestions or opportunities for future research (Yin, 2014). Most research questions focus on explanation, description, understanding, and exploration. However, researchers can also phrase questions for confirmatory case study designs. Qualitative and quantitative data can support the identification of patterns and relationships for building, extending, refining, or even testing theories (Gomm, 2000, Ridder, 2017). As a result, case study research has different objectives to contribute to theory and uses extant theories to guide the research process. Unfortunately, these diverging goals of this flexible research design are often neglected by researchers. Most often, undergraduates and graduates are taught that case studies primarily fit exploratory research purposes at a very early stage of the research process. To define the unit of analysis, or the case, poses a major challenge to case study research. As Burns (2000) points out, ‘‘the case study has unfortunately been used as a ‘catch-all’ category for anything that does not fit into experimental, survey, or historical methods” (p. 459). Tight (2010) also argues that everything might be considered a case study. The internet and literature are full of “case studies”. The unit of analysis can be a product, a process, an organizational structure, a company, an industry, a decision, an artifact, or anything else. A precise definition of the unit of analysis has no impact on data collection and analysis as situational factors need to be considered, but the research questions as well as the drawn conclusion become more meaningful.

8.1  General Description of Single Case Research Design

143

This is one reason why a case study does not employ typical methodologies. Everything might be a case, and everything might be important to understand this case. The distinguishing element of case studies is that there is basically no single type or source of data as this hardly ever yields a holistic picture. So, you have a whole plethora of data to collect and analyze, such as pictures, videos, documents, interviews, and figures. Case study research not only relies on qualitative data. But it is also not possible to conduct case study research with no qualitative data. Understanding a unit of analysis is not possible with no consideration of its context. So situational information needs to be collected and analyzed as well. The boundaries between the unit of analysis and its environment are not clear cut because of interactions between the two. Drawing the boundary of the unit of analysis is arbitrary but also not highly relevant as context needs to be included. To clarify a potential misunderstanding, our definition of case studies above does not represent a broad consensus. After thoroughly analyzing current literature on that topic, we conclude researchers may face initial confusion when trying to define “case study research”. Search machines like google offer thousands results about books and articles. Yet, most of the hits are not directly relevant to the discussion about case study research. Authors describe case studies predominantly in the qualitative research domain with unique characteristics and tasks (Tight, 2010; Yin, 1994). Case studies are the recommended research design when “how” or “why” questions are being asked, when the researcher has little control over events, and when the focus is on contemporary phenomena within some real-life context (Yin, 2014). Other authors, such as May (2011) understand this research design to bridge the gap between generalizing and particularizing, qualitative and quantitative, deductive, and inductive approaches. We believe one reason for this confusion is that case study research is rather a residual research design. If other, more specific designs are not suitable to answer a research question (e.g., cross-sectional research), then researchers are usually left with the case study research design. The description of case study research also requires contrasting this research design with “qualitative research” in general. Why is this the case? Let us first look at the conclusion drawn by Stake (2005) about the peculiarities of case research. He states that case researchers must: • • • • • •

bound, limit, and focus the case(s) under investigation, select specific phenomena or topics to develop research questions, search for patterns in the collected data, triangulate key observations for interpretation, select alternative interpretations to pursue, and develop assertions or even generalizations on the case(s).

We can conclude that, except for the first aspect, all steps could be identical to qualitative research. Tight (2010) even argues that the first aspect of focusing the study is crucial for every research design. He also states that Yin’s textbooks on case research is primarily

144

8  Single Case Research Design

concerned with offering guidance on how to collect, analyze, and write up data. Most of the guidance provided by Stake (2005) and Yin (2014) seems to be rather generic and could easily apply to other qualitative, and even some quantitative research designs (Tight, 2010). Tight (2003) goes even further when stating, “most pieces of research can be described as, in some sense, case studies. This is not, therefore, a very useful way for categorizing and differentiating between the outputs of research, whether on higher education or other aspects of society” (p. 9). He also suggests doing without the “case research label” and call this research design rather small-sample, in-depth studies (we followed this advice as you note when scrolling up to the very first sentence of this paragraph). This approach leads to stress more on the important sampling procedure, data collection, and data analysis part of this research design (Tight, 2010). Thus, case study research can be critically interpreted as a catch-all design to justify to fundamental descriptive studies that do not fit with other research strategies (Merriam, 2009). This gives rise to pay enough attention to select and define what researchers mean by the term “case”. As Creswell (2013) states, this approach ‘‘explores a real-life, contemporary bounded system (a case) or multiple bounded systems (cases) over time, through detailed, in-depth data collection involving multiple sources of information […] and reports a case description and case themes’’ (p. 97). Thus, case study research deserves this label only when focusing on well-reasoned case selection. Case selection is the precursor to case analysis. Every researcher needs to present this as a convincing argument (Hyett et al., 2014; Merriam, 2009). They must provide good justification when selecting cases so that the reader can understand why the case was selected from a larger population of cases. Once a case or cases are selected, researchers need to provide appropriate contextual descriptions to get insights into the setting and context in which the case is investigated (Hyett et al., 2014). To summarize, the quality of case study research critically depends on the case selection and the reasoning of the contextual bounding of the case(s). Otherwise, case study research equals the qualitative research approach, where the term “case” is no longer justified. u Important

We address a common misunderstanding about case study research. Case studies should not be confused with qualitative research or cross-sectional designs based on interviews. We can base them on any combination of quantitative and qualitative evidence. Thus, case study is not an isolated, distinctive research design to empirical research. In fact, case study research designs draw on several empirical methods. These methods span the entire spectrum of data collection and data analysis techniques. Thus, cases also embrace secondary data. For example, researchers can consider documents like brochures, pictures, videos, reports, studies, diaries, minutes, and websites. Also, quantitative data and statistical analysis may be combined with qualitative data.

8.1  General Description of Single Case Research Design

145

Bryman (2004) supports this view on case studies as a research design. He contrasts case research with cross-sectional, longitudinal, experimental, and comparative designs. He defines case studies as follows. “the most common use of the term associates the case study with a location, such as a community or organization. The emphasis is upon an intensive examination of the setting. There is a tendency to associate case studies with qualitative research, but such an identification is not appropriate […] case studies are frequently sites for the employment of both quantitative and qualitative research”(p. 49). Piekkari et al. (2009) analyzed 135 published case studies in different international business journals. Their analysis reveals that most research does not conduct holistic, in-depth, and rich case studies. Most of these published cases are declared as explorative and lack an explicit statement of the theoretical contribution of the case study. Case studies are often designed rather as multiple “case studies” (but mistakenly called cases) with cross-sectional designs based on interviews. This is also confirmed based on the experiences by the authors of this textbook. In summary, case study research can be a promising research design because it allows us to. • deal intensively and holistically with a real-life phenomenon, • gain understanding of developments, processes, and cause-effect relationships in a particular business and social context, • collect a lot of details on the case that would not be obtained by other research designs, • combine many different data collection and data analysis techniques, • conduct in-depth analyses and can find information and data not expected before the research project started. So, researchers can develop new hypotheses for later statistical testing, • close in on real-life situations and test perspectives and views directly in relation to phenomena as they unfold in practice (Flyvbjerg, 2006), and • get as close as possible to our research area of interest. This is possible through direct observation and access to many subjective factors. Let us glance at an excerpt of a good example of a case study research design (Woods, 2009). Example

Let us look at an article published in Management Accounting Research, a top-rated academic journal according to the VHB-JOURQUAL 3-ranking. JOURQUAL3 ranks relevant journals in business research based upon the judgement of its VHB members. Woods (2009) addressed risk management within Birmingham City Council.

146

8  Single Case Research Design

She used case study materials to extend existing theory by developing a contingency theory for the public sector. In her paper, she outlined the research design used to conduct an in-depth study of the risk management system within a large public sector organization. Her research aim was to record and understand the control structure used for risk management purposes. Also, she looked for factors, or contingent variables, influencing both the selection and maintenance of the risk management. In the following, we look at some excerpts from her paper that addresses the research method (Woods, 2009, pp. 71–72): “The case study method was adopted for the research on the grounds that it facilitates the development of a deeper understanding of the role of different types of controls and their impact upon organizational performance (Otley and Berry 1994). Case studies are particularly useful for exploratory research, where an inductive approach can be adopted, using theory to explain empirical observations and inform refinements and extension of theory (Berry et al. 1991, Otley and Berry 1994). Researchers have also explicitly recognized the usefulness of case-based research in the field of management accounting practice (Scapens 1990), although the authors face the inevitable challenge of linking multiple detailed experiences back to core academic theory. A key component of case study research is the interview. In this research, interviews were critical because of the relative novelty of the issues being discussed (Horton et al. 2004), which left the researcher with some initial uncertainties regarding what were the most important questions to ask. Interview questions were developed out of a literature review of the areas of both management control systems in general and risk management in particular. The review findings formed the basis of a series of semi-structured interviews complemented by a less structured discussion. The interview format created a flexibility that enabled interviewees to develop issues and “think aloud” about areas that they saw as being of particular concern. This approach also facilitated the generation of supplementary questions for use in later interviews, based upon key issues identified by staff working within the organisation. The interview format thus allowed for discussion of matters which had not arisen in the initial literature review. […]. The use of extended interviews and attendance at management meetings and training sessions provided evidence on both the components of the control system as well as how it is used in practice. The evidence was in both verbal and documentary format, providing additional detail on the internal working of the risk management system. A total of five separate interviews were conducted, encompassing the Head of Internal Audit, other members of staff in Birmingham Audit, and the council’s Chief Executive. The interviews varied in length between thirty minutes and two hours, were digitally recorded and then fully transcribed to ensure accuracy. The approach to questioning that was adopted in the interviews was largely open ended. For example, interviewees were asked to explain the perceived role of a particular function or control and offer personal views on the reasons why systems had

8.1  General Description of Single Case Research Design

147

evolved in a particular way, or how things may change in the future. In the analysis of the transcripts these views were then compared with each other and the documentary evidence to test for consistency. Additional information was collected via attendance as an observer at a meeting of the Resources Directorate Risk Representatives Group, where both policy and operational issues relating to risk were discussed. Understanding of the process of internal communication and training in risk management methods was obtained through participation in a two-hour risk training session for new staff in internal audit, and a guided walk through two key pieces of software that support the management control systems within the council. The software that was explained was Project Management Office Support (PMOS), which is a project management operating system, and Magique a risk management system purchased from a commercial vendor.” (Woods, 2009, pp. 71–72). ◄ We can conclude the following from her research: • often, case study research is considered inferior to pure quantitative approaches. Yet, this example shows that also well-conducted case study research can be published in top-ranked academic journals. • Woods (2009) argued that her findings confirm the existing literature, concluding that the basic structures of risk management are common across large organizations. Yet, her findings also challenge the current literature by indicating that at the detailed operational level, the risk control system is contingent upon specific variables. These findings are new and only revealed by her in-depth analysis. • case study research does not equal interviews. Specifically, students of undergraduate or graduate programs confuse case studies with interviews. Woods (2009) combined different data sources. Apart from interviews, she also collected information as an observer at a meeting and analyzed many documents. As you have learned, case study research designs post immense challenges. That is why we consider this design not the default research design but should be viewed as a subsidiary one. If there is no research design better suited or answering my research question that I am not able or willing to specify better, then I can think about using the case study research design. Based on this general description, we now detail the distinctive specialties of case study research design.

148

8  Single Case Research Design

8.2 Particularities of Single Case Research Design In this section, we specifically address the elements that make case studies a discrete research design differentiated from others. Next to the characteristics of case studies, we address the main issues and decisions to be made within this research design, and the major pitfalls. Case study research looks easy, but it is not. Yin (2009) impressively shows that case study research poses high challenges to researchers, possibly even significantly higher than pure quantitative research. Conducting case study research is time-consuming and may tie up a considerable number of resources (General Accounting Office, 1990). Important preconditions for conducting case study research are a high level of knowledge about the (iterative) research process and the case(s) under investigation. Researchers must know that despite software, literature does not offer a silver bullet to analyze and interpret qualitative data. Also, researchers often have false expectations about the outcome of case study research and the unjust generalization of the findings may lead to negative consequences (Baškarada 2014). Also, very few articles deal with the concrete steps that researchers must take for conducting effective case study research (Hancock & Algozzine, 2016). Baškarada (2014) also stresses the need to have handy guidelines that researchers can follow. We acknowledge that this research design has its advantages (in-depth; contextual factors) but only if done correctly. If it is not done correctly, researchers do not add intellectual contribution. And this is readily detectable by the reader. Here the question remains: so, what? At a planning stage it is not ultimately decidable, which data needs to be collected and analyzed for a holistic picture. Therefore, there is no clear-cut methodology for conducting a case study. So, the responsibility of the researcher is much higher because you need to think about every single step and justify it. Everybody who thinks about doing a case study by conducting three interviews is sorely mistaken. In the following, we provide some guidance on how to deal with major challenges to address for future researchers. At some points, we base our recommendations on Merriam (2002), Stake (1995) and Yin (2014), three authors who significantly contributed to the literature on case study research.

8.2.1 Characteristics of Single Case Research In this section, we elaborate on the key characteristics of case study research along the steps of the research process. Conclusion The typical conclusion of a case study is the comprehensive description of the case with its relevant elements, their (causal) relationships and its conditions. Depending on the

8.2  Particularities of Single Case Research Design

149

theoretical foundation of the case study, these elements, relationships, and conditions confirm existing theory or are not included in current theory. The conclusion can be only a summary of the comprehensive description, focusing on the most relevant or surprising findings. Intellectual contribution The intellectual contribution of a case study is finding new, or differentiating known elements, (causal and other) relationships and conditions for the description of this one case. This confirms that the current theory can describe the case to a certain degree, but also shows which part cannot be described using existing theory. Theory might be elaborated or created based on a logical generalization of the prevailing characteristics of the case. “If this is true in this case than it can be assumed that it should be true in similar cases.” There is a huge continuum about this contribution ranging from “in theory this element, relationship, condition is neglected” to “any kind of case can only be comprehensively described and understood as a whole”. Argument The key argument a case study makes is-based on the rich, comprehensive data gathered and analyzed-that these elements, relationships, and conditions are important to describe and understand the case. They are necessary (but not necessarily sufficient) to describe and understand the case. Any description neglecting them cannot be comprehensible and will not enable understanding the case. Results The result of a case study is a rich, comprehensive description (descriptive) and explanation (explanatory) of the case, depicting all the elements, (causal and other) relationships, and conditions necessary to describe and understand the case. Methods Typically, you may use multiple methods for analyzing different data and triangulating the data analysis. Data Case study research usually requires gathering varied data from many different sources. Qualitative data are often used, but quantitative data might also be relevant. The desired comprehensiveness calls for as much data (types) from different sources as possible. Data needs to be collected if valuable information is retrieved. Research question The research question of a case study usually starts with “why” or “how”. A typical research question in case study research looks like this:

150

8  Single Case Research Design

• how does this [system / process] work or function? • why does this system behave in this way? It is important to remember that a research question starting with “how” does not automatically make the design a case study. “How did the DAX stock market index develop in the last decade?” could be a case study, but it is probably only the unprecise version of “are there identifiable trends the DAX stock market index followed in the past decade? “.

8.2.2 Issues to Address in Sinlge Case Research In detailing the research design, you face many case study specific problems and decisions. We list the most important ones and the main options you have in the following. As with every research design, case study research also has some issues and challenges to consider. Luckily, most of them can be addressed by researchers or represent false expectations of case study research. Case researchers must pay attention to the valid critique of Maoz (2002): “the use of the case study absolves the author from any kind of methodological considerations. Case studies have become most times a synonym for freeform research where anything goes” (pp. 164–165). Indeed, we must fight hard against certain preconceived assumptions of case study research. Most of the criticism is justifiable. For example, critics point to the absence of rigorous techniques and procedures for case study research. Yin (2009) confirms this issue. He states we face a relative absence of structured and valid guidelines. In the following, we address some of the most important issues and provide some guidance on how to deal with them to counter this criticism. Intellectual contribution Case study research contributes important insights to the body of knowledge. By analyzing in depth, a single case, it becomes possible to confirm generalized research. Following deductive reasoning, if a theory holds true for all objects, it has also to hold true for this specific object. So, confirming the variables used in the model (see chapter intellectual contribution) and their impact is a contribution of some value, as the generalized models use probability (significance) levels. However, case study research enables to analyze the yet unexplained variance or variations in the study object. This is usually the starting point for further research based on the conclusions drawn. For example, “in this case XY was important. This is not yet part of a generalized model. But we can assume based on the characteristics of the case that XY might also be important in instances with similar characteristics (a, b, c). So, we should research further on including XY in models referring to objects showing A, B, C”.

8.2  Particularities of Single Case Research Design

151

Standard textbooks on case research often lack an explicit explanation of how case study research is linked to intellectual contribution. Neglecting these potential contributions weakens the contribution of case studies to scientific progress. This may be also a reason case study research may be considered an inferior research design (Ridder, 2017). We differentiate three goals of how case study research designs deal with theories. They all contribute to scientific progress, but in different ways. It is important to understand that all case study research designs can be located on a “theory continuum”, thus, they are always linked to theoretical considerations. Case study research may build, develop, or test theories. In the literature, however, we note a strong emphasis on the exploratory role of case study research. This also means that researchers do not (yet) acknowledge the richness of case study research design to challenge, refine, and even test theories (Ridder, 2017; Welch et al., 2011). The theory building case study is suitable for new and complex phenomena. Only little to no research exists so far. No theoretical foundation can guide the research process. Even if potential theories exist, the researchers deliberately start without the consideration of any theory to be unbiased and not limited at the very beginning of the research process. Thus, the phenomenon of interest is outside existing theory. The research process takes place at an early discovery stage and serves to gain pre-theoretical knowledge. New concepts and relationships may result from such a research design. The case study can even be very intrinsic, meaning the comprehensive understanding of the case per se is of interest rather than answering a specific research question. Such cases provide first answers to “what” and “how"-questions and may build a tentative theory. For example, researchers may be interested in gaining insights on the success or failure of investments projects. One research goal could be to identify specific success factors to explain why some projects are more successful than others and to develop a first tentative theoretical framework. These findings may serve as a basis for future researchers for refining these factors with a theory developing case study. Once the tentative theory has grown more mature, a theory testing case study may confirm or falsify the previous theoretical considerations (Meyer & Kittel-Wegner, 2002; Ridder, 2017). With theory developing case studies, we strive for closing gaps in existing theories with the goal of advancing theoretical frameworks. Theory developing cases can somewhat be between theory building and theory testing case studies (Keating, 1995; Ridder, 2016, 2017). In comparison with theory discovery cases, theory refinement cases make use of a theoretical starting point and focused research question. However, researchers who conduct theory developing cases must stay flexible about the discovery of new concepts and relationships between concepts (Keating, 1995). A well-known author representing this case study design is Yin (2014). He is one of the most cited authors on how to design single and multiple case studies to refine and advance existing theories. Yin clearly states that an existing theory is always the starting point of a case study research design. We believe that many undergraduates or graduates overlook the fact that Yin advocates theory developing research. Rather, researchers are tempted to focus too much on

152

8  Single Case Research Design

the “operational part” of how to execute a case study step-by-step in his textbooks. Thus, this research design is geared towards the advancement of existing theories (Ridder, 2017). Several possibilities to advance a theory exist. Preexisting concepts and constructs may be extended to other environmental contexts. Also, the goals could be to modify existing theoretical perspectives (Edmondson & McManus, 2007). This research design assumes that a real-life phenomenon is only partially understood. Compared to the theory-building research design, the focus is on challenging existing theoretical propositions, and not on the development of these. Predictions are based on existing theories. The collected empirical data must be confronted with the theoretical predictions (Yin, 2014). With theory testing case studies, we address the context of justification. Confirmatory cases assess theoretical frameworks that have been developed by other researchers. They are suitable for preliminary plausibility tests of alternative hypotheses. Sometimes, researchers face different, conflicting hypotheses about the same topics. Considering the limited resources of researchers, we recommend applying case study research to test the more plausible hypotheses first. For example, we test the hypotheses of the most credible success factors of investment projects by a single case study approach. So, case study research can indeed check hypotheses and theories. Yet, most researchers conduct more exploratory cases. One reason for this is that many academics doubt that case study research is suitable to test theories. Remember, hypotheses are statements like laws that are supposed to be transferable to a larger number of cases (Boos 1992). Yet, multiple case studies are more appropriate for confirmation. Obviously, the sampling approach is crucial. The chosen cases must either be very similar in their preconditions or very different to test hypotheses (Meyer and Kittel Wegner 2002). Even single case study research results can be used for hypothesis testing, in case of predictions being relatively precise and a low measurement error (Levy, 2008). Concretely, we need to derive distinctive theoretical propositions based on an existing theory. If we can define the specific conditions within which the theory is capable to explain the real-life phenomenon, a single case study can indeed challenge (“test”) an existing theory (Ridder, 2017). Unit of analysis A major challenge in case study research is to define a unit of analysis. This is also one critique Stake (2005) explicitly addresses when stating “here and there, researchers will call anything they please a case study” (p. 445). The unit of analysis defines the research object, the case. It is important to circumscribe the unit of analysis as precisely as possible because of the focus it adds to the research. It also shapes the expectations of the holistic picture. Take, for example, the case of a failed product launch. It makes a world of difference if your unit of analysis is the failed product launch (why did the product launch fail?) or if your unit of analysis is management’s opinion about the reasons for failure. The content of the holistic picture to be depicted will only be similar, but not equal. Sure, both questions address the product launch (similar) but while one focuses on the objective reasons for failure and will start by collecting information about launch

153

8.2  Particularities of Single Case Research Design

Table 8.1  Common sampling approaches in case study research. (adapted from GAO 1990 and Flyvbjerg 2001, with own additions) Selection criterion

Situation to use

Research question modification

Convenience and accessibility

Case selected because it was expedient for data collection purposes

Precondition: only if sufficient access is available there is a chance of a holistic picture. If this is the only selection criterion all the other research questions modifications are not feasible

Bracketing

What is happening at extremes? What explains such differences?

Either focusing on outliers that are not considered in other studies or the maximum

Best case(s)

What accounts for an effective program?

Focus on the learnings from a case considered good or best in class and what makes this case especially good

Worst case(s)

Why is the program not working?

Focus on the reasons something is considered not working or bad and what makes this case especially bad

Cluster (a multiple case research according to our definition)

How do different types of programs compare with each other?

Focus on explaining and comparing the differences

Representative

Instances chosen to represent important variations

Impact and importance of the variations

Typical

Instance chosen to represent a typical case

Holistic narrative of an example without surprises

Special interest

Instances chosen based on an unusual or special attribute

Impact or importance of that special attribute of unit of analysis or context

Maximum variation Cases which are very different (a multiple case research in our on one dimension differentiation)

Impact or importance of that dimension and of differences on that dimension. Like bracketing but focusing on one dimension

Critical case

A case with strategic importance to the general problem

Focus on what makes the case so important

Extreme case

Extreme or unusual case

-

preparation and so on, the second focuses on the beliefs of the persons involved. These beliefs might be based on facts but might also only be loosely connected to facts.

154

8  Single Case Research Design

The definition of the unit of analysis also has an enormous impact on the expectations of the reader. Often the research question promises one, but the unit of analysis is in fact the other. Thus, it is mandatory to define your unit of analysis precisely and, if possible, to include the definition in the research question. This is not the same as setting boundaries to the research. To understand the case, we need to consider the context or environment. Limiting the research to certain boundaries is contra productive as we might exclude the information that would be relevant. Perhaps understanding the behavior of the CEO requires understanding the current trouble at the CEO’s home. The distinction of Yin (2004) whether the unit of analysis is embedded or not, in fact refers to the differences in context. Embedded units of analysis share a similar, perhaps even equal environment, “stand-alone” units of analysis do not. In case studies with one unit of analysis, the distinction between a unit of analyses and embedding case and context is mainly one of semantics without impact on the research itself. It is possible to research more than one unit of analysis. The focus of the research question then shifts to the comparison of the cases. This is the sole justification for adding additional cases, as more cases still do not offer (statistical) generalizability. We will address multiple case studies in a separate chapter as another research design. Sampling and its impact on the research question Case study research designs employ nonrandom sampling. It is not aimed at representing a larger population. Rather, researchers select a case because it is of particular interest. Many published case studies cannot identify the rationale for case selection (Dubé and Paré 2003). For example, cases may be selected based on opportunity, convenience, purpose, variation, and bracketing. Common sampling approaches are depicted in Table 8.1. Please note that these are not mutually exclusive. The selection affects the research questions. We added how this typically changes the research question. According to Yin, reasons for using single-case studies include studying a critical case, an extreme case, a representative or typical case, a revelatory case (involving a novel situation), and a longitudinal case. Purposive case selection provides an opportunity to collect the most relevant data (Edmonds and Kennedy 2012), and longitudinal cases offer an opportunity to identify trends over time (GAO 1990, cited in Baškarada, 2014). Drawing probability samples is not relevant for case studies. Some researchers argue that sampling procedures in case studies are not well reasoned. The unit of analysis defines what we mean by “the case” or “the cases”. For example, we can consider an event, a process, an individual, a group, or an organization as cases (GAO 1990). As we face conflicting literature about the terminology, it is crucial to define relevant terms. With an event or a process, defining the time boundaries (i.e., the beginning and the end of the case) is imperative. While it may sound obvious and simple, identifying the unit of analysis requires careful consideration, as any confusion over it may invalidate the entire study (Gerring 2004). Yet, a review of qualitative case studies in operations management

8.2  Particularities of Single Case Research Design

155

found that roughly 80% of articles did not clearly state their unit of analysis (Barratt et al., 2011, cited in Baškarada, 2014). Accessibility For an in-depth analysis, you need access to data sources. As you draw a holistic picture, this means more access than “I have one person who agreed to an interview.” You need the possibility and thus the agreement and support of relevant decision makers of the organization that is or embeds the unit of analysis to conduct your research and collect data. Also, you need a lot of data and distinct data types from multiple sources. Multiple data and data sources  Case researchers do not rely on one method for data collection. Yet, the methods used should be chosen adequately according to the goal of the project to collect relevant data to answer the research questions. The most used methods for data collection are interviews, surveys, and observation. As these are not mutually exclusive, they are often used in combination. For example, researchers combine personal interviews (face-to-face or via telephone) with written surveys as (online or physical) questionnaires. However, the most popular approach in case studies is the personal interview. We differentiate between focused interviews, semi-standardized interviews, expert interviews, the problem-centered interview, and narrative interviews. Indeed, interviews are suitable for case studies with the goal of the exploration of a complex, in-depth, holistic, yet little researched phenomenon. As an alternative to data collection by questionnaires, observation offers additional possibilities to gain insights into the case under investigation. Researchers can collect data about observations of individual events or chains of events over a longer period. Observers either take an active part in the researched case or they are non-participating and observing from a neutral perspective. An internal observation is present when the researcher himself takes over the observation. Also, a valuable source for case study research may collect documents. Researchers can use many sources such as minutes, contracts, policies, codes and much more. All these documents can be subjected to a content analysis. This step equals the procedure with interview and observation protocols. We may choose from three basic forms of content analysis. With summaries, we can develop an overview of the data and its key findings. This helps us to determine the structure and logic of the protocol, but also possible gaps and questions can be identified. Explication classifies documents in their factual and temporal context. In this way, the content and statement of the sources can be assessed in terms of correctness and completeness. At the stage of structuring, we can assess the extent to which the sources are meaningful and reliable (Mayring 2010). The in-depth analysis is the goal of your case study research. Any data might be relevant for depicting the holistic picture. Among other you might use (Yin, 2014): • documentation (e.g., letters, e-mails, proposals, articles, minutes, and notes), • archival records (e.g., public use files, budget records, maps, charts, and survey data),

156

8  Single Case Research Design

• direct observations (passive observation), • participant-observation (active observation), • physical artifacts (e.g., devices, tools, and art), and • interviews with members of the organization and with non-members about the organization. Adaptability, missing methodology and triangulation Because of the amount of data and the multiple data sources you must choose from, the research process cannot be entirely planned. The key word here is “entirely”. You plan your data collection and analysis, but you need to be open and adaptable to follow up on new insights. If, for example, in your interviews the name of a certain person crops up, speak with this person. If you visit the organization under analysis on site and you see pictures and statues of the founder everywhere, you might consider his or her impact on today’s values and behaviors. This might be just old junk collecting dust, but this might also be a hint for the domineering guiding principle “what would our founder think about this?” You can only know if you have collected and analyzed data about this. The lack of a typical methodology that you can use a guideline offers you high degrees of freedom but is also very challenging. The only method that is a must for any case study is triangulation, as you must draw a holistic picture (Dooley, 2002; Eisenhardt, 1989; Ridder, 2017).

8.2.3 Major Fallacies in Conducting Single Case Research While providing guidance and support for research projects, these are the major pitfalls students encounter in their case study projects. Appears misleadingly easy Conducting good case study research is difficult. Many students think about case study in terms of “I will write about XY. I conduct three interviews, and this is a case study”. This is wrong. The often-asked question “how many interviews do I have to conduct” hints at a severe lack of understanding of case study research design and is in fact a sign that case study is not the appropriate research design for the person asking. You must depict a comprehensive picture of the case in its context. How do you or your supervisor know in advance how many data you need to collect to do so? And how do you know that you only need interviews as a data source? Basically, you claim that everything besides these interviews is not relevant to understand the case. Obviously, this does not represent the mindset conductive for a case study research. The chances for corrective actions are low in case study research. You must consider this in the timeline and the risk management for your project. In other research designs with typical methodologies, you often make errors that can be rectified. If your case study does not yield a comprehensive picture of your unit of analysis and you do not

8.2  Particularities of Single Case Research Design

157

understand your case, what do you do? Basically, you would have to start over, collect more data, different data, from additional sources, and from the same sources. Especially the last might be awkward. Your data sources (e.g., interviewees) might no longer be available. Thus, case studies are often very unsatisfying and graded accordingly. Research question As already discussed, specifying the research question is a major and important step in the research process. This is especially true about single case research. The research questions of a case research design refer to the comprehensive and in-depth understanding of the unit of analysis. Thus, the research questions often start with “how” or “why”. Other questions can also be a case study. For example, “what are the reasons” is basically the description of the meaning of “why”. Also, the question “how does digitalization impact the Gross National Product of European Countries” is not a good research question as it is not specific enough. Answering this question does not require a case study research design. So, it is important to develop a research question that focuses on (completely) understanding a specific unit of analysis. The words of the question might be a strong hint. Yet, you must make sure that they really reflect the meaning of the research question and are not just used because of imprecise use of language or not yet thought through research questions. Number of cases There is no trade-off in single case research between the number of cases and the depth of analysis. There are other research designs (e.g., cross-sectional field study) that face such a trade-off. But those are not any longer single case research. You forgo the holistic picture. If you add cases, you must depict two holistic pictures that you then have to compare. So, you significantly increase the effort needed for data collection, analysis, and discussing the results. This only makes sense if your research question focuses on the comparison of the cases. Trying to add substance by adding cases that are not comprehensively analyzed does not work. If you do not need more than one case to answer your research question, then stick with one. Describe this case as completely as possible. If you cannot do this, choose another research question and research design. Sampling not in line with the research question In most cases, sampling will be entirely driven by convenience and accessibility, especially on bachelor’s and master’s level and with an in-depth analysis in mind. This sample may limit your research question. You cannot answer questions like “how do women on the board of directors impact the decision quality in company XYZ?” if your sample does not have any women on the board. Sticking to your research question usually leads to a sample with insufficient accessibility and therefore only a superficial description and analysis.

158

8  Single Case Research Design

Subjectivity One point of criticism often made about case studies is that they are subjective. However, every kind of research is subjective in choosing methods, proxies, etc. This also holds to for case studies but might be a bit more obvious. Same as with other designs, this means that you must explain and document what you do. Why did you interview these six people but not also those three? Why did you visit that office building but not this shop floor to make observations? Why did you take part in the meeting on the 5th but not in the one on the 23rd? In single case research, you must embrace these subjective choices and argue convincingly. Obviously, this is easier with more data. For example, if you compare the minutes of the meeting you participated in with the minutes of the meetings in the last six months you just read, you can argue that the visited meeting was apparently not an unusual one. This is more convincing than “by happenstance I was on site on that date.” By explaining and documenting every step you take, you also increase the (theoretical) replicability. Drawing conclusions not in line with the research design Many researchers believe case research is an inferior approach because it is difficult to establish validity, reliability, and objectivity. Specifically, the concern about poor validity and reliability represents a basic critique of qualitative research methods. Validity is one of the most important criteria for assessing the quality of research designs. Yin (2009) argues that external validity has been a major obstacle in doing single case research. Yet, case study research is an explicitly acknowledged interpretive basis for meanings, reasons, and understandings, as Berg and Lune (2010) explain: “quantitative measures appear objective, but only so long as we don’t ask questions about where and how the data were produced […] Pure objectivity is not a meaningful concept if the goal is to measure intangibles [as] these concepts only exist because we can interpret them” (p. 340). Also, Mitchell (1983) addresses this basic critique. He counters it by the rigorous procedures that can be applied in case research. „Case studies of whatever form are a reliable and respectable procedure of social analysis and […] much criticism of their reliability and validity has been based on a misconception of the basis upon which the analyst may justifiably extrapolate from an individual case study to the social process. The validity of the extrapolation depends not on the typicality or representativeness of the case but upon the cogency of the theoretical reasoning “ (p. 207). Results from case research can be hard to justify because of the inherent interpretation of the data. Conclusions drawn by the case researchers may be highly subjective. Cases are biased towards verification. This means case study research might confirm the researcher’s preconceived views and opinions. You need to be aware of the limitations of a case study. You cannot generalize your results if you research only one case. Comparing case study research with a cross-sectional design is not useful. Case findings are not meant to be generalized and thus do not represent a larger population in the statistical sense. Yet, analytical generalization is possible. Researchers may strive for

8.3  Writing a Single Case Research Paper

159

linking case study results to broader theoretical foundations. This analytical generalization process of case study research needs repeated testing by replicating the findings as often as possible. Yin (2009) states that if “once such replication has been made, the results might be accepted for a much larger number of similar individuals, even though further replications have not been performed” (p. 36). This replication logic depends on a consistent sampling procedure and on the number of replications. Yin (1994) calls this “analytic generalization” as opposed to “statistical generalization” (see also Walsham 1995, Halaweh et al. 2008). Flyvbjerg (2004) argues that in social research, only specific cases are approachable, and context dependent knowledge can be gained. About generalization, he states that it depends on the cases and how researchers sample them. Stake (1995) puts forward that generalization has never been the goal of case research. Rather, case study research aims for particularization. All the stated concerns about reliability, validity, and generalization have all been extensively dealt with in literature. Researchers have appropriate means available to counter these critiques. Thus, case study research is not an inferior research design per so, rather it lacks rigorous application by many researchers. You take many subjective decisions during the research process and, by design, interact with your research object quite often. This limits practical replicability: another researcher can study and reflect on your research but cannot replicate your case study. This is perfectly alright. The case study research design offers a lot of insight. But researchers often try to prove that for their case the limitations do not apply. They probably want to increase scientific and academic quality of their research. This usually backfires as those limitations are inherent to the research design. So, embrace and stress the advantages instead of trying to weaken the limitations.

8.3 Writing a Single Case Research Paper Writing a case study research paper follows the principles and structure detailed in Chap. 4. However, some important aspects regarding a single case research project report are (partly) different from other reports. We address these idiosyncrasies based on the standard structure of a scientific paper. Introduction Expectation management is very important. Hence, in your introduction, you precisely define your unit of analysis and your research question. Do not overpromise what you plan to accomplish. So, it may be useful to take care about the proper wording. While describing the outline of your report, mention several data types and the respective methods instead of just focusing on interviews (even if they are your primary data source). This prevents your reader from getting the first impression of “yet another interview collection that is claimed to be a case study”.

160

8  Single Case Research Design

Describe the organization that is your case (unit of analysis) or embed your case in a few sentences. Do not get too detailed as this point as a comprehensive description is part of your result section. Another option is to include the description of the case in your background chapter. Theoretical background Often the case description is included in the background chapter. If you do so, then we suggest renaming the chapter to reflect its content. Basically, you then create a new chapter. Do not confuse the description of the case with the theoretical foundations to base your research on. Usually for case research, theoretical foundations may be relevant and should be clearly articulated (Flynn et al., 1990). This includes differentiating between theory building and theory testing approaches. If the case study data uncovers constructs that do not fit within the theoretical propositions, researchers may extend it by considering additional constructs and propositions. The alternative controversial view argue against any theoretical preconception before data collection, with a view that any theory should purely emerge from the raw data (Eisenhardt, 1989). As studies may have multiple stakeholders (e.g., researchers and different organizations), it is also important to clearly differentiate between, or align, expected practical and theoretical contributions (Darke et al., 1998). Potential benefits to the case organization include benchmarking against best-practices and other organizations, and rich descriptions of the phenomenon of interest. Interviewees can also benefit by obtaining a deeper understanding of the research problem (Onwuegbuzie et al. 2012). There is no single theory that enables a holistic picture (i.e., no “world formula”). Thus, strongly believing in a distinct theory might in fact bias your research, hinder, or even prohibit you from achieving the aim of a case study. Therefore, disclosing underlying theories is especially important. For example, when researching why management made certain decisions that are detrimental to shareholders’ benefits, you should refer to the principal-agent theory. Some researchers try to abstain from theoretical foundations. This is because they aim at collecting and analyzing data as unbiased as possible. If this is the case, it also needs to be made abundantly clear. Literature review The literature review of a case study has usually three distinct purposes that should be reflected in its structure. • research with generalizable statements about elements, relationships, and conditions, which then also would pertain to your case. Such studies usually employ quantitative methodologies result in variables that to certain part explain an output variable and discard other variables that are not significant. For your case study, it is important what these significant variables and the explanatory power of the established models

8.3  Writing a Single Case Research Paper

161

are. The purpose of your case study is usually to find additional variables that (potentially) help explain the remaining variance. The insignificant variables are a good place to look for those. The significant variables need to be addressed in your case study to accomplish a holistic picture. If you leave significant variables out-or more accurately, the concepts that they represent–your picture is comprehensive and prone to criticism. • other case studies with similar research questions or units of analysis. Here there are several issues to address: – what kind of case do the existing studies comprise? A good understanding of the characteristics of your case as compared to others researched so far helps to argue the contribution of your study and to guide your research. Even if your case selection criterion is “convenience”, it helps for example to link “special interest” or “important variation” (representative) to your case. But you must look for and document these attributes and variations. Obviously, this part needs to be adapted after having conducted the case study. – what data and sources have been used? What has supported you to draw a holistic picture? This helps you to guide your research. What sources to include from the start and which to use as backups if their relevance becomes apparent? What additional data and sources can add to the comprehensiveness of your study? What has been neglected so far? You can take away a lot from excellent case studies referring to your unit of analysis, even if the research question is not the same, as they also depict a picture as holistic as possible. – what kind of methods have been used for data collection, data analyses and triangulation? • reasoning, what the research gap is and how your case research design contributes to closing this gap. Typical research gap The typical research gap in or leading to a case study is that there is too little theory or corroborating research about elements, conditions, and relationships. There might be phenomena that cannot satisfyingly be explained with the existing body of knowledge, or the body of knowledge itself is rather limited. Sometimes studies using cross-sectional or longitudinal research designs exist, but their power of explanation is rather low. In this case, the part of the variation in the dependent variable is caused by something else that is not yet known. Typical research aim Based on the research gap, you state the purpose and aim of your study. In case study research, this requires describing what your intellectual contribution is supposed to be. As mentioned in the characteristics of case study research, the range of potential intellectual contributions is rather large. Basically, case study research can be used for any of the research purposes introduced in Sect. 1.2. It contributes to creating, elaborating, or even

162

8  Single Case Research Design

confirming (to a limited degree) theories. Thus, the research aim needs to make very clear what your aim within this plethora of possibilities is. This aim will inevitably guide some of your decisions and interpretations. So, the reader needs to be fully informed about it. Ultimately, you make use of the unique characteristic of case research- a comprehensive and rich description of a case-to understand something (better). Typical research question Try to phrase your research question with „how “ or „why “ as interrogative to meet readers’ expectations. Make the research question as precise as possible and include an incisive definition of your unit of analysis. Typical research questions of in single case research are a variation of: • how does this [process / system] work in [unit of analysis]? • how did/does [unit of analysis] address an [issue]? • why does the [unit of analysis] behave this way? The terms in brackets need to be further specified as they can encompass basically anything. With [process / system] we mean for example: • any process like product development, strategic planning, etc., and • any system like leadership, management, innovation, etc. By [unit of analysis], we mean, for example: • • • • • •

an organization (company, association, NPO, etc.), a department in an organization, a team, a group of people, an industry, or a country.

By [issue] we mean the following. • any problem (fraud, economic crises, succession planning, changing laws, etc.), • any topic (gender quality, inclusion, environment protection, branding, risk awareness, etc.). Method section A case study’s method section looks (on an aggregated level) rather similar to any other method section as it describes sampling (on two levels: your unit of analysis and the data sources), data collection, data analysis and triangulation.

8.3  Writing a Single Case Research Paper

163

But in contrast to many other research designs, the sequence of planning and executing these steps is not linear but iterative. Thus, start with the steps you already know that you conduct them for sure. For example, you conduct interviews with four interviewees, you analyze marketing meeting minutes, and you take part in two marketing meetings. Then, you should list options you only execute if necessary (e.g., optional interview with interviewee XYZ, study of marketing outputs like market report, market study, product requirements, study of scheduled meetings of marketing personnel in the last six months, and quantitative analysis of sales of products introduced in the last two years). This optional list should help you to collect ideas. Hence, it is adaptable. Add new options as they arise. After having concluded the case study, you can summarize the options as not executed because of their limited additional contribution and time reasons. Establish (and adapt) a guideline for your case research design (Yin 2004). This guideline, like an interview guideline, lists the information you would like to get. This is the driving factor of your sampling, data collection and analysis and the yardstick, whether you should execute one, several or all the options gathered. The fundamental question here is what you need to know about the unit of analysis (so you can answer the research question). For example, if your research question is “why did the product launch fail?” you might want to know how the standing of your marketing department in the organization is. This is the information you would like to get. Basically, this is based on alternative explanations. You may ask this the Chief Marketing Officer and the Chief Executive Officer directly. You can ask them with whom they talked during the product development and launch phase. Another suggestion is to assess the calendar entries all involved personnel. Also, you can. • • • • • • • •

analyze the meeting minutes, record meetings and compare the recordings with the minutes, count the actual talking time of the persons in the meeting, count how often the marketing manager was interrupted and establish a ranking based on interruptions, observe the body language of the involved persons when speaking to each other, compare the market requirement specification with the realized product features, compare the market requirement specification with the competitor’s product features, and do all the above or any subset.

Usually, you start with a limited collection of methods that prevent biases and misinformation (what do you think the CEO tells you about the marketing officer (CMO)? Maybe that the CMO is not that competent, and the CEO should have dismissed him three years ago?). Thus, a major part of your method section should be.

164

8  Single Case Research Design

• a list of potential answers, partial answers, or contributors to answers to your research question based on all (including alternative) explanations you and your fellow researchers according to your literature can think of, • a list of the information you would like to get so you can decide which explanation or combination of explanations answers the research question. You should always be open to new explanations that might come up during your research, • a list of data sources, data collection and analysis methods to get all or part of this information (n:m relation) and ensure that the results are largely unbiased, divided into • methods you will apply • methods you might additionally apply if the results so far are inconclusive • triangulation methods for reaching a result from the application of several methods, and • a method how to evaluate the contribution of each (potentially interrelated) partial answer to the research question. For example, if the market analysis has only been partly correct and nobody listens to marketing anyway. Please keep in mind: we still apply methods to accomplish results (i.e., we do not yet interpret these results). Results The result of case study research is a rich and comprehensive case description. Writing such a description is difficult. On the one hand, you have the results of the different data collection and analysis methods, for example interviews and their analysis using coding. Please note that we have already addressed how to deal with issues of data interpretation, a particularly relevant topic here (see reflexivity in Sect. 1.4). On the other hand, you need to present the triangulation of the results of individual methods. Disclose contradictory results. For example, it is important to understand that different people voiced different concerns about the same topic or that results from different methods of analysis deviated from each other. Discussion The discussion of the comprehensively described case is tremendously important. Your aim was to understand something (better). Here, you interpret the case description about its impact on your understanding. Because of the subjective nature of the interpretation (on several levels) careful phrasing is called for. Presenting alternative explanations (for example, but certainly not limited, to the specific context, the methods used, the not yet holistic picture) helps to provide a well-adjusted picture. Also, part of the discussion is the comparison with existing studies and theory. You discuss the impact of the improved understanding. For example, could the newly detected element X explain additional variation in the observed phenomenon Z? Potential future research often includes replication and elaborated categorization (e.g., in multiple case research).

8.4  Related Research Designs

165

Conclusion The typical conclusion drawn from case study research is the relevance of (additional) elements, conditions, and (causal) relationships for understanding a case comprehensively, addressing theoretical considerations (e.g., theory elaboration or theory creation). Phrase your conclusions carefully (do not overclaim).

8.4 Related Research Designs In this section, we briefly describe or cross-reference research designs that are similar to the single case research design. They might partially overlap with or considered to be adjacent to single case research or in fact be a single case research design that has its own label in the literature. If the single case research design does not fully meet your intentions and expectations, look here for direction where to continue your search. Design science research If the focus is predominantly on creating a (new) solution and not on understanding how an existing solution works or why it generates the actual results, design science research (Chap. 6) might be more appropriate alternative. This shift also entails a prescriptive approach where developing criteria for the evaluation gain importance. This either requires knowledge about relevant criteria from existing research or needs to be included in the research itself. Action research In action research (Chap. 7), the focus is on implementation and change management. A shift to action research stills entails understanding the system, processes and contexts but allocates a more active (in the sense of change driving) role to the researcher (or another party, for example, the CEO of the company). This shift is only workable if the client looks for (or can be convinced to want) change and implementing a solution. It might also entail a cyclical approach as the desired system behavior might not be achieved in the first round. Multiple case research Whereas single case research emphasizes the holistic understanding of one unit of analysis, multiple case research focuses on similarities and differences between units of analysis. This comparison and categorization of similar and differing elements, conditions and relationships might be conducted either based on several single cases or as a staggered design. In this case, the single case studies need to be available. Or the data collection and analyses of each unit of analysis might be part of the multiple case research. If so, data collection and analysis of the single units is more focused because of theoretical guidance in the search for these similarities and differences (see Chap. 9).

166

8  Single Case Research Design

A side note to grounded theory In some books on research, you find an approach called “grounded theory”. In our opinion, it is not a standalone research design, but very similar to a case study research design. There are nuances as in many research designs or sub-designs. However, those nuances do not justify separate description as a research design. We like to provide you with a general understanding what grounded theory is about. Grounded theory could be defined as „the discovery of theory from data systematically obtained from social research (Glaser & Strauss, 1967). The aim of grounded theory is theory creation using data about the social context (Glaser & Strauss, 1967). Grounded theory is often labeled a method or a methodology. It is defined as a “qualitative research method that uses a systematic set of procedures to develop an inductively derived grounded theory about a phenomenon” (Strauss 1990, p. 24). We agree with Glaser’s “Grounded Theory Institute” that it is rather the systematic generation of theory from systematic research, using a set of rigorous research procedures leading to the emergence of conceptual categories. In our terminology we would consider grounded theory a research design containing a specific set of decisions. But these decisions are within the scope of the single case research design (see Halaweh, 2012 for a more detailed comparison and delineation). Grounded theory is not a unified concept. It was originally developed by Glaser and Strauss (1967). Strauss and Corbin (1990, 1998) developed this initial version further, but this elaborated version has been criticized by Glaser. The differences between the Strauss and Glaser lead to the so-called Glaserian approach and the Straussian approach to grounded theory (Hekkala, 2007). As grounded theory is entirely based on induction, coding is the key process (Strauss and Corbin 1990). Grounded theory uses two methods to code. The first is continuous comparative analysis of data collected resulting in the identification of concepts and categories and their properties. This requires a highly sensitive researcher who has gained this sensitivity by experience in the respective field and added to it from a literature review (Strauss and Corbin 1990). The other method entails reflecting the labeled concepts (e.g., event, idea, action, and incident) with the participants by asking what this label means or represents. Because of the inductive approach, it is quite difficult to phrase a precise research question in grounded theory. It basically boils down to some version of “what is happening here” (Dick, 2005) and requires some flexibility for later adjustments. Ng (2005) for example, followed the suggestion from Glaser (1998) and asked initially for the “main concerns”. The recommendation for rather unexperienced researchers is to start with a general subject or problem of the respective discipline (Dey, 1999), such as: “what are your major concerns regarding your costing system?” Ng (2005), again following Glaser (1978), used a causal model to develop supporting and clarifying interviewing questions. Please note that it is rather difficult to follow an entirely inductive approach as the development of part of the questions are guided by a preconception or an implicit or

8.4  Related Research Designs

167

explicit theory. Some of that might be eased by a careful phrasing like (from an example about the relationships of a distributor with its clients) “tell me your experience working as the distributor for this organization. How would you improve the relationship? What would you like to see more of in the relationship?” (Ng & Hase, 2008). As with single case research, the aim of grounded theory is creating and elaborating theories that are generalizable. Their applicability to other social contexts is based on abstraction (Strauss and Corbin 1990), quite like Yin’s (1994) “analytic generalization”. Key Aspects to Remember

A case study research design should be only your choice if no other research design is appropriate. A case study research design is used to describe and explain a complex social phenomenon - answering a “how” or “why” research question comprehensively in its specific context. Conducting three to five interviews is most often not adequate to grasp a complex social phenomenon nor its environment nor enable you to describe and explain the unit of analysis comprehensively. A comprehensive description and explanation are potentially open ended. When are a description and explanation comprehensive? Only when you fully understand the social phenomenon and its interdependencies with its environment, so probably never. It requires a vast amount of know-how and experience to argue conclusively that your description and explanation are comprehensive. A at least partially appropriate way to achieve this is to gather vast amounts of data, cross-check them and show that including this additional information does not change the description and the explanation. A comprehensive description and explanation should not be based on one type of data. Apart from the amount of data, the category of data and the number and types of sources used distinguish an excellent case study. It is self-evident that one type of data might not yield a comprehensive, multi-perspective picture. Thus, start with as many types of data and sources as you can imagine. A single case research design is not the same as researching a “case”. Everything can be considered a “case” in most languages and certainly in English: a product, a patient, a business, an industry, a country, a currency, an ethnicity, a social group etc. Researching such a “case” does not make your research design a case study. Case study is a specific research design geared to understand complex social phenomena and to answer “how” and “why” questions and should be used for researching your “case” only if the argument you intend to make requires this research design.

168

8  Single Case Research Design

Critical Thinking Questions

1. Why does case study research not employ typical methodologies? 2. What is meant by “logical generalization” as part of the intellectual contribution? 3. What major challenges do you face when applying a single case study research project? 4. How would you address the basic concern about poor validity and reliability of case study research? 5. How does the single case study research design differ from the action research design?

Recommendations for further Readings

If you are still unsure whether single case research design is suitable for your research project, you might find the following literature and readings helpful. • Baxter, P. & Jack, S. (2008). Qualitative Case Study Methodology: Study Design and Implementation for Novice Researchers. The Qualitative Report 13 (4), pp. 544–559. • de Vaus, D. A. (2010). Research design in social research. Reprinted. Los Angeles: SAGE. • Yin, R. K. (2014). Case study research. Design and methods. 5. edition. Los Angeles: SAGE. • Miles, M. B, Huberman, M. A. & Saldana, J. (2019). Qualitative Data Analysis – A Methods Sourcebook. Los Angeles: SAGE. • Willis, B. (2014). The Advantages and Limitations of Single Case Study Analysis. Retrieved 10 June 2021 from https://www.eir.info/pdf/50706.

References Baškarada, S. (2014). Qualitative case studies guidelines. The Qualitative Report, 19(40), 1–25. Berg, B., & Lune, H. (2012). Qualitative research methods for the social sciences. Pearson. Bryman, A. (2004). Social research methods (2nd ed.). Oxford University Press, 592. Burns, R. B. (2000). Introduction to research methods. United States of America. Creswell, J. W. (2013). Qualitative inquiry and research design. Choosing among five approaches (3rd ed.). SAGE. Darke, P., Shanks, G., & Broadbent, M. (1998). Successfully completing case study research: Combining rigour, relevance and pragmatism. Inform Syst J, 8(4), 273–289. Dey, I. (1999). Grounding grounded theory: Guidelines for qualitative inquiry. Academic Press. Dick, B. (2005). Grounded theory: A thumbnail sketch. Retrieved 11 June 2021 from http://www. scu.edu.au/schools/gcm/ar/arp/grounded.html.

References

169

Dooley, L. M. (2002). Case study research and theory building. Advances in Developing Human Resources, 4(3), 335–354. Edmonds, W. A., & Kennedy, T. D. (2012). An applied reference guide to research designs: Quantitative, qualitative, and mixed methods. Thousand Oaks, CA: Sage. Edmondson, A. & McManus, S. (2007). Methodological fit in management field research. The Academy of Management Review, 32(4), 1155–1179. Eisenhardt, K. M. (1989). Building theories from case study research. Academy of Management Review, 14(4), 532–550. Glaser, B., & Strauss, A. (1967). The discovery of grounded theory: Strategies for qualitative research. Sociology Press. Flynn, B. B., Sakakibara, S., Schroeder, R. G., Bates, K. A., & Flynn, E. J. (1990). Empirical research methods in operations management. Journal of Operations Management, 9(2), 250–284. Flyvbjerg, B. (2006). Five misunderstandings about case-study research. Qualitative Inquiry, 12(2), 219–245. General Accounting Office (1990). Case study evaluations. Retrieved May 15, 2021, from https:// www.gao.gov/assets/pemd-10.1.9.pdf. Gomm, R. (2000). Case study method. Key issues, key texts. SAGE. Halaweh, M. (2012). Integration of grounded theory and case study: An exemplary application from e-commerce security perception research. Journal of Information Technology Theory and Application (JITTA), 13 (1). Hancock, D., & Algozzine, B. (2016). Doing case study research: A practical guide for beginning researchers (3rd ed.). Teachers College Press. Hekkala, R. (2007). Grounded theory—the two faces of the methodology and their manifestation in IS research. In Proceedings of the 30th Information Systems Research Seminar in Scandinavia IRIS, 11–14 August, Tampere, Finland (pp. 1–12). Hyett, N., Kenny, A., & Dickson-Swift, V. (2014). Methodology or method? A critical review of qualitative case study reports. International Journal of Qualitative Studies on Health and WellBeing, 9, 23606. Keating, P. J. (1995). A framework for classifying and evaluating the theoretical contributions of case research in management accounting. Journal of Management Accounting Research, 7, 66. Levy, J. S. (2008). Case studies: Types, designs, and logics of inference. Conflict Management and Peace Science, 25(1), 1–18. Meyer, J.-A., & Kittel-Wegner, E. (2002). Die Fallstudie in der betriebswirtschaftlichen Forschung und Lehre. Stiftungslehrstuhl für ABWL, insb. kleine und mittlere Unternehmen, Universität. Mitchell, J. C. (1983). Case and situation analysis. The Sociological Review, 31(2), 187–211. Ng, Y. N. K. & Hase, S. (2008). Grounded suggestions for doing a grounded theory business research. Electronic Journal on Business Research Methods, 6(2). Ng. (2005). A principal-distributor collaboration moden in the crane industry. Ph.D. Thesis, Graduate College of Management, Southern Cross University, Australia. Ridder, H.-G. (2016). Case study research. Approaches, methods, contribution to theory. Sozialwissenschaftliche Forschungsmethoden (vol. 12). Rainer Hampp Verlag. Ridder, H.-G. (2017). The theory contribution of case study research designs. Business Research, 10(2), 281–305. Maoz, Z. (2002). Case study methodology in international studies: from storytelling to hypothesis testing. In F. P. Harvey & M. Brecher (Eds.). Evaluating methodology in international studies. University of Michigan Press. May, T. (2011). Social research: Issues, methods and process. Open University Press/Mc. Merriam, S. B. (2009). Qualitative research in practice: Examples for discussion and analysis.

170

8  Single Case Research Design

Onwuegbuzie, A. J., Leech, N. L., & Collins, K. M. (2012). Qualitative analysis techniques for the review of the literature. Qualitative Report, 17(56). Piekkari, R., Welch, C., & Paavilainen, E. (2009). The case study as disciplinary convention. Organizational Research Methods, 12(3), 567–589. Stake, R. E. (1995). The art of case study research. Sage. Stake, R. E. (2005). Qualitative case studies. The SAGE handbook of qualitative research (3rd ed.), ed. N. K. Denzin & Y. S. Lincoln (pp. 443–466). Strauss, A. L., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Sage publications. Strauss, A. L., & Corbin, J. (1998). Basics of qualitative research techniques and procedures for developing grounded theory. Sage. Tight, M. (2003). Researching higher education. Society for Research into Higher Education; Open University Press. Tight, M. (2010). The curious case of case study: A viewpoint. International Journal of Social Research Methodology, 13(4), 329–339. Walsham, G. (2006). Doing interpretive research. European Journal of Information Systems, 15(3), 320–330. Welch, C., Piekkari, R., Plakoyiannaki, E., & Paavilainen-Mäntymäki, E. (2011). Theorising from case studies: Towards a pluralist future for international business research. Journal of International Business Studies, 42(5), 740–762. Woods, M. (2009). A contingency theory perspective on the risk management control system within Birmingham City Council. Management Accounting Research, 20(1), 69–81. Yin, R. K. (1994). Discovering the future of the case study. Method in evaluation research. American Journal of Evaluation, 15 (3), 283–290. Yin, R. K. (2014). Case study research. Design and methods (5th ed.). SAGE.

9

Multiple Case Research Design

Learning Objectives

When you have finished studying this chapter, you will be able to: • understand the purpose of multiple case research: in-depth analysis of a small sample in its environmental context • explain why there is a trade-off between in-depth analysis and sample size • embrace the contextual conditions as part of the research process • understand that multiple case research is based on non-random sampling • understand that this design focuses on differences and similarities between cases

Multiple case research

• • • • •

in-depth analysis of a small sample in its environmental context trade-off between in-depth and size of sample contextual conditions are part of the research process non-random sampling of cases focuses on differences and similarities between cases

© The Author(s), under exclusive license to Springer Fachmedien Wiesbaden GmbH, part of Springer Nature 2021 S. Hunziker and M. Blankenagel, Research Design in Business and Management, https://doi.org/10.1007/978-3-658-34357-6_9

171

172

9  Multiple Case Research Design

9.1 General Description of Multiple Case Research Design We see potential advantages of multiple case research in the cross-case analysis. A comprehensive comparison in cross-case analysis reveals similarities and differences and how they impact findings. Every case is analyzed as a stand-alone case to compare the mechanisms identified, contributing to potential theoretical conclusions (Vaughan, 1992). Case study research is a means of advancing theories by comparing similarities and differences among multiple cases (Ridder, 2017). Regarding the number of cases examined, we distinguish between single case studies and multiple case studies (Stake, 2005). A single case can be, for example, the operation of one drug-rehab clinic. A multiple case might involve looking at several drug-rehab clinics operating in the health care industry. Individual cases may represent criticality, extremeness, uniqueness, representativeness, and typicality. Individual case studies are useful to examine conflicting theoretical findings. Also, they serve as a basis to gain new insights into an unexplored phenomenon. With a multiple case study, researchers conduct an in-depth analysis of several cases. First, an investigation of individual cases is conducted. Later, these individual results are combined. Researchers try to find similarities and differences. A major advantage of multiple case studies over individual case studies is that researchers can compare their findings. Thus, the results of multiple case research are more robust but might not be as detailed as in single case research. The major disadvantage compared to single case research is that they tie up more resources and are more costly if they try to achieve a similar depth of analysis. Sometimes, a comparative analysis is not possible because of a lack of comparable cases. Yin (2014) differentiates not only between single and multiple case research. He adds two further distinctions “holistic” (a comprehensive analysis of a system) and “embedded” (the unit of analysis part of another system, e.g., the nursery within a hospital). Yin (2014) suggests five rationales for conducting only single case research, that is for critical, extreme, typical, revelatory, or longitudinal cases. Thus, we cannot provide a general answer to whether a single case study or multiple case study is preferable. This always depends on the specific aim, the cases, and the resources. We distinguish several units of analysis within a case. This allows the object of analysis to be analyzed from different perspectives. We conclude that there is no ideal number of cases. It always depends on the research question, the resources, and the accessibility of the cases. Yet, a multiple case research design may lead to more robust results. This specifically holds true in inductive theory building (Eisenhardt & Graebner, 2007). We believe that gaining access to suitable cases (e.g., companies of a specific industry) represents one of the most challenging steps in the entire research process (Walsham, 2006). Thus, looking for cases and checking accessibility is an important step that must be done before honing the research question(s). Yin states that in multiple case research, each case must be selected so that it predicts similar results (literal replication) or predicts contrasting results but for

9.2 Particularities of Multiple Case Research Design

173

expected reasons (theoretical replication). If multiple cases lead to contradictory results, the preliminary theory must be revised and tested with other cases. Both single and multiple case designs can be holistic (one unit of analysis per case) or embedded (multiple units of analysis per case) (Yin, 2014). Opportunistic and convenient sampling

A general challenge in empirical studies is the access to a sufficiently large number of interview partners who reserve time for an interview (or other methods of data retrieval). To find companies for a research project, we can apply an opportunistic and convenient sampling strategy based on the approach of Bruns and McKinnon (1993): “the corporations taking part in our study comprise a non-random sample selected on the basis of location and accessibility, personal contacts and expected willingness to help with the research process” (p. 90). This obviously constitutes a sample bias, but at least the researcher can retrieve information. Thus, personal contacts and contacts through the researcher’s network can be used for the selection of the sample. Also, companies can be contacted by letter with the request for a possible interview. Finally, we may search interview partners publishing articles in relevant academic and professional journals. Purposive sampling. For example, a research project about risk management only considers non-financial companies for sampling purposes. This is reasoned as follows. Financial companies can be understood as so-called “risk management entities”. Their business activities and their stricter regulatory environment require significantly different risk management approaches. Therefore, they are not comparable to non-financial companies and are excluded from the sample. ◄

9.2 Particularities of Multiple Case Research Design In this section, we specifically address the elements that make a multiple case a discrete research design. Next to the characteristics of multiple case research, we address the main issues and decisions to be made within this research design, and the major pitfalls.

9.2.1 Characteristics of Multiple Case Research In this section, we elaborate on the key characteristics of multiple case research along the steps of the research process.

174

9  Multiple Case Research Design

Conclusion The typical conclusion of a multiple case study is that cases are similar or different from certain elements, relationships, and conditions. This might be expressed in a categorization of cases. The elements, relationships and conditions might need to be included in a general theory. Intellectual contribution The intellectual contribution of a multiple case study is establishing preliminary categories of elements, relationships and conditions and elaborating theories by including these categories (either by adding them to the theory or by differentiating existing elements, relationships, and conditions) by logical generalization. This might be condensed into establishing a testable theory. Argument The key argument in multiple case studies is that we can explain differences in phenomena by differences in elements, relationships, and conditions in different (categories of) cases. Hence, these differences are important for a comprehensive description and explanation of the cases and potentially for more or all cases. Results Results are similarities and differences, grouped into categories of elements, relationships, and conditions from rich and comprehensive case descriptions. Methods Methods involve mainly categorization and clustering (apart from the methods used in single case studies) and can be qualitative and quantitative. Data Data used are varied and from multiple sources that provide information about potential categories, either guided by theory or with no guidance. The data is basically the same as in single case studies, but the search might be more focused by theories or preconceptions about similarities and differences. All data necessary for an envisioned categorization need to be collected. Research question The typical research question of a multiple case study is “what are the similarities and differences or different categories of cases that might explain variations in a phenomenon?”.

9.2 Particularities of Multiple Case Research Design

175

9.2.2 Issues to Address in Multiple Case Research In detailing the research design, you face many multiple case research specific problems and decisions. We list the main options you have in the following. Multiple case as a research design on its own A multiple case research design shifts the focus from understanding a single case to the differences and similarities between cases. Thus, it is not just conducting another (second, third, etc.) case study. Rather, it is the next step in developing a theory about factors driving differences and similarities. Often case studies result from tackling research gaps left from models with unsatisfying explanatory power. The case study then tries to understand one phenomenon comprehensively, looking for elements that can contribute to explain the case. The multiple case research design aims at finding and establishing systematics or taxonomies to group and classify these elements. These classifications might be rather tentative, based on the similarities and differences between two cases or already approaching the operationalization of variables that can be used in the next generation of a testable model. Number of cases How many cases form a multiple case research design? We suggest anything between two and (a bit arbitrary) roughly 20. It rarely makes sense to examine over twenty cases as your approach sample sizes where applications of testable models become workable. This does not mean that quantitative methods are better. They are just used in distinct steps of gathering knowledge and establishing and refining theories. The number of cases is only limited by the in-depth analysis you can conduct. In-depth analysis of the cases Multiple case research is still case research as it tries to understand the cases comprehensively. However, as more empirical evidence emerges about which elements are (more) important for comprehensive understanding, targeting these elements becomes easier and more manageable. The better defined your theory about these elements is, the faster you can draw a comprehensive picture up to the point where you can define variables and proxies to measure them. If there are many unique elements and interrelationships between them, this might call for many cases to increase the likelihood of integrating many combinations. However, if you have a limited number of possible combinations of elements, then another research design (e.g., cross-sectional research) might yield additional arguments than just adding more of the same (e.g., there is no or only limited added value to analyze the 45th organization where a combination of X, Y and Z prevented the successful introduction of U).

176

9  Multiple Case Research Design

Theory about important elements Based on existing empirical research, you establish a theory about what elements of the unit of analysis or its environment are relevant in the sense that they contribute to the understanding of the system and the system’s behavior. Often you can be guided by the following question: would the system and its behavior be understandable without integrating this element? This theory guides your description of your cases. The more refined the theory is, the more you can focus on obtaining the proper information. Even in a single case study, your search for information is often driven by an underlying theory about the importance of elements. In the end, it is more a question of focus: the comprehensive description of a case (see single case study) or the establishing of a theory about the important elements for a comprehensive description (multiple case study). Data analysis methods Usually, we do not refer to specific methods as they are well known and better described elsewhere, we like to stress the possibility to use “qualitative” and “quantitative” methods in various research methods. In multiple case research analysis, this looks like a spurious claim. In fact, the qualitative comparative analysis (QCA) straddles this chasm (Ragin, 2009). Qualitative comparative analysis uses categorical variables and the respective n-tuples to classify units of analysis. The potential combinations are then compared to the actual observations. In a second step, inferential logic or Boolean algebra is used to reduce the number of relationships. So, you can arrive at the minimum set of necessary and sufficient conditions to predict values of a category. This method was specially developed to study samples that are too small for linear regression. Qualitative comparative analysis fits nicely with the purpose of multiple case research designs. We like to use this example as teaser to encourage you to look for appropriate new methods.

9.2.3 Major Fallacies in Conducting Multiple Case Research While providing guidance and support for research projects, these are the major pitfalls students encounter in their multiple case research projects. Multiple cases as an excuse to forego in-depth analysis Many studies use a multiple case study research design to solve insufficient access to a single case. As more cases means not as much in-depth analysis, perhaps even only a single interview. This is a fallacy as this only works if you already know very well what information you need to gain a comprehensive picture. Only conducting ten interviews to “have” ten cases is not enough. First you need to start with a very good guide (based on empirical evidence) about the information you like to get. Second, you need to verify the information, making sure that the information pertains to the case and is not just the opinion of the interviewee.

9.3  Writing a Multiple Case Research Paper

177

Jumping the chain with insufficient evidence The feasibility of conducting successful multiple case research is very much depending on the availability of empirical evidence and the research gap left by this evidence. Jumping from single case research as empirical evidence to multiple case research with about 20 cases rarely works. This is because the single case rarely offers enough evidence to establish a theory about the relevance of elements and their interdependencies. It is usually better to introduce a multiple case research project with perhaps three to eight cases as an intermediate step. Staggered design versus multiple case research It is difficult to differentiate between multiple case research and a staggered design comprising (several) single cases followed by a multiple case research design. The (one stage) multiple case research design would limit or prune the data collection and the analysis of the cases to specific categories based on existing theory. The staggered design, on the other hand, uses fully fledged single case research (first stage) as input for categorization and comparison in the multiple case research (second stage). For example, Yin (2014) states that a multiple case study basically means conducting case studies and a comparison of the cases. In fact, we consider this a staggered design combining different research designs to answer different questions. We first conduct case study research and then we look for the similarities and differences (i.e., the relevant elements that describe the cases sufficiently). Not realizing the staggered design leads to confusion regarding the research question and lacks the required comprehensiveness of the in-depth analysis.

9.3 Writing a Multiple Case Research Paper Writing a multiple case research paper follows the principles and structure detailed in Chap. 4. However, there are some aspects especially important for reports about multiple case projects or (partly) different from reports about other research projects. We address these idiosyncrasies following the standard structure of a scientific paper. Introduction Introducing a multiple case research means to clarify the focus of the research and its reasoning. Often a multiple case research design acts as a bridge between single case research and cross-sectional (or longitudinal) research. Their initial set-ups or starting points are rather clearly defined. However, regarding the multiple case research, it is important that we delineate which part of the bridge–from where to where–our research design establishes. This is in line with the general purpose of expectation management in the introduction. In multiple case research, the purpose is usually theory elaboration. What will be elaborated? What is the starting point? This needs to be defined in the introduction. The outline of the preliminary aims affects the expectations for the theoretical background and the literature review.

178

9  Multiple Case Research Design

Theoretical background Here, you describe the theory that is elaborated in your research project. At one extreme of the theory continuum, there is no theoretical foundation available (apart from some underlying grand theories, see Sect. 2.3.1). This goes together with the research aim to identify similarities and preliminary categories. At the other extreme, there might be a rather well-developed theory lacking only nuances to develop testable hypotheses. Then the research aim is to define constructs and categories and to confirm the construct and instrument validity. As mentioned, these are the two extremes. So, anything in between is also possible. This makes a detailed guidance on this section impossible, apart from the necessity to check carefully for consistency between the theoretical background and the research objective. Any mismatch here seriously impairs the intellectual contribution. Literature review The same holds true for the literature review. The literature review of all multiple case research studies comprises no or only few studies that aim to generalize a theory. And those few usually feature a low power of explanation. Here, you demonstrate that no or only limited studies exist about your research topic. There is a large continuum across existing case research that corresponds to the state of theory development. If there is little or no theory, we would expect only few case studies in the literature review. If the theory is already more developed, there are likely to be more case studies available that have already started to elaborate the theory by establishing categories for elements, conditions, and relationships. The third part of the literature review is optional and depends on your experience as a researcher. Talking about constructs and categories, you do not have to reinvent the wheel. There might be other research available that deals with similar issues in another context or refers to other phenomena that help you get ideas, concepts, and instruments for your own research projects. Typical research gap The typical research gap of multiple case research follows directly from the issues addressed in the theoretical background and the literature review: • the power of explanation of statistically testable models is too low, • there are no statistically testable models, • there is no or no satisfying theory about the system, elements, and relationships that can be tested or applied to specific types of systems or environments (cases), • there are no case studies, only few case studies, but there has been no theory established what elements are relevant for describing and explaining the system, or • there is only an initial idea (starting point) about a theory, but it is yet unclear about details and operationalization and can thus not yet be tested.

9.3  Writing a Multiple Case Research Paper

179

The extent (small to large) of the identified research gap has a tremendous impact on your research aim. Typical research aims The typical research aim of a multiple case research design is to contribute to establishing a (refined) theory by classifying and characterizing cases and their relevant elements and relationships. As already mentioned, the specifics and the extent of the aim depend on the existing theories, research, and the corresponding research gap. Typical research question Typical research questions of multiple case research design projects are: • what are the similarities and differences of [cases] about [dimension(s)]? • what are the relevant dimensions for describing [cases] about [problem or situation]? We can illustrate this further. [Cases] could, for example, mean one of the following. • organizations, • departments of organization, • teams, • systems, • process, • industries, or • countries, By [dimensions] we mean any kind of criteria or characteristics of the cases, for example: • size, • ownership structure, • management structure, • organization, • markets, • products and services, • revenue, • innovation process, or • degree of outsourcing. By [problem or situation] we mean any kind of condition that is relevant for the cases, for example:

180

• • • •

9  Multiple Case Research Design

during a pandemic, after a financial crisis, in a recession, or victim of a cyber-attack.

The difference between [dimension] and [problem / situation] is not clearly cut, which in our opinion is not an obstacle to multiple case research (i.e., you can largely ignore it). Method The sample needs to be described. As the focus is on similarities and differences between cases, the selection process needs to be described. Data collection methods also use an information guideline like the single case research, that needs to be developed and substantiated based on the theory so far. The data collection is basically the same as in single case research but tends to be more standardized to facilitate comparison and the larger sample. Data analysis methods are split in two parts. One part of the analysis methods deals with the data analysis of each case individually, which again is basically the same as in single case research. Yet, the multiple case specific analysis methods need to be described. Those refer to categorization and comparison, mainly operationalization of the establishing of categories (which categories and how to allocate to a category), and what constitutes being “similar” and “different” in each respect. Results As the differentiation of the analysis methods, the results section covers the results of the individual cases, the results of the case comparison, and the case categorization. You present the rich individual case descriptions (probably not as comprehensive as compared to single case research designs) providing a complete picture about the relevant dimensions. The less standardized and more comprehensive the individual case descriptions are, the more important they become. The results on the level of categorization and comparison should differentiate in three parts: • identifying criteria for the comprehensive description of the units of analysis, • comparing the units of analysis regarding these criteria, and • comparing and clustering the units of analysis. The relative importance and meaningfulness of the results differ depending on the data collection (obviously, data collected to allow a categorization about certain criteria is less suited to identify additional criteria than, for example, single case research).

9.4  Related Research Designs

181

Discussion With qualitative data, there is already some interpretation involved in the data analysis. You discuss the results in this section. Here are some issues you must discuss, among others: • can the criteria sufficiently describe the units of analysis? • is the comparison of the units of analysis based on these criteria meaningful (in relation to the comparison of the entire units)? Do we understand differences in the units of analysis as differences in distinct criteria? Are these criteria-related differences sufficient for understanding the differences and the similarities between units of analysis? • can you identify clusters? Can you use distinct criteria for allocating units of analysis to clusters (categories)? • can you elaborate on the existing theory? Has the theory now reached a state where it might be tested? What lacks to arrive at a testable hypothesis? Referring to the picture of the bridge from “no theory at all to theory confirmation”, how far have you come? Conclusion The typical conclusion of a multiple case research is that the units of analysis are similar or different regarding certain elements, conditions, and relationships (summarized as criteria) and that the units of analysis can be grouped into similar and delineated from different clusters of cases, using certain criteria. An additional conclusion may be the elaboration of an existing theory with the addition, elimination or differentiation of certain elements, conditions and relationships and preparing theory testing by identifying testable hypotheses.

9.4 Related Research Designs In this section, we briefly describe or cross-reference research designs that are similar to the multiple case research design. They might partially overlap with or considered to be adjacent to multiple case research or in fact be a multiple case research design that has its own label in the literature. If the multiple case research design does not fully meet your intentions and expectations, look here for direction where to continue your search. Single case research Single-case research (see Chap. 8) draws on a comprehensive holistic picture of a phenomenon. The purpose is to understand “how” and “why” the unit of analysis behaves and acts in real life. This does not require an underlying theory (but it also does not preclude its existence). So, if there is no preconception about categories that denote

182

9  Multiple Case Research Design

similarities and differences, focus on one unit of analysis to understand the relevant elements, conditions, and relationships better. These results can-at a later stage (in a staggered design)-be used to compare these elements with those derived from other units of analysis. Cross-sectional field study A cross-sectional field study (CSFS) is a type of multiple case research and is within the peculiarities and issues of multiple case research designs. Cross-sectional field studies were introduced by Lillis and Mundy (2005) and intended to bridge the gaps between qualitative and quantitative research. We deem it worthwhile to present it in a bit more detail. The major characteristics and properties of cross-sectional field study are: • it uses a larger sample than (multiple) case studies with less in-depth data, usually retrieved by relatively shorter interviews, • it is better suited to deal with case typical “how” and “why”-questions than surveys (used in cross-sectional research) (Eisenhardt 1991), • it can provide a better understanding of “complex phenomena” than surveys (used in cross-sectional research), and • it may help to discover ambiguities or the need for additional differentiation or categorization in prior research. In short, it addresses the missing dialogue between “pure” case studies and cross-sectional (survey based quantitative) research. In line with our argumentation for the intellectual contribution of multiple case research, a cross-sectional field study tries to bridge case studies’ problems of generalizability with the insufficient explanatory power of cross-sectional research (high degree of unexplained variation of the dependent variable). The relation of these three designs can be depicted in Fig. 9.1.

Case Study

CSFS

Survey

exploring new areas, complex phenomea, importance of contextual issues

reduces complexity of phenomena, contextual issues of less importance

Limited access to «human experience», evaluate relations between phenomena

➔high complexity of phenomena

➔medium complexity of phenomena

➔low complexity of phenomena

Fig. 9.1  Main differences between case study, cross-sectional field study (CSFS) and cross-sectional study (Lillis & Mundy, 2005)

9.4  Related Research Designs

183

A cross-sectional field study attempts a compromise by combining somewhat “indepth” analysis with somewhat “large” samples of around 12 - 40 to identify categories and patterns. Cross-sectional field study allocates this sample size well beyond the sample sizes of multiple case studies and distinguishes between the two, which we, the authors of this textbook, do not (see Fig. 9.2). Arguing with Keating’s (1995) critique that theory elaboration (refinement) is neglected in literature, a cross-sectional field study aims to address especially this type of intellectual contribution. Based on existing, but not yet testable theory, the topic should be researched in more depth and the theory should be refined to allow for future theory testing (Ferreira & Merchant, 1992). This again is in line with our understanding of the purpose of multiple case research. Table 9.1 summarizes the suitability and the methods used in cross-sectional field study and clarifies each criterion with examples. Cross-sectional research

Cross-sectional research (see Chap. 10) focuses on generalizable observations at one point in time. If the number of potentially relevant variables is sufficiently low, the variables rather well defined (because of existing theories or studies), and the number of data sets rather large, you might consider a cross-sectional research. Key Aspects to Remember

Understand the advantages and disadvantages of multiple case designs A major advantage of multiple case research over individual case studies is that researchers can compare their findings. A systematic comparison by the means of a cross-case analysis reveals similarities and differences and how they affect findings. Thus, the results of multiple case studies are more convincing, trustworthy, and robust. Yet, the

high

single case research

depth of analysis

Fig. 9.2   Classification of research designs based on depth and sample size (Lillis & Mundy, 2005)

multiple case study (Lillis and Mundy)

multiple case research (Hunziker and Blankenagel)

cross-sectional field study (Lillis and Mundy)

cross-sectional research

low low

sample size

high

184

9  Multiple Case Research Design

Table 9.1  Suitability and the methods used in cross-sectional field study (adopted from Lillis and Mundy (2005)) Eligibility criteria of the Cross-Sectional Field Study

Example

There are already theoretical approaches to the phenomena under investigation. It is assumed, however, that existing theories do not cover all aspects of empirical phenomena

There are theoretical approaches to internal control and risk management, but there seems to be little empirical research on the coordination of instruments

It is reasonable to doubt that certain constructs or variables cannot be measured precisely in quantitative research

In particular, the terms and concepts of efficiency, the integration of internal control and risk management and the demarcation between internal control and risk management require in-depth analysis and are also interpreted differently in practice

Clearly defined research questions, with a relatively narrow field of research

It is primarily about the question of efficient internal control depending on context variables and design parameters

Sampling strategy that maximizes the probabil- Care should be taken to ensure that, on the ity of generating meaningful and comparable one hand, sufficiently large companies in the data non-financial industry are chosen which, on the other hand, differ sufficiently in ex ante defined dimensions (business field, size, diversification, etc.) Semi-structured interviews will be used. This allows on the one hand to generate comparable data between companies and on the other hand to collect extensive, more complex, and unexpected answers (“narrative data”)

It is planned to conduct semi-structured interviews

Data analysis should make it possible to identify and highlight patterns that cut across all cases. The data should be systematically analyzed. The link back to theory will be created

For example, the analysis of patterns will be based on the proven Miles and Huberman Matrix (1994). Software packages such as NUD*IST and Atlas.ti allow data coding and analysis of possible “patterns” between companies

disadvantage compared to single case studies is that they tie up more resources and are more costly. Do not confuse interviews with cases Many studies use a multiple case research design to solve insufficient access to a single case. Because of more cases they analyze each one not in as much depth, and perhaps even conduct only a single interview. This is a fallacy as this only works if you already know very well what information you need for a rather comprehensive picture. So just conducting ten interviews to “have” ten cases is not enough. First you need to start with

9.4  Related Research Designs

185

a very good guide (based on empirical evidence) about the information you would like to get and secondly you need to verify the information, making sure that the information pertains to the case and is not just the opinion of the interviewee. Differentiate between holistic and embedded cases In multiple case research designs, each case must be selected so that it predicts similar results (literal replication) or predicts contrasting results, but for expected reasons (theoretical replication). If multiple cases lead to contradictory results, the preliminary theory must be revised and tested with other cases. Both single and multiple designs can be holistic (one unit of analysis per case) or embedded (multiple units of analysis per case). Understand the intellectual contribution of multiple case research designs The intellectual contribution of a multiple case study is establishing preliminary categories of elements, relationships and conditions and elaborating theories by including these categories (either by adding them to the theory or by differentiating existing elements, relationships, and conditions) by logical generalization. This might be condensed into establishing a testable theory.

Critical Thinking Questions

1. Is there an ideal number of cases in multiple case research designs? 2. Why is an interview with a company representative not a case? 3. What major challenges do you face when applying multiple case research design? 4. Why is this research design sometimes used as an excuse to forego in-depth analysis? 5. How does a multiple research design produce intellectual contributions?

Recommendations for further Readings

If you are still unsure whether multiple case research design is suitable for your research project, you might find the following literature and readings helpful. • Creswell, J. W., & Poth, C. N. (2017). Qualitative inquiry and research design: Choosing among five approaches. Los Angeles, CA: Sage. • Günes, E., & Bahçivan, E. (2016). A multiple case study of preservice science teachers’ TPACK: Embedded in a comprehensive belief system. International Journal of Environmental and Science Education, 11(15), 8040–8054.

186

9  Multiple Case Research Design

• Keating, P. J. (1995). A framework for classifying and evaluating the theoretical contributions of case research in management accounting. In Journal of management accounting research 7, p. 66–86. • Lillis, A. M. & Mundy, J. (2005). Cross-Sectional Field Studies in Management Accounting Research—Closing the Gaps between Surveys and Case Studies. In Journal of management accounting research 17 (1), pp. 119–141. • Ridder, H-G. (2017). The theory contribution of case study research designs. In Bus Res 10 (2), pp. 281–305. • Williams, J. J. & Seaman, Alfred E. (2002). Management accounting systems change and departmental performance: the influence of managerial information and task uncertainty. In Management Accounting Research 13 (4), pp. 419–445. • Yin, R. K. (2014). Case study research. Design and methods. 5. edition. Los Angeles, London, New Delhi, Singapore, Washington, DC: SAGE.

References Bruns, W. J., & McKinnon, S. M. (1993). Information and managers: A field study. Journal of Management Accounting Research, 5, 84–108. Eisenhardt, K. M., & Graebner, M. E. (2007). Theory building from cases: Opportunities and challenges. Academy of Management Journal, 50(1), 25–32. Ferreira, L. D. & Merchant, K. A. (1992). Field research in management accounting and control: A review and evaluation. Emerald Group Publishing Limited. Keating, P. J. (1995). A framework for classifying and evaluating the theoretical contributions of case research in management accounting. Journal of Management Accounting Research, 7, 66–86. Lillis, A. M., & Mundy, J. (2005). Cross-sectional field studies in management accounting research—closing the gaps between surveys and case studies. Journal of Management Accounting Research, 17(1), 119–141. Ragin, C. C. (2009). Reflections on casing and case-oriented research (pp. 522–534). The Sage handbook of case-based method. Ridder, H.-G. (2017). The theory contribution of case study research designs. Business Research, 10(2), 281–305. Stake, R. E. (2005). Qualitative case studies. In N.K. Denzin & Y.S. Lincoln (Eds.), The SAGE handbook of qualitative research (3rd ed., pp. 443–466). Vaughan, D. (1992). Theory elaboration: The heuristics of case analysis. What is a case?. In C.C. Ragin & H.S. Becker (Eds.), Exploring the foundations of social inquiry (pp. 173–202). Cambridge University Press. Walsham, G. (2006). Doing interpretive research. European Journal of Information Systems, 15(3), 320–330. Yin, R. K. (2014). Case study research. Design and methods (5th ed.). SAGE.

10

Cross-Sectional Research Design

Learning Objectives

When you have finished studying this chapter, you will be able to: • understand the purpose of cross-sectional research, as it aims for generalizable statements about elements, relationships, and conditions • explain the three distinctive features of this design: no time dimension, a focus on existing differences rather than change following an intervention, and groups are chosen based on existing differences rather than random allocation • deal with major fallacies in conducting cross-sectional research

Cross-sectional research

• • • •

aims for generalizable statements about elements, relationships, and conditions focus on one point in time samples data that allows inference for the population statistical analysis whether the observed sample might be a random occurrence

10.1 General Description of Cross-Sectional Research A cross-sectional research aims at describing generalized relationships between distinct elements and conditions. For example, “the higher R&D expenditure the lower the profit in manufacturing companies in Europe”. The specific case and its particularities are not the focus, but all instances and cases. So cross-sectional research tries to establish © The Author(s), under exclusive license to Springer Fachmedien Wiesbaden GmbH, part of Springer Nature 2021 S. Hunziker and M. Blankenagel, Research Design in Business and Management, https://doi.org/10.1007/978-3-658-34357-6_10

187

188

10  Cross-Sectional Research Design

general models that link a combination of elements with other elements under certain conditions. The results are tested (or rejected) theories about these relationships. Cross-sectional research designs have three distinctive features: • observation refer to one point in time, • a focus on existing differences rather than change following an intervention, and • groups are selected based on existing differences rather than random allocation (for example gender or income bracket). The cross-sectional design can measure differences between or from among a variety of people, subjects, or phenomena rather than a process of change. Using this research design, you can only employ a relatively passive approach to drawing causal inferences based on findings (USC, 2021). The cross-sectional research design has several advantages (Allen, 2017; Lauren, 2020). Especially the first advantage makes them rather appealing for student’s research papers: • as they focus on one point in time, they can be carried out rather efficiently. • multiple variables can be investigated together. For example, gender, age, income, salaries, access to services or company size, industry, R&D expenditure, and profit. • cross-sectional research allows researchers to collect data from large data sets and compare differences between groups. However, they also have several disadvantages (Lauren, 2020): • cross-sectional research can identify relationships between variables. However, they cannot establish cause-and-effect relationships, as they only represent one measurement of the hypothesized causes and effects. • as cross-sectional research analyzes a single moment in time, they cannot investigate behavior over time or changes like trends. The researcher cannot know if the results would be different at another point in time. • they are prone to participation and sampling biases, or low response rates from participants. Cross-sectional research designs often collect data conducting surveys or structured interviews involving human respondents as the primary units of analysis. In such research designs, research questions or hypotheses are proposed. Yet, the primary goal should be to describe a population of interest or compare subgroups of that population across a set of measures (Allen, 2017). Although most cross-sectional studies are quantitative, cross-sectional research can also use qualitative or mixed methods. Quantitative cross-sectional research designs

10.2 Particularities of Cross-Sectional Research

189

use data to make statistical inferences about the population of interest or to compare subgroups within a population, while qualitative-based research designs focus on interpretive descriptive accounts of a specific sample. Mixed-method designs address aspects of both quantitative and qualitative approaches. Cross-sectional designs are used in many social scientific fields, such as medical research and economics (Allen, 2017).

10.2 Particularities of Cross-Sectional Research In this section, we specifically address the elements that make cross-sectional a discrete research design differentiated from others. Next to the characteristics of cross-sectional studies, we address the main issues and decisions to be made within this research design, and the major pitfalls.

10.2.1 Characteristics of Cross-Sectional Research In this section, we elaborate on the key characteristics of cross-sectional research designs along the steps of the research process. Conclusion The typical conclusion of a cross-sectional study is the existence or non-existence of non-random relationships between certain elements under certain conditions. Changes in certain elements and conditions appear together with changes in other elements. Intellectual Contribution The intellectual contributions lie in the inclusion, exclusion, or categorization of elements and conditions that have a relationship with other elements, the direction of the relationship (for example “the bigger X, the bigger Y” or “category C had bigger or smaller Y”), and ultimately by explaining a larger part of the variation of the phenomenon (the “other” elements), by determining impact sizes and significance level. Argument The key argument is that based on the data analyzed, there is a probability that there is a relationship between certain elements under certain conditions (significance), that the elements probably affect each other with a certain effect size and that all analyzed elements, relationships, and conditions probably explain a part of the variation in other elements, i.e., the phenomenon (explanatory power).

190

10  Cross-Sectional Research Design

Results The results are the elements and conditions that probably correlate or do not correlate with the researched phenomenon, their significance, their effect sizes, and the explanatory power of the model as a combination of the elements and conditions. Methods The method is usually a statistical analysis of a model containing the elements and conditions as dependent and independent variables and their relationships. The model is fitted to the data and probabilities are calculated. Data Data retrieved and analyzed are quantitative and categorical data in complete data sets, drawn for a certain, usually quite large, sample (so that the sample allows the application of statistical analysis). Research question The typical research question is “what is a suitable model to depict generalizable relationships between elements under conditions?”.

10.2.2 Issues to Address in Cross-Sectional Research In detailing the research design, you face many cross-sectional research specific problems and decisions. We list the most important ones and the main options you have in the following. In cross-sectional research, many issues refer to the model you use to describe relationships. If you establish a model, three characteristics describe the quality of the model: significance, impact size and explanatory power. Significance Statistical significance basically represents the likelihood that the observed data might be observed in the absence of the assumed relationship, so basically, if it is just coincidence. Significant results are results with a very high likelihood of being more than just coincidence. Usually, researchers use three significance levels, i.e., 90%, 95% and 99%, with 99% being the highest significance (i.e., with the highest probability of the results being more than just coincidence). Impact size of elements When establishing a model, not only the significance of the elements is important, but at least as much the impact or impact size of the elements is. Is a miniscule change in that element related to a significant change in another element or not?

10.2  Particularities of Cross-Sectional Research

191

Imagine you would like to introduce a weight loss pill. What would be the better or more relevant result? That participants in the study lost 2 kg weight over the duration of 12 months with a statistical significance of 95% or that the participants lost 20 kg of weight over the duration of 12 months with a statistical significance of 70%? We have a significant model (95%), but only very limited impact size (2 kg), also, we have a nonsignificant model (70%) but a large impact (20 kg). If a person wants to lose weight, would they use the “scientifically proven” pill and be happy with the 2 kg, or rather try the 70% chance to lose 20 kg? Fortunately, there is no underlying fundamental tradeoff between significance and impact size. They are two characteristics of the model that both need to be interpreted. Power of explanation The third important characteristic of a model is its power of explanation. How much of the variation in one variable can be explained by the variation of the other elements? If the model has a power of explanation of 100%, it fits perfectly. All variations of one element can be explained by variations of the other elements. If the power of explanation is zero, then no variation of one element can be explained by variations of the other elements at all. Significance and power of explanation do not refer to the same characteristics of the model. You can find highly significant elements (i.e., it is very unlikely that the relation is only coincidence) but the explanatory power of the model is still low (i.e., a lot of variation of one element cannot be explained by the elements in the model). For example, renumeration, job tasks and working hours might explain job satisfaction partly, but there are still differences in job satisfaction between individuals despite the same renumeration, job tasks, and working hours. Use of proxies This is not something unique to cross-sectional research of even quantitative methods, but here it becomes immediately clear. Theories do not require exact measurements. For example, the theory or assumption “the bigger the company, the higher the profit” is perfectly fine. Though, “big” is a construct that is not operationalized. To include “big” in a statistically testable model, we use proxies instead of the construct, assuming that the proxy is a good representation of the construct. Keep in mind that your model uses these proxies and not the concept itself. False positives and false negatives A test yields a false positive if it shows a condition that is not there in reality. It yields a false negative if it shows the absence of a condition that is there in reality. As we deal with probabilities in the methods of analysis most used in cross-sectional research, we need to be aware of these mistakes.

192

10  Cross-Sectional Research Design

Rejection of the opposite hypothesis A statistical test cannot prove the existence of a relationship. It basically calculates the likelihood, that the empirical data is only coincidence, i.e., resulted purely by chance because no relationship between the elements exists. If the probability is sufficiently low, then the opposite hypothesis (i.e., absence of a relationship) can be rejected. Requirements Each statistical test comes with a list of preconditions or assumptions (e.g., linearity assumption, homoscedasticity). All these preconditions need to be met to apply the test and use the results. Outliers Empirical data usually contains some outliers that potentially skew the whole test. Therefore, these are usually eliminated or changed as described in a lot of statistical textbooks. The method of how you detected outliers and what you do with them needs to be explained. But you need to keep in mind the important ramification of outliers: If the outlier is not the result of somehow falsified data, then this data set represents something that can contribute tremendously to the increase of the body of knowledge, something that does not follow the established rules and theories. Recall our Stone Age example (see Sect. 2.2). The person with the lens would have been eliminated as an outlier. Then the science of making fire would still revolve around type and thickness of wood, angle and depth of indention and point, methods to spin the first wood faster, and so on. Using human respondents Cross-sectional designs involving human participants face challenges. Researchers considering cross-sectional designs that include human respondents should know several issues, including representativeness and generalizability, sample size, and inclusion criteria and nonresponse bias (Allen, 2017). As mentioned, cross-sectional studies are suitable to describe a population of interest at a single point in time. The data sample must accurately reflect the entire population of interest. This feature is known as representativeness. Data collected from a representative sample provide an unbiased sign of what the overall population would show (if time and resources allow all members of that population to respond). If data are collected from a nonrepresentative sample, the validity will be diminished since the study will not accurately reflect the population of interest. Researchers can increase their representativeness through careful planning and use of sampling procedures like probability and quota sampling (Allen, 2017). Sample size is also important to ensure representativeness. Samples should be sufficiently large. The larger the sample size, the less likely results are because of coincidence. You determine an appropriate sample size by conducting analyses using known

10.2  Particularities of Cross-Sectional Research

193

estimates of the population, including population size, desired confidence interval and confidence level, and the expected variance in responses (Allen, 2017). As you collect all data simultaneously in cross-sectional designs, inclusion and exclusion criteria of participants must be predetermined before data collection. Identifying and determining how you can best access the population of interest is important in crosssectional designs. The development of an adequate sampling strategy and the list of members of a population from which a sample is drawn, is crucial to sampling only the desired participants that fit the study criteria. Researchers may waste resources if these criteria are not appropriately defined and identified. Also, nonresponse of participants may pose a major threat to cross-sectional designs and can severely bias results. This is an issue when non-responders differ on specific characteristics from those who participated in the study (Allen, 2017).

10.2.3 Major Fallacies in Conducting Cross-Sectional Research While providing guidance and support for research projects, these are the major pitfalls students encounter in their cross-sectional projects. Dismissal of non-significant results In academic literature, results that are not statistically significant are often completely dismissed (see the critique by Ziliak & McCloskey, 2008). However, a model that entails elements that are related to other elements 89% of the time is not a poor model, even if it cannot reach the arbitrarily set 90%. In fact, it is a huge intellectual contribution, if the significance of the elements used in models so far is much lower. Causality and correlation Cross-sectional studies test for correlations, but not for causality. Despite the model being structured in a way that explains one variable by other variables, this cannot be interpreted as that the variable is caused by the other variables. We consider this one of the most disturbing pitfalls in students’ theses. Thus, we like to make this abundantly clear. Cross-sectional research is by its design static. All observations refer to a single point in time. Theories in business and management, especially prescriptive theories, are in most cases dynamic. They involve changing circumstances, and those changes affect other changes. To understand the relationship between R&D and profit, it is not enough to observe R&D expenses and profit for each company. The underlying theory is that increases in R&D expenses have a positive impact on the profit in later periods. The cross-sectional research cannot test this hypothesis. It might even find a significant negatively directed correlation: the higher this period’s R&D expenses, the lower this period’s profit (Maxwell & Cole, 2007; Ployhart & Vandenberg, 2010). Singer and Willett (2003) note concisely that differences between units of analysis at one

194

10  Cross-Sectional Research Design

point do not represent change. There is a plethora of alternative explanations other than a causal relationship why a correlation is deemed significant. The fallacy is acting in two ways. One is wrongly inferring causality based on crosssectional research. The second is choosing cross-sectional research to test an inherently “dynamic theory”. Usage of test despite failing requirements Particularly in student’s theses, there is often a focus on one-sided examination of the requirements for the application of the chosen method: “I have the data and the data analysis, so the requirements must be met, otherwise I cannot finish my research”. As already mentioned, this is a very dangerous and completely unscientific approach: what has been showed is the willingness to sacrifice the scientific approach which often leads to an automatic failure and integrating wrong results into the body of knowledge. There are many techniques available to transform the data to better meet the methods requirements or alternative methods that require less strict preconditions to be met. Linear and log scales One of these requirements mentioned involves the scale of the data. Using log scales is in fact such a data transformation to meet linearity requirements. Often the use of log scales is stipulated from the start instead of beginning with a linear scale and only switching to log scales if requirement testing for linear scaled data failed. Categorization of continuous data To simplify the data analysis or to achieve sufficient sample sizes, continuous data sometimes get collapsed into categories. For example, instead of using the number of employees, two categories above and below 250 employees are established. The first problem is the establishing of the divide. Is it the median, the average, or some other value? The second problem is distortion and interpretation of the results, as companies with 250 and 251 employees are much more similar in size than companies with 10 and 250 employees despite belonging to different categories.

10.3 Writing a Cross-Sectional Research Paper Writing a cross-sectional research paper follows the principles and structure detailed in Chap. 4. However, there are some aspects especially important for reports about crosssectional research projects or (partly) different from reports about other research projects. We address these idiosyncrasies following the standard structure of a scientific paper.

10.3  Writing a Cross-Sectional Research Paper

195

Introduction The introduction deals mainly with expectation management. In cross-sectional studies this is especially important, as words like “proof” and “causal relationship” should never be associated with this research design as it simply cannot deliver. Cross-sectional studies can test the probability that a set of data results from random occurrences and that is it. So be aware that you do not overpromise. Theoretical background Cross-sectional research designs are commonly used to test theories. Obviously, these theories or the deduction of the tested hypothesis from existing theories needs to be shown. The clear definition of the elements and conditions analyzed are very important in cross-sectional studies. The applies for the definition and calculation of proxies. Present the definition of constructs used in the theoretical background. The choice of proxies is often less clear, might be affected by those used in studies in the literature review and can therefore be postponed to the method section. As the cross-sectional research design involves differences between groups, the participation or allocation to these groups needs to be depicted as well. Literature review The literature review of a cross-sectional research design focuses on the research gap, the samples used, the elements and conditions to be included in the model, and the proxies used. Hence, other cross-sectional designs and multiple case designs are especially of interest because here you can find this information about categories of elements and conditions and their proxies. Studies on causal relationships are also interesting (longitudinal studies and experiments) to make sure that you do not try to confirm the relationships that other studies already confirmed as causal. Typical research gap Typical research gaps shown in a cross-sectional study’s literature review refer to the representativeness of the samples used so far, the inclusion or exclusion of certain elements, and the differentiation of conditions under which the relationship exists. We may find another research gap in the robustness of the proxies used to represent the elements (constructs) in the model. Here, excellent argumentation is the key. Just replicating existing studies for a different sample or using different proxies might not meet the desired intellectual contribution unless you can show that the characteristics of the samples used so far put their representativeness and therefore the generalizability of the results in question and that your sample either has a higher representativeness or has characteristics that have not yet been sufficiently included in the samples analyzed so far. The same holds true for proxies. Just exchanging one proxy for another is not an intellectual contribution, but criticizing the employed proxies and offering a better, justified alternative and replicating results with this more advanced proxy is.

196

10  Cross-Sectional Research Design

Typical research aim The typical research aim of a cross-sectional study is to explain more variations of one element (construct or phenomenon) by variations of other elements and conditions and finding relationships that are probably non-random. Basically, it boils down to improving models used to describe the relationship between elements under certain conditions. Typical research question Research question typically are like “is there a difference between certain groups or categories?” or “is there a correlation between two (or more variables), potentially under certain conditions?”, or “what is a good model to depict generalizable relationships between elements under conditions?”. The research question in a cross-sectional can and should be reformulated as a hypothesis. The alternative hypothesis H1 stating there is a correlation between elements under certain conditions. The actual statistical test refers to the null hypothesis H0 stating that there is no relationship (and the probability to observe the data if the null hypothesis is true). Method In the method section, you describe the data sample, the data retrieval, the transformation, the operationalization of the data, and their use in the calculation of the proxies. Depict the qualification and treatment of outliers. Also, show the statistical test used to analyze the data, including all the requirements for employing the test and the decision rule for accepting or rejecting the hypothesis. Results First, the results of a cross-sectional study are represented by the final data sample used for analyses. Thus, the outliers must be identified and treated. The test of this final data set regarding meeting the method requirements is also part of the results. Second, a major part of this section results from the statistical tests, i.e., the significance, the explanatory power, and the impact sizes. Also depict all intermediate results. For example, if the correlation of an element to the dependent element does not show as significant, it will be eliminated from the ultimate model, but this test for significance is part of the results. Discussion As the results of a cross-sectional study appear to be so clear cut and straightforward, it is especially important to interpret them in the light of existing studies and alternative explanations. Here, it is again mandatory to point out that the study did not establish causal relationships, nor did it prove anything. This needs to be reflected in the wording. Potential explanations of the generated results therefore need to be discussed in this section.

10.4  Related Research Designs

197

Conclusion The conclusion is the qualification of the analyzed relationships between elements and conditions and the explanation(s) of these relationships, and the future research needed to whittle down the number of rivalling explanations.

10.4 Related Research Designs In this section, we briefly describe or cross-reference research designs that are similar to the cross-sectional research design. They might partially overlap with or considered to be adjacent to cross-sectional research or in fact be a cross-sectional research design that has its own label in the literature. If the cross-sectional research design does not fully meet your intentions and expectations, look here for direction where to continue your search. Longitudinal research In contrast to cross-sectional research where researchers collect data from many subjects at a single point in time, longitudinal research (Chap. 11) collects data repeatedly from the same subjects over time (Allen, 2017). Including the dimension of time offers the possibility to infer the probability of causal relationships, as time is supposed to be linear: if A happens before B, B cannot be the cause for A. Repeated cross-sectional design Sometimes, cross-sectional designs are used repeatedly to establish a causal time-order. So-called repeated cross-sectional designs can establish a deeper understanding of a population of interest by observing the same variables at different points in time, usually defined by some predetermined interval, be it days, weeks, months, or years. Each observation represents an own cross-sectional study that observes a single population at a single time point, but together the set of observations supports the identification of trends, patterns, and rates of change on a subject of interest to researchers (Allen, 2017). Repeated cross-sectional designs observe different samples of the same population over time. Unlike panel studies and cohort studies, they do not observe the same subjects at multiple points in time, and they do not track the same participants over a longer period. These studies provide descriptive descriptions of the population at one point in time and to detail changes since previous observations. While these research designs improve the ability of researchers to account for the time ordering of causes and outcomes, cross-sectional research designs and repeated cross-sectional research designs lack control and randomization offered only by experimental designs. Repeated crosssectional designs are widespread in the social sciences and include media use polls, knowledge and opinion surveys, and political opinion polls (Allen, 2017).

198

10  Cross-Sectional Research Design

Cross-sectional field studies and multiple case research designs Cross-sectional field studies and multiple case studies have fewer data sets than crosssectional studies and hence cannot use statistical analyses. Because of the limited number of cases, they can delve more deeply into the cases and not restricted to the collection of data about pre-defined elements and conditions (see Chap. 9). Key Aspects to Remember

Understand the three distinct features of cross-sectional research designs Cross-sectional research designs have three distinctive features: observation refer to one point in time, a focus on existing differences rather than change following an intervention, and groups are chosen based on existing differences rather than random allocation (for example, gender or income bracket). The cross-sectional design can measure differences between or from among a variety of people, subjects, or phenomena rather than a process of change. Consider the power of explanation as an important issue The power of explanation is used to analyze how much of the variation in one variable is explained by the variation of the other variables. If the model has a power of explanation of 100%, it fits perfectly that the model is a perfect fit. This means that all variations of one element can be explained by variations of the other elements. If the power of explanation is zero, then no variation of one element can be explained by variations of the other elements at all. Be aware that significance and power of explanation do not refer to the same characteristic of the model. Researchers can find highly significant elements, but the explanatory power of the model is still low. Do not dismiss non-significant results in the first place In academic literature, results that are not statistically significant are often dismissed. However, a model that comprises elements that are related to other elements 89% of the time is not a poor model, even if it cannot reach the arbitrarily set 90%. In fact, it is a huge intellectual contribution, if the significance of the elements used in models so far is much lower. Understand the intellectual contribution of cross-sectional research designs The intellectual contributions lie in the inclusion, exclusion, or categorization of elements and conditions that have a relationship with other elements, the direction of the relationship (for example “the bigger x, the bigger y” or “category a had bigger / smaller y”), and ultimately by explaining a larger part of the variation of the phenomenon (the “other” elements), by determining impact sizes and significance level.

References

199

Critical Thinking Questions

1. What are the three distinctive features of cross-sectional research designs? 2. What kind of intellectual contributions can researchers produce with this research design? 3. What major challenges do you face when applying cross-sectional research designs? 4. What is the difference between correlation and causality? 5. What is meant by false positives and false negatives?

Recommendations for further Readings

If you are still unsure whether “cross-sectional” research design is suitable for your research project, you might find the following literature and readings helpful. • Williams, J. J. & Seaman, Alfred E. (2002). Management accounting systems change and departmental performance: the influence of managerial information and task uncertainty. In Management Accounting Research 13 (4), pp. 419–445. • Ziliak, S. T. & McCloskey, D. (2008). The Cult of Statistical Significance: How the Standard Error Costs Us Jobs, Justice, and Lives, The University of Michigan Press, Ann Arbor.

References Allen, M. (2017). Cross-Sectional Design. The SAGE encyclopedia of communication research methods. SAGE Publications, Inc. Lauren, T. (2020). What is a cross-sectional study? Retrieved June 14, 2021, from https://www. scribbr.com/methodology/cross-sectional-study/. Maxwell, S. E., & Cole, D. A. (2007). Bias in cross-sectional analyses of longitudinal mediation. Psychological Methods, 12, 23–44. Ployhart, R. E., & Vandenberg, R. J. (2010). Longitudinal Research: The theory, design, and analysis of change. Journal of Management, 36, 94–120. Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis. Oxford University Press. USC University of Southern California (2021). Research guides. Retrieved April 05, 2021, from https://libguides.usc.edu/writingguide/researchdesigns. Williams, J. J., & Seaman, A. E. (2002). Management accounting systems change and departmental performance: The influence of managerial information and task uncertainty. Management Accounting Research, 13(4), 419–445. Ziliak, S. T., & McCloskey, D. (2008). The cult of statistical significance: How the standard error costs Us jobs, justice, and lives. The University of Michigan Press.

11

Longitudinal Research Design

Learning Objectives

When you have finished studying this chapter, you will be able to: • understand the purpose of longitudinal research as establishing generalizable statements about elements at different points in time, relationships, and conditions, one condition being time periods or events • see that this design focuses on change and comparison of the same unit of analysis at several points in time • argue why this design allows inference for the population • avoid the major fallacies of longitudinal research designs

Longitudinal research

• aims for generalizable statements about elements at different points in time, relationships, and conditions, one condition being time periods or events • focus on change and comparison of the same unit of analysis at several points in time • allows to infer the probability of temporal sequences • collecting of sample data that allows inference for the population • statistical analysis whether the observed sample might be a random occurrence

© The Author(s), under exclusive license to Springer Fachmedien Wiesbaden GmbH, part of Springer Nature 2021 S. Hunziker and M. Blankenagel, Research Design in Business and Management, https://doi.org/10.1007/978-3-658-34357-6_11

201

202

11  Longitudinal Research Design

11.1 General Description of Longitudinal Research It is safe to say that most theories in business and management are explicitly or implicitly longitudinal. The normative purpose of our science requires thinking of change and cause-effect relationships. This purpose cannot be addressed by only researching phenomena at a single point in time (Ployhart and Vandenberg 2010). So, longitudinal research designs represent an examination of correlated phenomena over a period. Their analysis stresses changes over time. The relevance of variables that do not vary over time cannot be captured by such studies (Ployhart & Vandenberg, 2010). Despite the relevance of the longitudinal nature, definitions of longitudinal research are rare. Exceptions are for example Taris (2000), who argued that longitudinal data are collected for the same research units (which might differ from the sampling units or respondents) for two or more occasions, basically allowing for intra-individual comparison across time (Wang et al., 2017). Ployhart and Vandenberg (2010) define “longitudinal research as research emphasizing the study of change and containing at minimum three repeated observations (although over three is better) on at least one of the substantive constructs of interest” (p. 97). We like the emphasis on change in Ployhart and Vandenberg’s definition but find it otherwise too restrictive, specifically about the number of observations. Following Wang et al (2017), we consider an analysis of changes between times 1 and times 2 with an event taking place in between also a longitudinal study. Some criteria might even only be measured once as the event explicitly changes that criterion (like promotions or dismissals). Taris’ (2000) focus on intra-individual comparison also seems unnecessarily restrictive. Thus, Wang et al. (2017) opt for a simpler and more straightforward definition of longitudinal research as they state that this is simply research where data are collected over a meaningful span of time. This is a good basis for longitudinal research. We like to add the aim of producing generalizable conclusion to this definition. So, we can distinguish longitudinal research design from single case, multiple case, and action research designs (Wang et al., 2017). Longitudinal (including time-lagged designs) are required to answer four types of questions: • • • •

causal priority, future prediction, change, and temporal external validity (Wang et al., 2017).

The aim of a longitudinal research design is to enable or to improve the validity of inferences that cross-sectional research designs cannot accomplish as they consider only one point in time (Shadish et al., 2002). Unfortunately, this improvement is not an inherent feature but the aim of the design that some studies do not fulfill. This specifically holds true if we talk about causality: “At the simplest level, in examining whether an X causes

11.2  Particularities of Longitudinal Research Design

203

a Y, we need to know when X occurs and when Y occurs. Without theoretical or empirical guides about when to measure X and Y, we risk inappropriate measurement, analysis, and, ultimately, inferences about the strength, order, and direction of causal relationships (Mitchell & James, 2001, p. 530, cited in Wang et al., 2017).” For example, by measuring at the wrong points in time, we might miss the effect because it did not yet take place, or it is already over. Time is key because a linear understanding of it is the basis of causality. The cause needs to precede the effect (James et al., 1982). The design decisions in longitudinal research are manifold like attrition, spacing and timing of measurements and their consequences are important. Only looking at several points in time does not automatically mean we observe change (Ployhart & Vandenberg, 2010). Mitchell and James (2001) state succinctly: “with impoverished theory about issues such as when events occur, when they change, or how quickly they change, the empirical researcher is in a quandary. Decisions about when to measure and how frequently to measure critical variables are left to intuition, chance, convenience, or tradition. None of these are particularly reliable guides” (p. 533).

11.2 Particularities of Longitudinal Research Design In this section, we address the elements that make longitudinal research a discrete research design differentiated from others. Next to the characteristics of longitudinal research, we address the main issues and decisions to be made within this research design, and the major pitfalls.

11.2.1 Characteristics of Longitudinal Research Design In this section, we elaborate on the key characteristics of longitudinal research along the steps of the research process. Conclusion The typical conclusion of longitudinal research is that and how elements, relationships and conditions change over time. The change can be caused by certain conditions or events. Intellectual Contribution The intellectual contributions lie in the inclusion, exclusion, or categorization of elements and conditions that have a relationship with other elements, the direction of the relationship (for example “the bigger X, the bigger Y” or “category C had bigger or smaller Y”), and ultimately by explaining a larger part of the variation of the phenomenon (the “other” elements), by determining impact sizes and significance level.

204

11  Longitudinal Research Design

Additional to these contributions that are the same as for cross-sectional studies, timing and sequences of elements and conditions are included. This is done by using periods as categorization of conditions and by confirming the probability of the existence of cause-effect relationships based on temporal sequences. Argument The key argument is that based on the data analyzed, there is a probability that there is a relationship between certain elements under certain conditions (significance, time is one of those conditions), that the elements probably affect each other and themselves at the same time and over with a certain effect size. Also, we expect that all analyzed elements, relationships, and conditions (one of those being time) probably explain a part of the variation in other elements, i. e., the phenomenon (explanatory power). Results The results are the elements and conditions including time, events and sequences that probably have or have not a relationship with the researched phenomenon, their significance, their effect sizes, and the explanatory power of the model as a combination of the elements and conditions. Methods The method is usually a statistical analysis of a model containing the elements and conditions as dependent and independent variables and their relationships. The model is fitted to the data and probabilities calculated, how likely it is that the datasets can be observed, if the depicted relationship does not exist. In longitudinal research, each element and condition are also characterized by the time they refer to. The model can include elements from different points in time and even the same element from different point in times. It can also contain a variable depicting whether an event has taken place. Data We collect and analyze quantitative and categorical data, including time or events in complete data sets. Research question The typical research question is “what variation in the phenomenon can be explained by the sequence (including cause effect) of elements over time or an event taking place?”.

11.2.2 Issues to Address In detailing the research design, you face many longitudinal research specific problems and decisions. We list the most important ones and the main options you have in the

11.2  Particularities of Longitudinal Research Design

205

following. Please be aware that simple addressing them is not sufficient. Make sure that the overall research design is set up so you can answer your research question. Incorporating change into your research How do you incorporate change into your research and which changes? It is worthwhile to think about the following questions posed by Mitchell and James (2001) and to include your thoughts into the theoretical background of your research: • what is the time lag between X and Y? How long after X occurs or can be detected does Y occur or can be detected? The difference between occurrence and observability leads to the next question. • how long is the duration of X and Y? Not all variables (and changes) occur instantaneously or uniformly. So, when within the duration of the variable do you measure and which measurement do you attribute to the variable (point in time, highest / lowest / average over duration, etc.). • do X and Y change over time either way? What is the rate of change independent from each other? When looking at the impact of personnel branding on hiring, the seasonal changes in the unemployment rate might be needed considered. • is there a dynamic relationship between X and Y and what is it? X and Y might both change, and this might affect their relationship. How is the relationship of X to Y affected by the changes and levels of X and Y? • is there a reciprocal relationship between X and Y? Does X cause Y and Y causes X? This doubles the first four questions as they need to be addressed for the relationship Y to X as well. We like to add a sixth question based on Ployhart and Vandenberg (2010): • what exactly is the change you look at? Group-level changes (change in employee satisfaction), intra-unit change (change of employee satisfaction of employee A), or inter-unit differences of intra-unit change (difference of changes of employee satisfaction between employee A and B or category I and category J). Addressing these questions in your research is mandatory, as they have a powerful impact on your research design. Your preconceptions (hopefully based on the existing body of knowledge) determines your initial answer you state as your theoretical background: “to the extent possible, specify a theory of change by noting the specific form and duration of change and predictors of change” (Ployhart and Vandenberg 2010, p. 103, cited in Wang et al., 2017). Time in longitudinal research As longitudinal studies are built on observations at different points in time, it might astonish that time is an issue. The understanding of the role of time is important for understanding longitudinal research (George & Jones, 2000).

206

11  Longitudinal Research Design

The first consideration is more of a theoretical than a methodological nature. Most constructs do not change because of time. They change over time. Wang et al.’s (2017) example exhibits this very succinctly: time does not cause children to grow. They grow into adults over time because of genetics and the environment. Using descriptive and explanatory research might shed additional light on this subject: Descriptive longitudinal research shows the changes and the form of a change of phenomena over time. Explanatory longitudinal research tries to identify the reason for the changes, how elements and their relationships caused changes in other elements and relationships in the change process (Wang et al., 2017). But time has a different ontological status from the elements or variables in a longitudinal study (Wang et al. (2017): • time is not a construct that needs to be tracked or measured in a longitudinal study of changes over time. It does not make much sense to talk about the construct validity of time. Do not confuse “time” with a construct of “perceived time” for example. • a point in time is simply the temporal marker of the state of the element at the point of measurement. It is not the state or value of the substantive variable that we are interested in for identifying changes over time (Wang et al., 2017). So, we are not interested in 2016 and 2017 but in the company’s profits in 2016 and 2017. • time is distinct from the processes underlying change. It is only the medium through which one or several such processes occur. It is sometimes difficult to distinguish between the process and time as they are intrinsically linked in our perception, like time and age. The reason for this usually lies in the insufficient detailed analysis of the process and operationalization of the respective element. For example, we may use age of an employee to group several characteristics together like maturity, experience, and others that we conveniently link with the progress of time, again without looking at the change underlying processes. Social sciences differentiate between “ageing” and “period” effects (de Vaus 2001). We think this is a useful grouping of effects to better understand different causes. It is no call to include time as a cause. Time as a variable Longitudinal research designs can, with some precautions, improve confidence in inferences about causality. If this is the goal, time does not need to be measured or included as a variable in the analysis, yet, the interval between measurements must be reported because rate of change and cause are interconnected. Growth models, for example, often do not include representations of time itself but temporal markers that specify the state of an element (e.g., R&D expenditure of period t) (Wang et al., 2017). This does not mean that time has no place as a variable in longitudinal studies. It is (rightly so) often used to represent the prevailing conditions in the sense that in the year X the economic environment has been the same for all units of analysis. So, for a growth model like “existing customers (end of period t) equals existing customers (end of period t-1) plus newly gained customers (in period t) minus lost customer (in period

11.2  Particularities of Longitudinal Research Design

207

t)” you do not need the period as a separate variable. This model is sufficient to examine, for example, exponential growth where a bigger customer base causes higher customer acquisition. If you want to examine why this growth pattern changed, you might consider including the period as a variable representing the prevailing conditions in that period (e.g., a financial crisis). Number of observation waves First, the number of observations or waves as they are often termed in longitudinal research depends on your desired conclusion and the hypothesized form of change. If you hypothesize that change is linear and confirming the form of change is unimportant to your conclusion, you are happy with two observations. Second, if you assume or speculate on the existence of non-linear changes, you need a minimum of three observations. The same holds true if you want at least a tentative confirmation that your hypothesized form of change is linear. The general rule here is the more waves the better, limited by the feasibility of data collecting (Ployhart and Vandenberg 2010, Wang et al., 2017). Ployhart and Vandenberg (2010), following Singer and Willett (2003), summarize the reasons as follows. In two-wave studies: • the form of change is linear by default. Hence, it is impossible to determine the form of change over time. The change is merely the increment of the difference between the two points in time, therefore we cannot assess whether the change was steady, or delayed, whether it plateaued and then changed again, and • genuine change and measurement errors get confounded. A researcher may wrongly conclude change between times 1 and 2 when in fact the measurement at time 1 or 2 contained measurement error. Same unit of analysis Longitudinal studies require observations at different points in time. Except for purely prescriptive lagged designs, this requires observations of the same units of analysis (Wang et al., 2017). These units can be individuals, teams, organizations, etc. What makes up the unit of analysis thus becomes an issue to address (i.e., is a team with two unique members still the same team?) An emphasis on change permits researchers to capture two important characteristics of change: • within-unit change across time (e.g., the attitude of A changed over time), or growth trajectories, and • inter-unit differences in change that can be predicted or used for prediction (e.g., the result in a test predicts on average success in a new job) (Singer and Willett 2003, Bollen & Curran, 2006). Duration of an observation If an observation is not implicitly defined (e.g., a financial statement), the operationalization of the observation or measurement becomes important to specify the variable

208

11  Longitudinal Research Design

characteristic. One of that operationalizations refers to the duration of the observation. What counts as one observation? And how long is the time span that still counts for this observation? If determining employee satisfaction caused by a promotion, it might be important to clarify if you gather how the employee felt immediately after receiving the promotion, or within the next week after receiving the promotion (or within a month or until the new job started) and how to specify the data (e.g., ecstatic, predominantly satisfied, on average happy, getting more and more worried about the new responsibilities). Does the purchase of a new car after the launch of a new marketing campaign counts if it happens within three months after the launch? This also needs to be considered when designing questionnaires. The length of intervals might also be linked to other operationalization issues. Is the single observation of a behavior (buying ice cream) enough to postulate a different behavior, especially if it only happened once in a period? Introduce an additional wave to capture this development, specifically if you hypothesize systematic changes in the variable during the interval. Intervals between waves Like the number of observations and their durations, the decision about the intervals between waves should be based on your hypothesized form of change. An important factor here is the time until change takes place (lag) and the duration of the change. If the interval is too short, no effect has taken place. Yet if the interval is too long, the effect might already be over or even overcompensated for (Ployhart and Vandenberg 2010). Obviously, this can theoretically be countered by more observations that have its own detriments, especially if you collect primary data. Theoretically, the time interval for data collection is ideal if the time points are appropriately spaced so that it allows the true pattern of change over time to be observed. If the observed time interval is too short or too long as compared to the optimal time interval, true patterns of change cannot be uncovered or false patterns of change may be observed (Wang et al., 2017). The challenge is that we usually do not know what this optimal time interval is, even if researchers have a relatively appropriate theory underpinning the change phenomenon. This is because our theories of research phenomena are in most cases static. Even if theories are dynamic and focus on change processes, they almost never inform about the specific length of the temporal dimension through which the processes occur over time (Chan, 2014, cited in Wang et al., 2017). Prediction vs Explanation Causal explanation requires other design decisions than predictions (see different theories in Sect. 2.3.2). For prediction, it suffices to find correlations in a so-called lagged design (Wang et al., 2017). To evaluate the predictive power of a psychological test in the selection process for the future success in the job, you need to measure the results of the psychological test at time 1 and the success in the job at time 2. This might yield a huge predictive power for the test and calls for its application in the hiring process, but it does not explain the reasons for the success in the job. Ployhart and Vandenberg (2010)

11.2  Particularities of Longitudinal Research Design

209

qualify such a lagged design for a cross-sectional study, but we prefer the consistency provided by “multiple points in time”. Events as points in time and quasi experiments Longitudinal research can be used as quasi experiments by introducing specific events such as an Initial Public Offering (IPO) or the launch of a new product. These interrupted time series designs use repeated observations to evaluate trends before and after some manipulation to model possible maturation or maturation-by-selection effects. Also, regression discontinuous designs use a pre-test to assign participants to the conditions before the manipulation and consequently use the pre-test value to model selection effects (Shadish et al., 2002, Stone-Romero, 2010, cited in Wang et al., 2017). In the event analysis, the duration is particularly difficult to determine, as it encompasses the anticipation and some lag of the impact as well. For example, the event IPO might start with the expectation of the IPO, or the announcement might end 2 to 5 trading days after the IPO. Prospective versus retrospective Prospective and retrospective refer to time of data collection. In prospective designs, you collect the data at the point of time of the observation. For example, you ask customers in June 2020 about their buying experience in June 2020 and in June 2021 about their purchases in June 2021 (perhaps even directly after the purchase). In retrospective designs, you collect all the data at one point in time. This means you ask the customers in June 2021 about their buying experience in June 2021 and in June 2020. The retrospective data collection is much faster and requires less effort but is prone to distortions as soon as cognitive processes are involved (e.g., remembering issues.) The danger of distortion is lessened if official or semi-official records or data (i.e., secondary data) are used. If we assume that the records do not change, the data can obviously also be retrieved at several points in time as the collection process does not affect the collected data. Burden on participants Longitudinal research places a lot of burden on participants. They need to be conducted several times over a rather long period. Also, the next wave(s) of observations might take place at inconvenient times. They also must waive anonymity, especially if intra-unit changes are examined (Wang et al., 2017). Cross-sectional research has the advantages of allowing broader sampling of participants. This is because of faster and cheaper studies that involve less participant burden. Also, a broader sampling of constructs is possible because of the opportunity for participant anonymity in cross-sectional designs. This allows more honest and comprehensive measurement of sensitive concepts, as for example “counterproductive work behavior” (Wang et al., 2017).

210

11  Longitudinal Research Design

Primary and secondary data Many issues of longitudinal studies like costliness, duration of the study, risk of attrition refer to the use of primary data. Topics like behavior (e.g., decision making), attitude (e.g., brand awareness) and related phenomena often require primary data. Because of the issues involved, this usually makes longitudinal studies impractical for bachelor’s and master’s theses (doctoral dissertations might be an exception). The risk is simply too great that the data collection in the last wave hinders you to conduct the corresponding analysis. However, especially in management and business, we have access to many excellent databases storing information on corporations and stock markets. Using secondary data from these databases improves the problems of data collection for longitudinal studies: • data retrieval is fast and inexpensive, • all data can be retrieved at a single point in time, and • attrition can be seen and evaluated at once. Obviously, this does not mean that data requirements can or should be changed to available secondary data. But it means that longitudinal studies in business and management are still workable, but usually only for a distinct type of research questions that require already available secondary data to answer them. A retrospective design (collecting the data for all waves or observation times) offers a similar approach but leads to potential distortions with primary data. This is because you may ask people to remember what they felt, did, experienced at one distinct point in history. Dealing with cohorts Cohorts are groups of units of analysis that have experienced the same event and environment. Generation “Z” for example is a cohort. Going for cohorts in sampling has advantages and disadvantages and hence needs to be considered carefully. For example, we use all employees that started their jobs in 2020 to evaluate an employee introduction program. This might be the right decision if you plan to analyze how well the program performed under the special home office requirements. However, this does not work if you want to make a more general statement about the quality of the program, as you do not know what the impact of the special circumstances and of the program have been. For the latter, a more randomized sampling approach is preferable. Sometimes, you unwittingly experience cohort effects that might pose alternative explanations. For example, a sample of companies in the year 2009 almost only comprises survivors of the financial crisis. In the literature, cohorts and panels are treated differently, panels being random samples. We understand a cohort as a special type of panel.

11.2  Particularities of Longitudinal Research Design

211

Replacement of dropouts Because of attrition, the panel size becomes smaller over time. If you do not replace dropouts, this might lead to insufficiently large panels at the end of the study. If we replace them, we first must make sure that the replacements have similar characteristics as the dropouts. This procedure is difficult. As we potentially do not have any longer sufficient data from the same unit of analysis, change can only be analyzed on a group level but not any longer on an intra-unit level. Another method to counter attrition uses rotating panels where several panels with different, staggered starting points are used (e.g., panel 1 from 2013 to 2017, panel 2 from 2014 to 2018, etc.). This shortens the length of participation for each panel and allows to compare different panels (“older” ones with higher attrition and “newer” ones with less attrition) to adjust for changes caused by dropouts (de Vaus 2001).

11.2.3 Major Fallacies in Conducting Longitudinal Studies While providing guidance and support for research projects, these are the major pitfalls students encounter. Inferring non-existing causality Longitudinal studies have the potential to provide insights into cause-effect relationships. This does not mean that every change constitutes such a causal relationship. The same holds true for the opposite: just because no change has been observed does not mean there are no causal relationships. The demonstrated complexity of the longitudinal design offers many alternative explanations for detecting or non-detecting change. No clear conceptualization of time Often longitudinal studies start with the idea of several observations over time, but researchers do not know how to incorporate time into their study. Is time a qualification of other variables, is it a variable itself, or is it both? What are the consequences of this conceptualization, especially for interpreting the results? The outcome then usually is just a collection of data from arbitrarily timed observations that offers no intellectual contribution. Number of observations does not match the research question nor the hypothesized change We find many reasons to deviate from the optimal conceptualized number of observations. The most predominant is the amount of effort and the study length. The main fallacy here is that the number of observations is among the most important design variables in longitudinal studies. Infringing the number of observations might lead to an inability to detect the hypothesized form of change and hinders us to answer the research question.

212

11  Longitudinal Research Design

Mixing intervals, time lags, and duration Often the operationalization of time is not clear and inappropriate. For example, researching the impact of yearly R&D expenditure on yearly profits of a company your observation interval is one year and the lag between observations is zero. Even the R&D conducted on the 31 December would be counted as impacting profit generated in the first week of the next period. Despite of this, the lag is here reported as one year as the R&D expenditures in year 0 are put into a relationship with the profits of year 1. Attrition of sample Participation is key for achieving a large data sample. Whereas this is true for all research using primary data, this is aggravated for retrieving data from the same units of analysis at different points in time. This is usually addressed for longitudinal and experimental research but is also true for case related research. This phenomenon is called attrition: units of analysis that participated in a wave (or pre-test, interview, etc.) do not participate in the next one. Ultimately, you only have complete data sets from units of analysis that took part in every wave (and answered every question). Reasons for dropping out might be leaving the organization, absence because of illness, lost motivation to participate, and many more. This issue has been addressed in some publications (e.g., Fumagalli et al., 2013). The literature suggests providing monetary (or sometimes non-monetary) incentives prior to completing the survey (Laurie and Lynn 2008). However, researchers must consider that offering incentives might already bias the sample and the answers. Prior to thinking about potentially costly avoidance of attrition, consider the attrition in your sample planning. You need to determine the optimal number of observations and then estimate the sample size to start with taking the (worst case) dropout rate into account, e.g., 50% over all waves (Ployhart and Vandenberg 2010). This means you start with a sample twice as large as you expected. Ployhart and Vandenberg (2010) suggest trying to include the hypothesized (but substantiated) reason for missing data (e.g., from dropping out) into the research. This is also be a reasonable way to think about how attrition might change the sample. The sample in the last wave with the complete data sets might not any longer have the same characteristics as the initial one (and thus might not any longer be representative). Increasing the panel without adjusting the level of change analyzed Specifically close to end of a longitudinal study, the temptation increases to bolster diminishing panel sizes with new participants replacing dropouts. Matching the replacements with a dropout is difficult and usually imperfect, as we cannot characterize the units of analysis comprehensively. Such replacement and the matching that it requires then is often not described. The shift of the analysis to changes on the group level is usually ignored. So, the research still looks like an examination of intra-unit change. This voluntary or involuntary obfuscation invalidates the research. Alternatives would be to

11.3  Writing a Longitudinal Research Paper

213

keep the panels separate (the original and the one including the replacements) for further analysis. In-migration and out-migration Attrition addresses whether the sample is still representative because of dropouts. In contrast, migration focuses on changes in the population that are not reflected in the sample as this usually remains the same to allow for intra-unit developments. If, for example, we draw a sample panel of start-ups in Germany (we leave the term “in Germany” intentionally vague), we might have an overrepresentation of start-ups that have been setup by people originally living in Germany but moving to another country in preparation or during the founding of the start-up (out-migration) and an underrepresentation of start-ups that moved to Germany (in-migration).

11.3 Writing a Longitudinal Research Paper Writing a longitudinal research paper follows the principles and structure detailed in Chap. 4. However, there are some aspects especially important for reports about longitudinal projects or (partly) different from reports about other research projects. We address these idiosyncrasies following the standard structure of a scientific paper. Introduction Introducing a longitudinal study should put emphasis on the change and development that you describe and explain. Longitudinal studies are usually quite complex, so you must entice the reader to bear with you with the promise of a substantial intellectual contribution, potentially in form of causal relationships. Theoretical background The theoretical background is very important in a longitudinal research design. Here, you explain the hypothesized cause-effect relationship and the form of the change. This establishes the basis for all the following decisions about constructs, lags of effects, number of observations, duration of the observations, sample sizes, incorporation of time and many more. Literature review The literature review remains basically the same, but the annotation of the existing studies needs to be more comprehensive. This is because you must consider the consequences of the operationalizations to a much larger degree than for example in crosssectional research, where the change of the proxy used might sometimes be rather arbitrarily. Yet, in longitudinal research, you must discuss why you stick with the number of observations or change the number of observations, considering the researcher’s

214

11  Longitudinal Research Design

theory. Otherwise, the validity of the entire study might be infringed. However, do not only include longitudinal studies in the literature review. Cross-sectional studies are often thought to be of limited value for longitudinal studies, as usually the main point of criticism is design based: they focus on a single point on time and causal inferences are not possible. They can substantiate theories about change, especially if brought in sequence, and they might offer valuable insight into the sample (panel) to be used in the longitudinal research as sampling usually aims for highest probability for observation of change and not convenience. Multiple case research designs might provide insights about the hypothesized causeeffect relationships and the constructs to be included in the research. If the cases consider developments over time, these constructs might be more relevant than the ones used in cross-sectional studies as those are not geared to capture change and development. Typical research gap Unfortunately, the research gap is often quite large in longitudinal studies and typically includes the following aspects: • no or only few studies on change and development of the elements, • no or insufficient confirmed body of knowledge about cause effect relationships, • no or insufficient confirmed theory about the form of change and its underlying processes, • no or insufficient elimination of alternative explanations, • operationalization of constructs not in line with your theory of change, • number of waves, their duration, intervals, and effect lags are not in line with your theory of change, • sample sizes too small, attrition rates too high, samples not representative, and • time frames do or do not include specific conditions or events This list corresponds to a large degree with the major fallacies of longitudinal research. Arguing with the problems of existing studies raises expectations for improvement. Especially regarding longitudinal studies, these are sometimes hard to accomplish. To manage these expectations, take again care about the wording of your research report (do not overpromise or overclaim). Typical research aim The typical research aim is to describe, explain, and predict change, phenomena, and developments over time. Longitudinal studies are the most versatile about the aspired intellectual contribution. Their objective can be. • to describe or analyze change over time, • to predict future phenomena without explaining them,

11.3  Writing a Longitudinal Research Paper

215

• to explain development without predicting future development, or • to achieve any kind of combination of those objectives. This makes the precise, unambiguous statement of the research objective very important as there is no implied generic goal. Typical research question The typical research question is “what variation in the phenomenon can be explained by the sequence (including cause effect) of elements over time or an event taking place?”. The research question in a longitudinal study can and should be reformulated as a hypothesis. The alternative hypothesis H1 states there is a relationship between elements under certain conditions. The actual statistical test refers to the null hypothesis H0 stating that there is no relationship (and the probability to observe the data if the null hypothesis is true). Method In the method section, the data sample, the data retrieval, the transformation and operationalization of the data and their usage in the calculation of the proxies is described. If rotated panels are used, all the different panels and the sequential use of the panels need to be explained. The handling of dropouts is depicted. If you replace dropouts, include the matching procedures. If you adjusted the panels with the results of subsequent panel, explain this change. The qualification and treatment of outliers is depicted. Incorporate time and show how time is coded. Number, duration, intervals, and specific timing of the waves is described as well as inclusion of events, if relevant. Describe the data analysis method (normally a statistical method) and its requirements. As the data analysis refers to a model of the elements, conditions, and relationships, depict the model including the coding for time, generally as a mathematical formula but also describing all variables, indices and so on. Results First, the results of a longitudinal study are represented by the final data samples or panels for each wave used for analyses. Hence, identify the outliers and report attrition and countermeasures. The test of this final data set regarding meeting the method requirements is also part of the results. Second, a major part of this section results from the statistical tests, basically the significance, the explanatory power, and the impact sizes. The intermediate results are also depicted. For example, if the relationship of an element did not show significance, it is eliminated from the ultimate model. This test for significance is part of the results. In a longitudinal study, the results especially include covariances between distinct elements at different points in time and between the same element at different points in time. Also, in more advanced methods, the variation of the regression coefficients might also be a result. They are basically depicting the (changing) slope of the development.

216

11  Longitudinal Research Design

Discussion The results of a longitudinal study need to be discussed in the light of the initial theory of change and existing studies. Because of the complexity of the setup, we find a vast number of alternative explanations, especially when including method related issues. Longitudinal studies can infer the probable existence of causal relationships as usually under certain conditions X precedes Y. This makes sense from a theoretical point of view. However, this is no proof of the causal relationship. This needs to be reflected in the wording. Potential explanations of the generated results therefore need to be discussed in this section. In addition to the relationship itself, the form of the relationship and its change over time needs to be discussed. Conclusion The conclusion is the qualification of the analyzed (causal) relationships between elements and conditions and the explanation(s) of these relationships. This includes form and development, the relationships and the future research required to whittle down the number of rivalling explanations.

11.4 Related Research Designs In this section, we briefly describe or cross-reference research designs that are similar to the longitudinal research design. They might partially overlap with or considered to be adjacent to longitudinal research or in fact be a longitudinal research design that has its own label in the literature. If the longitudinal research design does not fully meet your intentions and expectations, look here for direction where to continue your search. Event analysis and simple panel design The simple prospective panel design or two-point panel design, also called event analysis, resembles an experiment. You collect elements from the sample panel or sample at time 1 and time 2. An intervention takes places between the two points in time. You may observe an active intervention, the effects of an event that might occur naturally, or it stems from an intervention not affected by the researcher. As compared to an experiment, there is no randomized control group. This might somewhat be eased by a large and diverse panel but generally, we cannot remove the possibility that differences in the panel after the event are caused by other factors than the event. For example, after an initial public offering (IPO) as the event we can divide the panel at time 2 into companies listed in the stock exchange and those that are not. However, the differences between the two groups might not be caused by the initial public offering, but by other factors. Or the differences even might have affected the decision to make an initial public offering or not (de Vaus 2001).

11.4  Related Research Designs

217

Time series and multi-point panel design This panel design is like the two-point or wave design used in the event design described above but involves more waves. Potentially, an event takes place in any of the intervals between the waves. The purpose of this design is to. • • • •

examine long- and short-term effects, track when changes occur, identify factors preceding change or non-change, and identify the form of the change (de Vaus 2001).

Because of the length and the multiple waves, attrition is a bigger problem in multiple point studies. Lagged design Lagged designs abolish the idea of measuring the same elements for the same units of analysis or panels, but purposely measure a construct or element X at time 1 and construct or element Y at time 2 (Y measured after X by a delay of interval t (Wang et al., 2017). This design focuses only on prediction at the expense of explanation, for example using a test to predict future employee performance. Experimental research design Longitudinal studies lack control groups. Causal inferences are therefore always limited by the potential existence of outside factors that might have triggered and achieved the change but are not part of the study. Experiments (see Chap. 12) use control groups to establish case effect relationships between elements. By controlling for all other variables, the changes of Y need to be caused by X. Cross-sectional research design Cross-sectional studies (see Chap. 10) focus on observations at one single point in time. Thus, inferencing causal relationship is not possible with a cross-sectional design. If the envisioned arguments do not include theories about cause-effect relationships, a crosssectional study might be sufficient and circumvent several issues of the rather complex longitudinal study. Sequential cross-sectional studies As a “staggered” design, the execution of two or more cross-sectional studies (see Chap. 10) at different points in time incorporates the time element and even addresses issues about panel representativeness of longitudinal studies. However, as the sample for the cross-sectional studies does not comprise the same units, we cannot observer intra-unit change, only changes on group-level.

218

11  Longitudinal Research Design

Multiple case research design Sample sizes in longitudinal studies are large, predominantly to meet statistical requirements. If sample sizes are insufficient, multiple case research designs might offer an appropriate design (see Chap. 9). A multiple case research design encompasses the changes and development of the cases (intra-unit) and allows including cause-effect relationships (even with limited validity, but not necessarily more limited than in longitudinal studies). Key Aspects to Remember

Differentiate between the terms “dynamic” and “static” These terms explain the disconnection between theory and the often applied cross-sectional research design used to test that theory. The most important aspect of the theories within the organizational sciences is explanation: to describe how the parts of the theory work together to better understand why we expect certain outcomes based on certain inputs. In summary, the variables supporting the theory and their relationships are described in dynamic terms. For example, if we raise an employee’s commitment, this will reduce the employee’s probability to leave the organization over time. Yet, this form of explanation has been rarely offered because of the persistent use of cross-sectional research designs employing static concepts (Ployhart & Vandenberg, 2010). Understand how longitudinal research designs contribute to the body of knowledge The intellectual contributions lie in inclusion, exclusion, or categorization of elements and conditions that have a relationship with other elements, the direction of the relationship (for example “the bigger X, the bigger Y” or “category A had bigger or smaller Y”), and ultimately by explaining a bigger part of the variation of the phenomenon (the “other” elements), by determining impact sizes and significance level. Additional to these contributions that are the same as for cross-sectional studies, timing and sequences of elements and conditions are included, by using periods as categorization of conditions and by confirming the probability of the existence of cause-effect relationships based on temporal sequences. Do not conclude causality if it does not exist Longitudinal studies have the potential to provide insights into cause-effect relationships. This does not mean that every kind of change makes up such a causal relationship. The same holds true for the opposite: just because no change has been observed does not mean there are no causal relationships. The demonstrated complexity of the longitudinal design offers a lot of alternative explanation for detecting or non-detecting change. Differentiate between intervals, time-lags, and duration Often, the operationalization of time is not clear nor appropriate. For example, researching the impact of yearly R&D expenditure on yearly profits of a company your

References

219

observation interval is one year and the lag between observations is zero: even the R&D conducted on the 31st of December would be counted as affecting profit generated in the first week of the next period. Despite of this, the lag is here reported as one year as the R&D expenditures in year 0 are put into a relationship with the profits of year 1.

Critical Thinking Questions

1. How can you incorporate change into the longitudinal research design? 2. What is the difference between prediction and explanation? 3. Why does relying on primary data often pose a major risk to this research design? 4. Why is it so important to think about a clear conceptualization of time in experiments? 5. What is the difference of a sequential cross-sectional design and a longitudinal research design?

Recommendations for further Readings

If you are still unsure whether “longitudinal” research design is suitable for your research project, you might find the following literature and readings helpful. • de Vaus, D. A. (2001). Research design in social research. Reprinted. Los Angeles: SAGE Publications, Inc. • Ployhart R. E. & Vandenberg R. J. (2010). Longitudinal Research: The theory, design, and analysis of change. Journal of Management, 36, 94–120. • Taris T. (2000). Longitudinal data analysis. London, UK: Sage Publications. • Wang, M., Beal, D. J., Chan, D., Newman, D. A., Vancouver, J. B. & Vandenberg R. J. (2017). Longitudinal Research: A Panel Discussion on Conceptual Issues, Research Design, and Statistical Techniques, Work, Aging and Retirement (3)1, pp. 1–24.

References Bollen, K. A., & Curran, P. J. (2006). Latent curve models: A structural equation perspective. John Wiley. Chan D. (2014). Time and methodological choices. In A. J. Shipp & Y. Fried (Eds.), Time and work (Vol. 2): How time impacts groups, organizations, and methodological choices. Psychology Press. de Vaus, D. A. (2001). Research design in social research. Reprinted. SAGE Publications Inc.

220

11  Longitudinal Research Design

Fumagalli, L., Laurie, H., & Lynn, P. (2013). Experiments with methods to reduce attrition in longitudinal surveys. Journal of the Royal Statistical Society: Series A (statistics in Society), 176, 499–519. George, J. M., & Jones, G. R. (2000). The role of time in theory and theory building. Journal of Management, 26, 657–684. James, L. R., Mulaik, S. A., & Brett, J. M. (1982). Causal analysis: Assumptions, models, and data. Sage Publications. Laurie, H. & Lynn, P. (2008). The use of respondent incentives on longitudinal surveys. ISER Working Paper no. 2008-42. Colchester: University of Essex. Mitchell, T. R., & James, L. R. (2001). Building better theory: Time and the specification of when things happen. Academy of Management Review, 26, 530–547. Ployhart R. E. & Vandenberg R. J. (2010). Longitudinal Research: The theory, design, and analysis of change. Journal of Management, 36, 94–120. Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Houghton Mifflin. Singer, J. D., & Willett, J. B. (2003). Applied longitudinal data analysis. New York: Oxford University Press. Stone-Romero, E. F., & Rosopa, P. J. (2010). Research design options for testing mediation models and their implications for facets of validity. Journal of Managerial Psychology, 25, 697–712. Taris, T. (2000). Longitudinal data analysis. Sage Publications. Wang, M., Beal, D. J., Chan, D., Newman, D. A., Vancouver, J. B., & Vandenberg, R. J. (2017). Longitudinal research: A panel discussion on conceptual issues. Research Design, and Statistical Techniques, Work, Aging and Retirement, 3(1), 1–24.

12

Experimental Research Design

Learning Objectives

When you have finished studying this chapter, you will be able to: • understand the purpose of experimental research as aiming for generalizable statements about cause-effect relationships • control for all elements outside the causal relationship of your experimental research project • know how to collect data for a treatment group prior and after an intervention • understand the purpose of the statistical analysis of this design: whether the observed change in the treatment group is different from the change in the control group

Experimental research

• aims for generalizable statements about causal relationships • controls for all elements outside the causal relationship • collect data for a treatment group prior and after an intervention • collects data for a control group for comparison prior and after a non-intervention • statistical analysis whether the observed change in the treatment group is different from the change in the control group

© The Author(s), under exclusive license to Springer Fachmedien Wiesbaden GmbH, part of Springer Nature 2021 S. Hunziker and M. Blankenagel, Research Design in Business and Management, https://doi.org/10.1007/978-3-658-34357-6_12

221

222

12  Experimental Research Design

12.1 General Description of Experimental Research Designs Experiments have a long and important history in the social, natural, and medical sciences. Unfortunately, in business and management this is not the case. For example, as Koschate-Fischer & Schandelmeier (2014) show, experiments have been conducted in marketing (e.g., Hennig-Thurau et al., 2006; Homburg et al., 2005), human resource management (e.g., Mohnen et al., 2008), management (e.g., Franke et al., 2010; Harbring & Irlenbusch, 2011), entrepreneurship (Burmeister & Schade, 2007; Sandri et al., 2010), finance (e.g., Weber & Zuchel, 2005, Biais & Weber, 2009), accounting and controlling (e.g., Bowlin et al., 2009), and logistics (Lödding & Lohmann, 2012). But overall, the number is relatively low. This is surprising, as experiments are very suitable for analyzing cause-and-effect relationships. According to Aronson et al. (1990), “a true experiment is the best method for finding out whether one thing really causes another” (p. 9). In the words of a wellknown fictitious character “when you have eliminated the impossible, whatever remains, however improbable, must be the truth” (Doyle, 1890). That is the foundation experiments are based on. We eliminate all other explanations, so the remaining explanation must hold true. Experiments accomplish this state by controlling all elements, systematically altering one or a few elements and observing the effect of this intervention. The change in the “independent” variable then has to have caused the effect in the “dependent” variable (Stier, 1999). This leads to the question of how to achieve this level of absolute control? This requires two components that basically constitute the experiment as a research design. The first component is the situation representing the set of conditions. Therefore, experiments usually take place in an artificial setting that yields such a control over the prevailing conditions to the researcher. To achieve such a controlled situation in a real-world context would be highly intrusive and by necessity artificial again. The second component is the usage of a treatment and a control group. As it is not feasible to generate objects with the same characteristics (perhaps apart from physics) and even impossible as soon as living beings are involved, experiments use statistical means to produce controlled variables. This “control” does not mean that the value for every single variable is controlled, rather that we establish two groups of entities by drawing a sufficiently large sample and then randomly allocate entities to one of the two groups. This results in two similar groups, without systematic differences. The individual entities might differ, but the group’s overall does not differ systematically. On average the persons in group one should not be more intelligent than the persons in group two. The bigger the sample, the smaller the probability of differences between the groups. We have controlled the elements on the group level, and not only some, but “by design” all elements. This eliminates the risk to overlook an element, meaning that changes on the group level (not on individual level) prior and after the intervention can only be caused by that intervention.

12.2 Particularities of Experimental Research

223

Experimental research designs are often considered as the most “rigorous” of all research designs or as the “gold standard” against which all other research designs are assessed. If you execute an experimental research design well, then the experiment is probably the strongest research design regarding internal validity (Trochim, 2005).

12.2 Particularities of Experimental Research In this section, we specifically address the elements that make experimental research a discrete research design differentiated from others. Next to the characteristics of experimental research, we address the main issues and decisions to be made within this research design, and the major pitfalls.

12.2.1 Characteristics of Experimental Research Design In this section, we elaborate on the key characteristics of experimental research along the steps of the research process. Conclusion The typical conclusion of an experiment is that this element or group of elements causes that effect under these conditions. Intellectual contribution The intellectual contribution is finding and confirming the probable existence of causal relationships between elements under certain conditions. Argument The key argument of an experiment is that while controlling for all other elements and conditions, there is a significant difference in the phenomenon that must be caused by the changed element (treatment). Results The result is the explanatory power, the significance, and the impact size of the treatment (changed element or condition) for the phenomenon. Methods The methods used most often are statistical analyses of the differences between two groups, the treatment group, and the control group (Bortz & Döring, 2006; Christensen, 2007).

224

12  Experimental Research Design

Data Data to be collected and analyzed are descriptive data about the (minimum) two groups, information about the treatment (the changed element or condition), data about the independent or effected element, the phenomenon. Research question The typical research question of an experiment is “what impact has change of X on Y?”.

12.2.2 Issues to Address in Experimental Research In detailing the research design, you face many experimental research specific problems and decisions. We list the most important ones and the main options you have in the following. Controlling for the independent variables and the environment The key for a successful experiment is the control over all variables, either directly or by ensuring they are the same for all participants. At first glance this might seem easy, but in fact it is not. The whole experimental design is centered around meeting this requirement. The validity of the conclusion depends on it. Some researchers argue that having a control group suffices to meet this requirement (de Vaus 2001). In fact, they see the control group as the separating difference between experiments and longitudinal studies. Following our stipulated research process (see Chap. 3) with a focus on valid conclusions to be drawn from research, we suggest making sure that experimental research generates distinct valid arguments. This provides a more stringent differentiation than using such a method-based delineation that broadens the experimental research design (and mixes its type of conclusions with other research designs). Number of independent, systematically changed variables and treatments Experimental designs are best suited for a limited number of systematically changed independent variables, limited treatments (interventions) and a limited number of dependent variables (Krishnaswamy et al., 2009). To maintain design integrity and conclusion validity, the treatment for everybody in the treatment group needs to be the same. The difference between treatment and control group is “treatment” versus “no treatment”, not a bit more or less treatment. To capitalize on the statistical elimination of other influencing factors, treatment needs to be categorized. You can administer different treatments, but for each treatment you need an additional treatment group. For example, one group gets two days additional vacation, one gets one day, and one gets no additional days off (not to mention all other problems such a setup would cause). The same holds true for each additional independent variable that should be systematically changed, but with even more impact on your number of treatment groups, as you

12.2  Particularities of Experimental Research

225

also need to cover the combination of the elements. Elaborating on the above example, you have variable X1 (days off) with the treatment + 1, and you have the variable X2 (cash bonus) with the treatment + €500. Then you already need four groups (1: additional day off, 2: cash bonus, 3: additional day off and cash bonus, 4: control group, i.e., no additional day off and no cash bonus). The interaction of independent variables would interest of course (Fromkin & Streufert, 1976), but the complexity (and the size of the sample) added to the setup is significant. Random allocation to treatment and control group Draw a sample and then allocate the entities randomly to one group. Obviously, each entity can only participate in one group. Pulling different samples for each group might bias the group participants and is to be avoided. The same holds true for any interference with the random allocation. The larger the initial sample and the groups, the more similar the groups will be. Some authors (e.g., Robson, 2011) distinguish between “traditional experiments” that use such random allocation and “quasi experiments”. The latter lack the random allocation. There is some overlap with natural experiments where the inability to control the administering of the treatment, as the intervention is caused by nature or other not controlled factors, generates the allocation to the two groups. We consider the so-called quasi experiments longitudinal research (and natural experiments, too). Systematically altering a variable Controlling variables also means the ability to alter one variable, this alteration constituting the treatment, the variable depicting the potential cause. This excludes quite many experiments in which the intervention, for example, is a life altering experience like marriage, coming out, childbirth, death of a loved relative, or winning a lottery. Or it makes them unfeasible like administering a cash bonus to random employees in the example above. Unfortunately, terminology often used is not precise. In so-called “natural experiments”, the researcher lacks the ability to alter the variable depicting the intervention, as the treatment is administered by nature or factors outside the researcher’s control. In natural experiments, the researcher does not control the experimental conditions and developments. Rather, they are determined by nature or by other factors outside the control of the researcher. We follow Johnson (2001) and Robson (2011) and deem this research not experimental but longitudinal research. Sometimes, we find ways to apparently ease this restriction. This then is usually based on the ability to alter the perception of something instead of altering the real thing (which for the participants does not make a difference at that moment). But the ethical considerations and the potential repercussions of fooled subjects remain. Systematically altering constructs that are not directly observable becomes a challenge. So, Perdue and

226

12  Experimental Research Design

Summers (1986) state, “the design of an experiment involving one or more latent independent variables is one of the hardest tasks a researcher can undertake” (p. 325). Instrument attrition and decay Controlling the variables often entails keeping the participants in the dark about the actual experiment. If participants knew details of the experiment, they would be biased. For example, in one experiment, the time it took the participants to walk out of the building from the room, where the treatment (what the participants assumed to be the whole experiment) took place, was the dependent variable. All participants had to solve a scrambled-sentence task. For the treatment group, the sentences contained words like “Florida”, “bingo” and “ancient”, causing associations of old age and frailty and a longer time to walk to the exit (Bargh et al., 1996). This great setup works very well but only if nobody knows about it. So, keep the details of your experiment confidential. We can observe a similar development with repeated usage of the same test, for example, an intelligence test before and after the treatment. The validity of the test will decrease because of training effects of the participants. Instrument decay encompasses the changes in results because of changes in the instruments themselves (de Vaus 2001). Slightly different wording, like telephone call instead of personal meeting, interviewer A instead of B, might all affect the difference between pre- and post-test. Data collection Experiments always involve the collection of primary data. As any experiment specifically and only looks at one causal relationship between X and Y, the data of that experiment cannot be used for another experiment. Collecting primary data requires often difficult methods and is important in all research designs (if they do not secondary data). But the negative consequences of errors in the collection process are specifically high in experiments as they can barely be adjusted (same variables and conditions for all entities). This warrants a brief and cursory digress into collection methods that are used in experimental designs (Rogers, 2003): • observational measures, that refer to direct observations of the participating entities as direct measures (e.g., the time to walk from room one to room two) or categorization of behavior (e.g., “angry”, “friendly”), • self-report measures, that refer to responses of participants to interviews or surveys about themselves (e.g., their [change of] attitude, behavior), and • implicit measures that infer the responses based on reactions or reaction times to stimuli not directly addressing the question in focus. Any data collection method, apart from direct observations, involves operationalization issues and biases. Categorization of observations and implicit measures requires

12.2  Particularities of Experimental Research

227

transformation and combination (what constitutes “friendly”?). This requires confirmation of validity in the data collection. Self-report measures, as the most often used data collection in business and management related experiments, add the problem of biases to the operationalization issue like coding of the answers. People do not answer truthfully because of perceived social acceptability, desirability, self-confirmation, and other reasons (Schoen & Crilly, 2012). The wording of questions and the form of data collection might also cause biases. The creative setup of the “experiment” needs to be followed up by a thoughtful data collection, otherwise the effort of the experiment is wasted. Administering non-treatment Administering non-treatment might be as important as administering treatment. Obviously, sometimes it cannot be solved that entities in the groups realize whether they are in the treatment or in the control group. For example, either you participate in the job-rotation program with the Asian affiliated companies, or you do not. This realization might impact the groups and the internal validity of the impact of the treatment as this realization by necessity is different for the two groups. For example, if you do not participate in the job rotation, you might be more likely to drop out, skewing the characteristics of the control group. Or just being selected for participating in the job rotation program might make you happier and bolster your self-confidence and this affects the independent variable. Then changes in the independent variable might be caused by the job rotation itself or by the selection. The placebo effect is the most well-known instance of administering non-treatment: people believing they get treated (for example by receiving pills with no active substances) show usually similar changes about the independent variable than persons who got treated with the active substances. To keep treatment and control group as similar as possible, any indication to which group they belong should be avoided (unless they believe to belong is the altered variable). Ethical considerations Usually, ethical considerations in research are relevant on the method level (which questions to ask, ensuring confidentiality and anonymity, singling out specific groups of persons, etc.). Thus, they are not specifically addressed here (despite being very important). However, experimental research as a design might cause ethical questions. Ultimately, some entities get “treated”, and some not. If the treatment is assessed as desirable, you deny the control group this “treat”. Sometimes this is alleviated by (monetary) rewards for participation (before or after study, for all participants or only the members of the control group), but often the value of the treatment is too high for this being workable. Obviously, this compensation in combination with the required large samples might make the study rather expensive.

228

12  Experimental Research Design

External validity To control all variables and making them the same for both groups, the experiments are often conducted under artificial conditions. This control strengthens internal validity, but the artificial situation might infringe the external validity. For example, just because of participating in the experiment, people might be more alert and self-conscious than normal (see Friese et al., 2009 for differences in real life behavior and behavior in controlled environments). Appropriateness of pre-test The standard experiment set up requires a pre-test of the dependent variable (prior to the treatment) of the treatment and the control group. This is only required if you establish a baseline to evaluate change from. For example, if you like to measure the improvement in skills caused by a specific training method, you should determine the level of skill prior to the training. If you emphasize the difference between treatment and control group after the treatment, you could argue, based on the size of the sample and the random allocation to the two groups, that differences between the groups are caused by the treatment. Statistically, there should be no difference between the groups prior to the treatment (de Vaus 2001). We think this line of reasoning should be avoided when aiming at establishing causal relationships unless conducting the pre-test itself might have an influence on the results. For example, if you determine the relative preferences of beverages A and B after being exposed to a promotion of A (treatment), you should not conduct a pre-test offering A and B. People will stick with their original preference. Such a pre-test would influence the result of the experiment. However, you can consider offering anonymized beverages as a pre-test.

12.2.3 Major Fallacies in Conducting Experimental Research While providing guidance and support for research projects, these are the major pitfalls our students encountered in their experimental research projects. Biases Often the setup of the experiment introduces biases into the experiment. One step in the process where this happens is enough to invalidate the experiment’s conclusion. And there are many steps to stumble, starting from effects of being allocated to a specific group, effects from receiving non-treatment to the effects of the process of data collection to just name a few. Sample sizes Many (unsystematic) issues with controlling the variables and the conditions can be eased with a larger sample size. Limiting the sample size or including additional

12.3  Writing an Experimental Research Paper

229

treatments and/or independent variables to modify often deteriorate the validity. This is basically a case of “wanting too much with too little”. Drawing a large sample and focusing on one or two variables to be changed improves the probability of a valid result. Non-random allocation The internal validity of the results depends on the random allocation of the entities to the groups. Depending on the “treatment” there might be high pressure to allocate specific entities to specific groups. This is especially understandable in medical research, as the treatment might affect the subject’s health. Also, in business and management there might be treatments like “participation in job rotation” or “flexible working hours” that have a high intrinsic value for some participants.

12.3 Writing an Experimental Research Paper Writing an experimental research paper follows the principles and structure detailed in Chap. 4. However, there are some aspects especially important for reports about experimental research projects or (partly) different from reports about other research projects. We address these idiosyncrasies following the standard structure of a scientific paper. Introduction An experiment is a straightforward research design. Your introduction should mirror this without further ado: you want to eliminate the possibility of any alternative explanation and demonstrate that X causes Y. The importance of this demonstration is almost selfexplaining. You contribute to the body of knowledge by ultimately providing the closest thing to a proof in science (we are aware that “proof” is a critical term in research projects). Theoretical background By design, experiments are mainly geared to confirm theory. Hence, you present this theory. Having said this, the theory to be confirmed can still be explanatory, predictive or both. Literature review The literature review for an experiment tends to be rather focused. First, you show that no proof for the causal relationship exists yet (otherwise why bother with the experiment?). Second, you gather evidence about existence, direction, and causality of the relationship to be researched and all the alternative explanations that you want to falsify. Third, gather supporting studies for conducting your experiment, especially about preventing introducing biases: sampling, allocation to groups, managing confidentiality, collecting data, and dealing with validity and reliability.

230

12  Experimental Research Design

Typical research gap The typical research gap is that there are still alternative explanations for a phenomenon that have not been yet eliminated and the causal relationship between X and Y has not yet been finally confirmed. Typical research aim As already mentioned, you want to eliminate the possibility of any alternative explanation and demonstrate that X causes Y. Typical research question The typical research question of an experiment is “what impact has change of X on Y?”. The research question in an experiment should be phrased as a hypothesis. The alternative hypothesis H1 stating that X affects Y because there is a difference in Y between groups one and two, with group one having changed X to Xi (being treated). The actual statistical test refers to the null hypothesis H0 stating that there is no impact (and the probability to observe the data if the null hypothesis is true) because there is no difference in Y between the two groups. Method The method section in an experiment is rather elaborate as the experiment setup needs to be described to ensure reliability and validity. This comprises. • drawing of the sample, • allocation to treatment and control group, • comprehensive operationalization of data collection of the pre-test of the dependent variable for both groups, • administration of treatment and non-treatment, • dealing with changes of participants in treatment and control group (e.g., dropouts, refusal to participate in the treatment), • comprehensive operationalization of data collection of the post-test of the dependent variable for both groups (only the changes), and • statistical tests for differences between the groups. Results The results of an experiment are the significance of the difference between the groups and the impact size of the treatment. Discussion The discussion predominantly refers to potential errors in the experiment and their impact on internal validity. The interpretation of the results again is straightforward. The experiment is purposefully designed is such a way that there is only very little room for interpretation.

12.4  Related Research Designs

231

The implication of this result needs to be discussed. A major part of the existing body of knowledge needs to be reassessed because alternative explanations are eliminated. Basically, every study in the 2nd part of the literature review can now be reinterpreted based on the confirmed causal relationship. This is particularly important for the relationship between X and Y, as the experiments only yields categorical results as this treatment (change of X) caused this increase of Y on the group level. However, the existing body of knowledge may well include correlations and impact sizes and conditions under which the relationship might be modified (e.g., the form of change affected). Finally, this also delineates the direction of future research: the size and form of impact of X on Y and the conditions that change the size and form if this has not already been covered by existing research. Conclusion The conclusion again is straightforward. Changes of X cause changes in Y. An artificial setup impacts the external validity. This should be noted in the conclusion, if applicable. Experimental research is very rigorous about design and execution. The advantages lie in the very simple argumentation and conclusion.

12.4 Related Research Designs In this section, we briefly describe or cross-reference research designs that are similar to the experimental research design. They might partially overlap with or considered to be adjacent to experimental research or in fact be an experimental research design that has its own label in the literature. If the experimental research design does not fully meet your intentions and expectations, look here for direction where to continue your search. Cross-sectional research design If your focus is on the difference between treatment and control group after the treatment, and so you think about skipping the pre-test, you could also consider a cross-sectional study with the treatment being a dummy variable (1 for “treatment”, 0 for “no treatment”). Longitudinal research design The major difference between longitudinal research designs and experimental designs is the absence of the control group and of the control over all variables and conditions. Sometimes we call this a quasi-experiment where a control group exists, but the allocation is not random. This holds true also for a natural experiment, where the intervention is beyond the control of the researcher. In our terminology, these research designs are all longitudinal ones.

232

12  Experimental Research Design

Key Aspects to Remember

Achieving control is a major characteristic of experimental research designs Achieving control is a major advantage of experiments. This requires two components that make up the experiment as a research design. The first component is the situation representing the set of conditions. Thus, experiments usually take place in an artificial setting that yields such a control over the prevailing conditions to the researcher. To accomplish such a controlled situation in a real-world context is highly intrusive, thus by necessity artificial again. The second component is the use of a treatment and a control group. As it is not workable to generate objects with the same characteristics (perhaps apart from physics), experimental research makes use of statistical means to produce controlled variables. Understand the suitability of experimental research designs Experiments are a suitable method if the investigation comprises few independent variables and one or more dependent variables. The interplay of the independent variables is usually the focus of the investigation. Hence, two or three independent variables are included in an experiment to test the isolated impact of the independent variables on the dependent variable. A precondition for an experiment is that at least one of the independent variables can be varied. In contrast, experiments are of only limited suitability if complex multi-level chains of effects of a larger amount of variables are the focus of an investigation or if none of the independent variables can be varied (Koschate-Fischer & Schandelmeier, 2014). Be aware of the major fallacies of experimental research designs First, you must handle potential biases. One bias in an experiment may be enough to invalidate the experiment’s conclusion. And there are many steps to stumble, starting from effects of being allocated to a specific group, effects from receiving non-treatment to the effects of the process of data collection to just name a few. Second, sample size matters. You can ease many issues with controlling the variables and the conditions with a larger sample size. Third, internal validity of the results depends heavily on the random allocation of the entities to the groups. Depending on the “treatment”, there might be high pressure to allocate specific entities to specific groups. In medical research this is specifically relevant, as the treatment can affect the subject’s health. Also, in business and management, there might be treatments like “participation in job rotation” or “flexible working hours” that have a high intrinsic value for some participants. Take care about ethical considerations in experimental research Whereas usually ethical considerations in research are relevant on the method level, experimental research designs might cause ethical questions. In the end some entities get “treated” some not. If the treatment is assessed as desirable, you deny the control group this “treat”. Sometimes this is eased by (monetary) rewards for participation (before or

References

233

after study, for all participants or only the members of the control group), but usually the value of the treatment is too high for this being workable. Obviously, this compensation in combination with the required large samples might make the study rather expensive.

Critical Thinking Questions

1. How can we achieve “absolute control” in experimental research designs? 2. What is the difference between a traditional and quasi experiment? 3. How is “non-random allocation” related to internal validity? 4. What is the major difference between an experiment and a longitudinal study? 5. What biases may have a negative impact on the experiments’ validity?

Recommendations for further Readings

If you are still unsure whether “experimental” research design is suitable for your research project, you might find the following literature and readings helpful. • Aronson E., Ellsworth, P., Carlsmith J. & Gonzales M. (1990). Methods of research in social psychology, 2nd edition. McGraw-Hill, New York. • Christensen, L. B. (2007). Experimental methodology. Pearson/Allyn and Bacon, Boston. • Shadish, W. R., Cook T. D. & Campbell D. T. (2002). Experimental and quasiexperimental designs for generalized causal inference. Boston, MA: Houghton Mifflin. • Montgomery, D. C. (2019) Design and Analysis of Experiments, 10th Edition, Wiley.

References Aronson, E., Ellsworth, P., Carlsmith, J., & Gonzales, M. (1990). Methods of research in social psychology (2nd ed.). McGraw-Hill. Bargh, J., Chen, M., & Burrows, L. (1996). Automaticity of social behavior: Direct effects of trait construct and stereotype activation on action. Journal of Personality and Social Psychology., 71(2), 230–244. Biais, B., & Weber, M. (2009). Hindsight bias, risk perception, and investment performance. Management Science, 55(6), 1018–1029. Bortz, J. & Döring, N. (2006). Forschungsmethoden und evaluation. Springer. Bowlin, K. O., Hales, J., & Kachelmeier, S. J. (2009). Experimental evidence of how prior experience as an auditor influences managers’ strategic reporting decisions. Rev Account Stud, 14(1), 63–87.

234

12  Experimental Research Design

Burmeister, K., & Schade, C. (2007). Are entrepreneurs’ decisions more biased? An experimental investigation of the susceptibility to status quo bias. Journal of Business Venturing, 22(3), 340–362. Christensen, L. B. (2007). Experimental methodology. Pearson/Allyn and Bacon. de Vaus, D. A. (2001). Research design in social research. Reprinted. Los Angeles: SAGE Publications, Inc. Doyle, A. C. (1890). The sign of the four. Lippincott’s Monthly Magazine. Ward, Lock & Co. Franke, N., Schreier, M., & Kaiser, U. (2010). The “I designed it myself” effect in mass customization. Management Science, 56(1), 125–140. Friese, M., Wilhelm, H., & Michaela, W. (2009). The impulsive consumer: Predicting consumer behavior with implicit reaction time measurements. Social Psychology of Consumer Behavior (pp. 335–364). Psychology Press. Fromkin, H. L., & Streufert, S. (1976). Laboratory experimentation. In M. D. Dunnette (Ed.), Handbook of industrial and organizational psychology (pp. 415–465). Rand McNally College. Harbring, C., & Irlenbusch, B. (2011). Sabotage in tournaments: evidence from a laboratory experiment. Management Science, 57(4), 611–627. Hennig-Thurau, T., Groth, M., Paul, M., & Gremler, D. D. (2006). Are all smiles created equal? How emotional contagion and emotional labor affect service relationships. Journal of Marketing, 70(3), 58–73. Homburg, C., Koschate, N., & Hoyer, W. D. (2005). Do Satisfied customers really pay more? A study of the relationship between customer satisfaction and willingness to pay. Journal of Marketing, 69(2), 84–96. Johnson, B. (2001). Toward a new classification of nonexperimental quantitative research. Educational Researcher, 30(2), 3–13. Koschate-Fischer, N., & Schandelmeier, S. (2014). A Guideline for Designing Experimental Studies in Marketing research and a Critical Discussion of Selected Problem Areas. Journal of Business Economics, 84(6), 793–826. Krishnaswamy K. N., Sivakumar A. I. & Mathirajan M. (2009). Management research methodology: Integration of principles, methods and techniques. Dorling Kindersley. Lödding, H., & Lohmann, S. (2012). INCAP – applying short-term flexibility to control inventories. International Journal of Production Research, 50(3), 909–919. Mohnen, A., Pokorny, K., & Sliwka, D. (2008). Transparency, inequity aversion, and the dynamics of peer pressure in teams: Theory and evidence. Journal of Labor Economics, 26(4), 693–720. Perdue, B. C., & Summers, J. O. (1986). Checking the success of manipulations in marketing experiments. Journal of Marketing Research, 23(4), 317–326. Robson, C. (2011). Real world research: A resource for users of social research methods in applied settings (3rd ed.). Wiley. Rogers, W. S. (2003). Social psychology: Experimental and critical approaches. Open University Press. Sandri, S., Schade, C., Mußhoff, O., & Odening, M. (2010). Holding on for too long? An experimental study on inertia in entrepreneurs’ and non-entrepreneurs’ disinvestment choices. Journal of Economic Behavior & Organization, 76(1), 30–44. Schoen, K., & Crilly, N. (2012). Implicit methods for testing product preference: Exploratory studies with the affective simon task. In J. Brasset, J. McDonnell, & M. Malpass (Eds.), Proceedings of 8th international design and emotion conference, ed. London: Central Saint Martins College of Art and Design. Stier, W. (1999). Empirische Forschungsmethoden. Springer. Trochim, W. (2005). Research methods: The concise knowledge base. Atomic Dog Pub. Weber, M., & Zuchel, H. (2005). How do prior outcomes affect risk attitude? Comparing escalation of commitment and the house-money effect. Decision Analysis, 2(1), 30–43.

13

Literature Review Research Design

Learning Objectives

When you have finished studying this chapter, you will be able to: • understand the purpose of literature review research as part of every research paper (staggered design with literature review as one stage) and stand-alone research design • explain that summarizing the existing body of knowledge and the gaps in it are the main purpose of this design • understand that different research designs are possible to achieve the aims of literature review research. • deal with the major fallacies of literature review research designs

Literature review research

• part of every research paper (staggered design with literature review as one stage) and stand-alone research design • aims at summarizing the existing body of knowledge and identifying the gaps in it • different forms of literature review research design available to address the objective • theories and studies as data to be collected and analyzed

© The Author(s), under exclusive license to Springer Fachmedien Wiesbaden GmbH, part of Springer Nature 2021 S. Hunziker and M. Blankenagel, Research Design in Business and Management, https://doi.org/10.1007/978-3-658-34357-6_13

235

236

13  Literature Review Research Design

13.1 General Description of Literature Review Design Presenting literature review as a research design entails a change of approach as compared to the other research designs. Conducting a literature review requires making many (design) choices, so addressing these choices is in line with our approach. However, the distinguishing factor from other research designs are not any longer the decisions along the research process, but the object of the research, namely the existing body of knowledge. This “breach of approach” is worthwhile for four reasons: • (excellent) literature reviews have a value on their own and can and should be submitted as thesis and/or published. “State-of-the-art literature reviews are legitimate and publishable scholarly documents. Too many new scholars believe that empirical research is the only “real” research” (LeCompte et al., 2003, p. 124). Which researcher is not delighted to find a literature review in his or her area of research? • theory creation and elaboration as two types of intellectual contributions can be accomplished without empirical research (LeCompte et al., 2003). Most of the empirical research adds intellectual contribution in small steps as every supervisor’s mantra is “focus, narrow down your research question”. Hence, integration of the existing body of knowledge, potentially even combining fields of study, is worthwhile scientific research and goes beyond what is conventionally understood as literature review. In fact, a literature review is an excellent way to integrate research findings to show evidence at a meta-level and to integrate it with extant theory as an important step in building and elaborating theoretical frameworks and conceptual models (Webster & Watson, 2002; Snyder, 2019). • literature review is a compulsory part of every single research project because “a researcher cannot perform significant research without first understanding the literature in the field” (Boote & Beile, 2005, p. 3). This makes every research by necessity a staggered design, as the literature review is followed up with your own research. So, improving literature reviews improves the quality of most research conducted. Lather (1999) even argues that an excellent literature review acts as a gatekeeper, ensuring research with higher intellectual contribution. • there is scarce advice on how to conduct a literature review (Boote & Beile, 2005). Presenting the literature review as a research design facilitates the shift from the common “are three articles enough for my literature review?” to a scientific approach with a focus on the conclusion to be drawn from the research and the decisions made throughout the process. Thus, we integrated dealing with existing research as data into one research design instead of repeatedly mentioning it in every research design as “by the way, the data you might use in this research design could be another research”.

13.2  Particularities of Literature Review Research Design

237

Hart (1999) offers the following reasons for thoroughly reviewing the literature, including: • • • • • • • • • • •

distinguishing what has been done from what needs to be done, discovering important variables relevant to the topic, synthesizing and gaining a new perspective, identifying relationships between ideas and practices, establishing the context of the topic or problem, rationalizing the significance of the problem, enhancing and acquiring the subject vocabulary, understanding the structure of the subject, relating ideas and theory to applications, identifying the main methodologies and research techniques that have been used, and placing the research in a historical context to show familiarity with state-of-the-art developments (p. 27).

In the following section, we approach literature review as a stand-alone research design, not as part of a research report as in Sect. 4.4.4. Obviously, there are some similarities between the two and we focus on the differences.

13.2 Particularities of Literature Review Research Design In this section, we specifically address the elements that make literature review research a discrete research design differentiated from others. Next to the characteristics of literature review research, we address the main issues and decisions to be made within this research design, and the major pitfalls.

13.2.1 Characteristics of Literature Review Research Design In this section, we elaborate on the key characteristics of literature review research along the steps of the research process. Conclusion The typical conclusion of a literature review research design is: this is the body of knowledge, this is what we know about the subject, these are the current theories, these are the gaps in our knowledge and what we do not know, and this is the direction of research over the years.

238

13  Literature Review Research Design

Intellectual contribution The intellectual contribution is the collection of the existent body of knowledge and a summary of the research gaps. Argument The key arguments are: these are the extant theories, this is the empirical evidence collected, confirming, or disproving these theories, and this cannot yet be explained by the theories. Results The results are annotated theories and studies constituting the existing body of knowledge. Methods The predominant methods used are categorization and triangulation of the studies analyzed. Data The data collected and analyzed are the existing studies and theories and their content. Research question The typical research question of a literature review research design is “what is the body of knowledge and what gaps in this body of knowledge exist regarding [topic]?”.

13.2.2 Issues to Address In detailing the research design, you face many literature review research specific problems and decisions. We list the most important ones and the main options you have in the following. Sampling What articles and research should be considered as the starting sample? Common sampling criteria are: • year(s), • journal(s), • language(s), • database(s), and • topic (as described by a search string).

13.2  Particularities of Literature Review Research Design

239

We purposefully use the term “sample” to point out that this process introduces biases into the literature review, even if this sample does not constitute the final data set included in the research review. Detection or suspicion of prevailing biases might require revisiting the sample. Search string We addressed the search string extensively in Sect. 4.4.4. As part of the data collection process, it is important to add that the search string might be modified and refined in an iterative process that needs to be documented, especially the reasons for the adjustments. The screening of the selected data at a later stage might cause adjustments rather late. The search string needs to relate to the desired arguments and conclusion. In the research process, we stressed the importance of selecting the data that we can use to generate the results and to bolster our arguments. Finding the “wrong” data and not finding the “right” data might make it impossible to draw the hypothetical conclusion. Inclusion and exclusion of data The most straightforward way is to choose all the selected data for the analysis. Despite saving time in the selection process, this might increase the workload in the data analysis because of the large number of selected data sets (articles). Also, we could not detect biases and not assess the appropriateness of the data checked. Hence, we may develop a screening process that focuses on certain decisions in the design process (e.g., data analysis methods used, scope of the sample) or parts of the paper (e.g., summary, conclusion) to decide whether to include or exclude the dataset. A checking process might be in order, whether datasets (articles) should be included (because you know the author(s) contributed to this field, or this is an article that has been referenced in one or several of the pre-selected datasets) in the sample or not. If not, you might need to revise your sampling and your search string. Looking for referenced articles is called iterative literature search and has been applied, for example, by Behringer et al. (2019): Example

Behringer et al. (2019) applied in their paper on compliance in family firms an effective procedure that resembles a snowball system: “The systematic literature analysis was supplemented through an iterative searching process. For this purpose, the bibliographies of the already found papers from the systematic literature analysis were scanned for further possibly suitable articles. The abstracts of these possibly suitable articles were read directly to assess the relevance of the research topic. The VHB 3 Jourqual ranking helped in the decision for inclusion in the final sample. The aim of this iterative process was to increase the quality and quantity of the sample of papers. […] In the search of the full articles, relevant papers were found as well. So, the iterative searching process was structured as a

240

13  Literature Review Research Design

snowball system. Parallel a second single keyword search was started in the databases. The keyword basis was extended through keywords like unethical work behaviour, unethical behaviour, misconduct, misbehaviour, and organizational corruption. These keywords showed another perspective or other understanding of corruption in (family) firms” (p. 144). Operationalize criteria for inclusion and exclusion (relevance, quality, etc.) and ensure inter-researcher reliability (e.g., by independent control of two researchers). Again, the process needs to be reported meticulously. Datasets What do you consider a dataset? Can you only get this from empirical research? Or do you consider conceptual papers addressing theories and frameworks? Do not expect to encounter the latter category in all journals. This is one of the potential biases you might be prone of. If your hypothesized conclusion requires the inclusion of these datasets, you might have to conduct a second search geared to find and select these articles. Another type of dataset might stem from a different science. In this science, a topic adjacent to yours or using a different perspective might be addressed. As with any another science, different terminology is used, articles are published in different journals and so on. This also requires a specialized search and selection process. The reporting requirements for each different dataset remain the same. Outliers As with any kind of qualitative data, defining what is to be considered an outlier is difficult, as there are no rules (how arbitrary they might be about quantitative data). But the connotation of assessing an article as an outlier in a literature review is different. If the study is of poor quality, it should be excluded because of the (systematic and rigorous) selection process. But singling a study out as an outlier basically means that inclusion would distort the results of your research or these results of a scientific research should be disregarded. This smacks of repression of freedom of opinion and excessive political correctness that is detrimental to scientific progress. Exclusion of (qualitatively good) empirical studies as outliers should be avoided at all costs. The urge to do so might hint at unrealized personal biases. A countermeasure might be to revisit and refine the selection criteria. This does not mean tweaking the selection criteria to achieve an exclusion of the former “outlier” but to improve the selection process under full disclosure. Regarding theories as datasets, the problem of outliers is different. To make an extreme example of an outlier, what would you do with following theory “aliens mindcontrolled the test-subjects to falsify the results” that came up based on your search string? We cannot disprove this theory, but its probability is extremely low. This leads to the loaded question: do we include every theory in our literature review, even if it is nonsense? Or in other words: how do we handle such outliers? The answer is like the comments to the one regarding empirical studies. If you have established, discussed, and

13.2  Particularities of Literature Review Research Design

241

disclosed selection criteria that apply to all theories which resulted in the exclusion of this theory (among others), that is fine. Such a selection criterion could be the existence of some (partial) confirmation. It could also be the absence of disproving results (i.e., data that should not have been observed, if the theory is correct). This usually excludes most “outlying” theories. However, this would still not result in the exclusion of the mind-controlling aliens, as there is no disapproving evidence, and any kind of confirmation could be viewed as a confirmation of this theory. In such extreme cases, you could also add some definition of “usefulness for scientific progress” to the selection criteria. This finally leads to an exclusion of any theory that (in your opinion) does not advance the understanding of the world and our perception of it, nor does it offer the potential to increase this understanding in the future. The mind-controlling aliens would ultimately end any progress in understanding the world. But as with any powerful tool, the application of such a very subjective criterion needs to be conducted scrupulously. The danger to eliminate fruitful theories just because they are not in line with current politically correct mainstream thinking is high. As a famous example, consider the geocentric model with the earth as the center of the universe and the alternative explanations that were disregarded for a long time and proved to be true in the end. Actuality of data As with all research based on secondary data, you need to address the question when to stop collecting data, and when to close the time bracket the data refers to. In a literature review research design, you face a continuous stream of newly published articles. One way to solve this is to include the time as a selection criterion from the start, for example, the last complete year preceding the start of your research. This puts some pressure on you to conduct your research quickly. This is usually no problem as many degree programs stipulate rather short time frames for research projects either way. A second possibility includes the option to add very important new articles afterwards. This requires good reasoning why especially these articles and no others have been added to the selection. The third option is to use a two-step approach with a fixed time frame like in the first one. At the point you feel comfortable with the selection process, use the same process for articles that have been published in the meantime (e.g., for additional 1 to 12 months). We highly recommend doing without an ongoing time frame as this is not conductive to an efficient research process (Randolph, 2009). Data analysis Data analysis in the literature review basically is a way of coding the data sets. All datasets (articles) are characterized along different dimensions like type of research design, method of analysis, significance levels, sample sizes, sample characteristics, and proxies used. Further analyses depend on the specific research question and could be qualitative or quantitative (e.g., time series analysis of methods used in a particular field).

242

13  Literature Review Research Design

Integration As the literature review gathers the existing body of knowledge, it offers the ideal opportunity to integrate theory and studies and to synthesize new or elaborate existing theories. This is not necessarily the primary aim of each literature review research, as described in the study’s aim and research question, but a worthwhile option.

13.2.3 Major Fallacies in Conducting Literature Review Research While providing guidance and support for research projects, these are the major pitfalls students encounter in their literature review research design projects. No search strategy As shown in the different literature reviews below, there are several search strategies possible. Often none is used. Then the literature review deteriorates to a collection of randomly found articles. Based on this sample, the literature review, including its results, becomes arbitrary and cannot generate an intellectual contribution. At a first glance, literature review sounds like an “easy” research design because it can be conducted evading two often perceived problems: collecting primary data and the use of statistical methods. While it is possible to circumvent these methods, literature review remains a research design that requires rigor and effort for each step in the demanding design. Superficial data collection Whereas the fallacy above refers to the sample used in the literature review, this one focuses on the data retrieved from a collected article. Commonly, the data for an article is limited to the authors, and the significant results of a statistical analysis or a very general remark about findings in qualitative analysis. This is simply not enough to reflect the relevant body of knowledge. Especially the evaluation of the article, its research design, complete results, discussion, and conclusion completely lacks. This is like reporting on companies with no performance measure. No synthesis Most times, the literature review research is truncated. Researchers stop after the collection and listing of the data from the selected articles. Even after avoiding the abovementioned pitfalls, this leaves the reader with a list of data, with no further analysis, interpretation, and conclusion. The retrieved data needs to be grouped, categorized, aggregated, and synthesized to depict the relevant body of knowledge. The findings need to be discussed and interpreted, same as in any other research design. Even if “only” used as a step in the research process, formulating the research gap is a conclusion drawn from data. So, the concept should be very familiar. Usually, these essential steps lack or are very superficially dealt with in literature review research.

13.3  Writing a Literature Review Research Paper

243

13.3 Writing a Literature Review Research Paper Writing a literature review research paper follows the principles and structure detailed in Chap. 4. However, there are some aspects especially important or (partly) different for reports about literature review research projects. We address these idiosyncrasies following the standard structure of a scientific paper. Introduction The introduction needs to explain the aim of the literature review. This might seem pointless at first glance, but as we will see in the research aim and research question, the focus of the literature review is very important and might stem from a wider range of possibilities than expected. So, we are back to expectation management, meaning that you appropriately address the research aim and the scientific approach (meta-analysis) about studies and theories as dataset. Those two aspects are also important to gain the interest of the reader. Who wants to read a research paper about three other arbitrarily selected research papers? In the introduction, please sort out this misconception. Theoretical background The theoretical background of a literature review needs to present the underlying axioms of the research, the things that are not included or questioned because they are believed to be true. These convictions guide the selection process (e.g., no mind-controlling aliens) and should thus be disclosed. Literature review The literature review of a literature research design entails other meta-studies about research studies. Literature reviews about the same or adjacent topics should be critically analyzed (e.g., sampling, selection criteria, results). Literature reviews about other topics might yield valuable insights into the decisions made and its consequences. Typical research gap The typical research gap of a literature review is that there is no, or only a flawed, overview of the existing body of knowledge about the chosen topic. Typical research objective The research aim can be threefold: • collect, summarize, and analyze the existing body of knowledge and the conducted research, • identify the research gaps that the body of knowledge leaves unfilled, and • synthesize or establish a new or elaborated theory that integrates the existing knowledge.

244

13  Literature Review Research Design

As with the research question, the research objective of a literature review is much less pre-defined, as it comprises several potential designs. All of those use similar data as their characterizing component, i.e., labeling them as literature review. Typical research question Despite being a literature review, the specific research questions might differ to a large degree, as the data are the common starting point, not the hypothetical conclusion. Possible research questions are: • what is the existing body of knowledge about topic X? • what are the characteristics of the current research about topic X (about attributes A, B, C, etc.)? • what is the development of the body of knowledge about topic X about criteria A, B, C, etc.? • what is the development of the research about topic X about criteria A, B, C, etc.? • what are the gaps in the existing body of knowledge about topic X? • what are directions of future research to fill research gaps in the existing body of knowledge about topic X? • how does a theory look like that integrates the existing body of knowledge about topic X? Method Full disclosure of all decisions made: • sampling, • search string(s), • exclusion and inclusion, • screening, • iterations, • coding and categorization rules, • aggregation and grouping of data, and • statistical methods (if applicable). Results The results of the literature review are the coded and categorized research studies and theories and potentially their aggregation. The aggregation could be qualitative (studies addressing relationship X to Y, studies including condition A, studies referring to industry or country or continent C, etc.) or quantitative (number of studies in year X, number having samples bigger than X, etc.), or both. Quantitative aggregates might be further analyzed using descriptive or even inferential statistics, especially changes over time (e.g., topics, methods, journals, number of researchers) or current importance of topics or questions.

13.4  Related Research Designs

245

Discussion The discussion of the results should cover the first two and can address a third aspect, all involving synthesizing and interpreting the results: • what do we know (studies) or assume (theories)? What theories exist, how have they been elaborated over time and to which degree have they been confirmed? What alternative explanations exist that match all results (counter-theory)? • what do we not know but should know (research gap)? Looking at the extant theories and their level of confirmation, what direction of research would yield the most valuable (in the sense of a reduction of uncertainty) intellectual contribution? • integration of (some or all different aspects of) what we know into a theory (creation of a new theory or elaboration of existing ones). Conclusion Depending on the aim of the literature review, it may be a presentation of a research gap that needs to be closed or a theory that integrates (most of) the existing knowledge on the topic.

13.4 Related Research Designs In this section, we briefly describe or cross-reference research designs that are similar to the literature review research design. They might partially overlap with or considered to be adjacent to literature review research or in fact be a literature review research design that has its own label in the literature. If the literature review research design does not fully meet your intentions and expectations, look here for direction where to continue your search. There are more specialized versions of literature reviews, that are within the scope of the literature review design (and even hinted at in the design description) and might be more curtailed to your specific needs. Table 13.1 illustrates three distinct approaches to conduct a literature review. Systematic literature review Systematic literature reviews are adopted from the medical sciences, where they are used to summarize research findings. They are considered the gold standard among reviews (Davis et al., 2014). Systematic literature reviews are characterized by a systematic search process. All datasets that meet the selection criteria are included. This is especially suitable for collecting all existing evidence and a collection and summary of the body of knowledge. The analysis of the data and their critical annotation also follows an explicit and systematic process, minimizing biases and generating reliable results (Liberati et al., 2009, cited in Snyder 2019).

246

13  Literature Review Research Design

Table 13.1  Approaches of literature review research design (adapted from Snyder, 2019, p. 334.) Approach

Systematic

Semi-systematic

Typical Purpose

Synthesize and compare evidence

Overview research Critique and area and track develop- synthesize ment over time

Research questions

Specific

Broad

Narrow or broad

Search strategy

Systematic

Systematic or not systematic

Not systematic

Sample

Quantitative articles

All kinds of research articles

Research articles, books, and other published texts

Analysis

Quantitative

Qualitative/quantitative Qualitative

Examples of contribution

Evidence of effect Inform policy and practice

State of knowledge Themes in literature Historical overview Research agenda Theoretical model

Integrative

Taxonomy or classification Theoretical model or framework

We consider it worthwhile to stress the potential power of analysis methods available. For example, in a statistical meta-analysis, (statistical) results of the included studies on the same topic can be combined, and examined for patterns, inconsistencies or relationships (Davis et al., 2014). Also, we can analyze whether an effect is constant across studies, or which sample characteristics have an impact, e.g., to identify cultural differences. The primary studies must be abstracted, coded, and transformed into a common metric to allow for statistical analysis of overall effect sizes (Glass, 1976). This requires the included studies to share statistical measures (DerSimonian & Laird, 1986) and becomes challenging very quickly if the studies do not apply the same statistical methods (Tranfield et al., 2003, cited in Snyder, 2019). Tranfield et al. (2003), Davis et al. (2014) and Palmatier et al. (2018) tried to adapt the medical approach for the social sciences and to establish guidelines. Statistical metaanalyses in economics have already been published in highly ranked journals (Carrillat et al., 2018). One of the major problems is the quality assessment of the included results. In medical science, with its heavy emphasis on experimental research and the corresponding control groups, this problem is not that significant. In business and management, where most research does not involve control groups, the quality range is much larger. The so-called qualitative systematic review addresses this issue by using qualitative methods to assess the quality of results and to compare them (Greenhalgh et al., 2004). This also enlarges the type of studies included to encompass or even focus on studies using qualitative data collection and analysis methods (Grant & Booth, 2009). In a

13.4  Related Research Designs

247

nutshell, the qualitative systematic review uses a systematic search process and qualitative methods to evaluate the included studies. The systematic literature review with its systematic search process and strict criteria for inclusion is very well suited to aggregate results from studies addressing the same (or a very similar) research question. The search process and the meta-analysis methods demand a narrow focus on a particular topic or question. The breadth of the body of knowledge accumulated is rather small. Semi-systematic literature review Semi-systematic or narrative literature reviews are used to represent the relevant concepts from one discipline and to integrate concepts and frameworks from different disciplines into the body of knowledge (Snyder, 2019). This does not require an infeasible collection of all datasets meeting the selection criteria (this number will be tremendous because of the broadened search in two or even several disciplines (Wong et al., 2013) but can stop the collection process when the relevant concepts have been covered and substantiated. The focus of the analysis then is on the compatibility of the concepts and the discussion about the integration into the research and into existing theories. The semi-structured literature view can thus map a research field, summarize, and synthesize the body of knowledge, identify research gaps, and develop an agenda for future research (Snyder, 2019). Despite of dealing with rather broad and complex topics and very different studies, you need to fully disclose the research process. Semi-systematic reviews can also examine the development of research or research approaches in a particular field over time (Wong et al., 2013). Data analysis methods used in semi-structured literature reviews can, for example, be geared to identify, analyze, and report themes in research like thematic or content analysis (Braun & Clarke, 2006), but also quantitative methods can be used, like statistical meta-analysis (Borman & Dowling, 2008) or time series analysis (Snyder, 2019). Integrative literature review Integrative literature review also aims at integrating existing knowledge into new and elaborated theories. Yet, the integrative literature review follows a more deductive approach instead of the more inductive approach in a semi-systematic literature review (Torraco, 2005). Starting from a preconception of some unifying theory, integrative literature review specifically looks for supporting and repudiating research. This focusses the literature search but also requires several research processes curtailed to the different parts of the theory (Snyder, 2019). The data collection is purpose driven. The purpose in integrative literature review is to combine perspectives to new theoretical models (Snyder, 2019). Hence, the data collection is more creative and less structured (Whittemore & Knafl, 2005). It focuses on the main ideas and relationships of a topic and does not strive for comprehensiveness. This puts a rather heavy burden on the researcher and usually requires advanced skills, both about an overview of the domain and conceptual thinking (MacInnis, 2011). Despite the

248

13  Literature Review Research Design

potentially somewhat eclectic appearing, a full documentation and disclosure of data collection (e.g., sampling, inclusion criteria) and analysis process is required (Torraco, 2005, Snyder, 2019). Multiple case research design A multiple case research design focuses on categories of similarities and differences between the cases. A literature review emphasizing the categorization and grouping of research and the body of knowledge generally uses a multiple case research design (or even a cross-sectional field study design). Longitudinal research design Analyzing the development of the body of knowledge and the respective research uses longitudinal designs, usually with fixed time intervals but also possibly with growth modelling techniques. Design science research design Creating and elaborating theories equals designing an artifact. Thus, design science research considerations become applicable to this part of a literature review. The main evaluation and design criteria are the extent to which existing knowledge can be integrated. Key Aspects to Remember

Produce intellectual contributions without empirical research Theory creation and elaboration as two types of intellectual contributions can be accomplished without empirical research. Most of the empirical research adds intellectual contribution in small steps as every supervisor’s mantra is “focus, narrow down your research question”. Thus, integration of the existing body of knowledge, potentially even combining fields of study, is worthwhile scientific research and goes beyond what is conventionally understood as literature review as in listing existing studies. In fact, a literature review is an excellent way to integrate research findings to show evidence at a meta-level and to integrate it with extant theory as an important step in building and elaborating theoretical frameworks and conceptual models. Think of a sound search strategy if you conduct a literature review Often, no specific, systematic, and well-thought-out search strategy for literature is used in the literature review. Then the literature review deteriorates to a collection of randomly found articles. Based on this sample, the literature review, including its results, becomes arbitrary and cannot generate an intellectual contribution. At a first glance literature review sounds like an “easy” research design because it can be conducted evading two often perceived problems: collecting primary data and the use of statistical methods.

13.4  Related Research Designs

249

While it is possible to circumvent these methods, literature review remains a research design that requires rigor and effort for each step in the demanding design. Take care about a substantial and critical literature review synthesis Often, literature review research is truncated. It stops after the collection of the data from the selected articles. Even after avoiding major pitfalls, this leaves the reader with a list of data, with no further analysis, interpretation, and conclusion. The retrieved data needs to be grouped, categorized, aggregated, and synthesized to depict the relevant body of knowledge. The findings need to be discussed and interpreted, same as in any other research design. Even if “only” used as a step in the research process, formulating the research gap is a conclusion drawn from data. So, the concept should be very familiar. Often, these essential steps lack or are only superficially dealt with in literature review research. Pay special attention to outliers in this research design As with any kind of qualitative data, defining what we should consider an outlier is difficult. We have no rules for this. Judging an article in a literature review as an outlier is not straightforward. If the study is of poor quality, it should be excluded because of the (systematic and rigorous) selection process. But singling a study out as an outlier basically means that inclusion would distort the results of your research or these results of a scientific research should be disregarded. This might be considered a suppression of free speech and excessive political correctness. Both are detrimental to scientific progress. We should avoid excluding empirical studies as outliers at all costs. An effective countermeasure might be to reconsider and refine the selection criteria. This does not mean changing the selection criteria to achieve exclusion of the former “outlier,” but improving the selection process with full disclosure.

Critical Thinking Questions

1. Why does a literature review constitute a compulsory part of every research project? 2. How can you deal with so-called outliers in the research process? 3. What is meant by a “search strategy” in literature review designs? 4. What kind of intellectual contributions do literature reviews produce? 5. How does a typical research gap of a literature review research look like?

Recommendations for further Readings

If you are still unsure whether literature review research design is suitable for your research project, you might find the following literature and readings helpful.

250

13  Literature Review Research Design

• Davis, J., Mengersen, K., Bennett, S., & Mazerolle, L. (2014). Viewing systematic reviews and meta-analysis in social research through different lenses. SpringerPlus, 3, 511. • Torraco, R. J. (2005). Writing integrative literature reviews: Guidelines and examples. Human Resource Development Review, 4, 356–367. • Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14, 207–222. For a discussion of the advantages and disadvantages of the different literature reviews and their applicability, we recommend: • Snyder, H. (2019). Literature review as a research methodology: An overview and guidelines. Journal of Business Research, 104, 333–339.

References Behringer, S., Ulrich, P., & Unruh, A. (2019). Compliance management in family firms: A systematic literature analysis. Corporate Ownership & Control, 17(1), 140–157. Boote, D. N., & Beile, P. (2005). Scholars before researchers: On the centrality of the dissertation literature review in research preparation. Educational Researcher, 34(6), 3–15. Borman, G. D., & Dowling, N. M. (2008). Teacher attrition and retention: A meta-analytic and narrative review of the research. Review of Educational Research, 78, 367–409. Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3, 77–101. Carrillat, F. A., Legoux, R., & Hadida, A. L. (2018). Debates and assumptions about motion picture performance: A meta-analysis. Journal of the Academy of Marketing Science, 46, 273–299. Davis, J., Mengersen, K., Bennett, S., & Mazerolle, L. (2014). Viewing systematic reviews and meta-analysis in social research through different lenses. Springerplus, 3, 511. DerSimonian, R., & Laird, N. (1986). Meta-analysis in clinical trials. Controlled Clinical Trials, 7, 177–188. Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5, 3–8. Grant, M. J., & Booth, A. (2009). A typology of reviews: An analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26, 91–108. Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., & Kyriakidou, O. (2004). Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Quarterly, 82, 581–629. Hart, C. (1999). Doing a literature review: Releasing the social science research imagination. SAGE. Lather, P. (1999). To be of use: The work of reviewing. Review of Educational Research, 69(1), 2–7. LeCompte, M. D., Klingner, J. K., Campbell, S. A., & Menk, D. W. (2003). Editors’ introduction. Review of Educational Research, 73(2), 123–124.

References

251

Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A. & Moher, D. (2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine, 151, W-65. MacInnis, D. J. (2011). A framework for conceptual contributions in marketing. Journal of Marketing, 75, 136–154. Palmatier, R. W., Houston, M. B., & Hulland, J. (2018). Review articles: Purpose, process, and structure. Journal of the Academy of Marketing Science, 46, 1–5. Randolph, J. (2009). A guide to writing the dissertation literature review. Practical Assessment, Research, and Evaluation, 14, (13). Snyder, H. (2019). Literature review as a research methodology: An overview and guidelines. Journal of Business Research, 104, 333–339. (Quelle noch einfügen im Text bei LeComplte Abschnitt). Torraco, R. J. (2005). Writing integrative literature reviews: Guidelines and examples. Human Resource Development Review, 4, 356–367. Tranfield, D., Denyer, D., & Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review. British Journal of Management, 14, 207–222. Webster, J., & Watson, R. T. (2002). Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly, 26, 3. Whittemore, R., & Knafl, K. (2005). The integrative review: Updated methodology. Journal of Advanced Nursing, 52, 546–553. Wong, G., Greenhalgh, T., Westhorp, G., Buckingham, J., & Pawson, R. (2013). RAMESES publication standards: Meta-narrative reviews. BMC Medicine, 11, 20.