Philosophical Foundations of Mixed Methods Research: Dialogues Between Researchers and Philosophers 1032226072, 9781032226071

Citation preview

PHILOSOPHICAL FOUNDATIONS OF MIXED METHODS RESEARCH

Philosophical Foundations of Mixed Methods Research provides a comprehensive examination of the philosophical foundations of mixed methods research. It offers new defences of the seven main approaches to mixed methods (the pragmatist approach, the transformative approach, the indigenous approach, the dialectical approach, the dialectical pluralist approach, the performative approach, and the realist approach) written by leading mixed methods researchers. Each approach is accompanied by critical reflections chapter from philosophers’ point of view. The book shows the value of the use of mixed methods from a philosophical point of view and offers a systematic and critical examination of these positions and approaches from a philosophical point of view. The volume also offers a platform to promote a dialogue between mixed methods researchers and philosophers of science and provides foundations for further research and teaching of this hotly debated topic. This volume is ideal for researchers and advanced students, and anyone who is interested in research methods and the social sciences more generally. Yafeng Shan is Assistant Professor of Philosophy of Science at the Hong Kong University of Science and Technology, Hong Kong. His books include Doing Integrated History and Philosophy of Science: A  Case Study of the Origin of Genetics (Springer, 2020) and Evidential Pluralism in the Social Sciences (Routledge, 2023).

PHILOSOPHICAL FOUNDATIONS OF MIXED METHODS RESEARCH Dialogues between Researchers and Philosophers

Edited by Yafeng Shan

Designed cover image: Zifei Li First published 2024 by Routledge 4 Park Square, Milton Park, Abingdon, Oxon OX14 4RN and by Routledge 605 Third Avenue, New York, NY 10158 Routledge is an imprint of the Taylor & Francis Group, an informa business © 2024 selection and editorial matter, Yafeng Shan; individual chapters, the contributors The right of Yafeng Shan to be identified as the author of the editorial material, and of the authors for their individual chapters, has been asserted in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988. With the exception of Chapter 7, no part of this book may be reprinted or reproduced or utilised in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers. Chapter 7 of this book is available for free in PDF format as Open Access at www.taylorfrancis.com. It has been made available under a Creative Commons Attribution-Non Commercial-No Derivatives (CC-BY-NC-ND) 4.0 license. Trademark notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library ISBN: 978-1-032-22607-1 (hbk) ISBN: 978-1-032-22611-8 (pbk) ISBN: 978-1-003-27328-8 (ebk) DOI: 10.4324/9781003273288 Typeset in Optima by Apex CoVantage, LLC The Open Access version of Chapter 7 was funded by the University of Vienna.

CONTENTS

List of Contributors 1 Assessing Philosophical Foundations of Mixed Methods Research Yafeng Shan

vii

1

PART I

Thus Spoke Researchers 2 A Pragmatist Approach to Mixed Methods Research Martina Y. Feilzer 3 The Philosophical Foundations of a Transformative Approach to Mixed Methods Donna Mertens 4 Philosophical Underpinnings of Mixed Methods: Decolonizing Evaluation Practice Through Decolonizing Paradigms Bagele Chilisa 5 The Dialectic Stance: Navigating Difference Jori N. Hall

11 13

30

54 83

vi

Contents

6 Dialectical Pluralism and Integration in Mixed Methods Research R. Burke Johnson

100

7 A Performative Approach to Mixed Methods Research Judith Schoonenboom

127

8 A Realist Approach for Mixed Methods Research Joseph A. Maxwell

152

PART II

Thus Spoke Philosophers 9 Mixed Methods Research and Deweyan Pragmatism Reconsidered Gert Biesta 10 Mixed Methods Research and Critical Realism Reconsidered Rosa W. Runhardt 11 Mixed Methods and Causal Ontology Christopher Clarke 12 Evidential Partnerships and Multi-Method Research in Political Science: Methodological, Evidential, and Causal Pluralisms Sharon Crasnow Index

169 171

194 210

240

262

CONTRIBUTORS

Gert Biesta is Professor of Public Education at Centre for Public Education and Pedagogy, Maynooth University, Ireland, and Professor of Educational Theory and Pedagogy at Moray House School of Education and Sport, University of Edinburgh, UK. Bagele Chilisa is Professor of Postgraduate Research and Evaluation Programme at University of Botswana, Botswana. Christopher Clarke is Assistant Professor of Theoretical Philosophy at School of Philosophy, Erasmus University of Rotterdam, the Netherlands. Sharon Crasnow is Distinguished Professor Emerita of Philosophy at Norco

College, USA. Martina Y. Feilzer is Professor of Criminology and Criminal Justice at School of

History, Law and Social Sciences, University of Bangor, UK. Jori N. Hall is Professor in the Department of Educational Psychology, University of Illinois, Chicago, USA. R. Burke Johnson is Professor in the Department of Professional Studies, University of South Alabama, USA. Joseph A. Maxwell is Professor Emeritus of Education at School of Education and Human Development, George Mason University, USA.

viii Contributors

Donna Mertens is Professor Emeritus in the Department of Education, Gal-

laudet University, USA. Rosa W. Runhardt is Assistant Professor of Philosophy of Science at Faculty of Philosophy, Theology and Religious Studies, Radboud University Nijmegen, the Netherlands. Judith Schoonenboom is Professor of Empirical Pedagogy in the Department of Education, University of Vienna, Austria. Yafeng Shan is Assistant Professor of Philosophy of Science in the Division of

Humanities, Hong Kong University of Science and Technology, Hong Kong.

1 ASSESSING PHILOSOPHICAL FOUNDATIONS OF MIXED METHODS RESEARCH Yafeng Shan

1.1

Introduction

Mixed methods research, as a methodological movement, emerged in the late 1980s.1 It was mainly developed as a solution to the famous paradigm wars between quantitative research and qualitative research with their underlying philosophical assumptions. However, there is an immediate and urgent question to be addressed for anyone who embraces mixed methods research: how can one mix two methodologies with the conflicting philosophical assumptions? It is widely received that quantitative research and qualitative research differ radically in their ontological, epistemological, and axiological assumptions (e.g. Guba, 1990). Any attempt to integrate quantitative and qualitative methods or data needs to reconcile these incompatible philosophical assumptions in some way. Therefore, the philosophical foundations of mixed methods research have been extensively examined since its birth. There are three central issues of the philosophical foundations of mixed methods research. Motivational issue: What motivates mixed methods research? Justificatory issue: What justifies mixed methods research? Practical issue: What guides mixed methods research? The motivational issue is basically a why question: why does one use mixed methods in research? The justificatory issue is concerning the possibility of mixed methods: are mixed methods viable from a philosophical point of view? The practical issue is more about how to design a mixed methods study or integrate quantitative and qualitative elements in a single study.

DOI: 10.4324/9781003273288-1

2

Yafeng Shan

For the past four decades, a variety of philosophical positions have been developed to answer these questions, including the pragmatist position (e.g. Johnson & Onwuegbuzie, 2004; Morgan, 2007; Teddlie & Tashakkori, 2009; Feilzer, 2010; Johnson et al., 2017; Creswell & Plano Clark, 2018), the transformative position (e.g. Mertens, 2003, 2007; Mertens et al., 2010), the indigenous position (e.g. Wilson, 2008; Chilisa, 2012; Romm, 2018), the dialectical position (e.g. Greene et al., 1989; Greene, 2006; Greene & Hall, 2010), the dialectical pluralist position (e.g. Johnson, 2017), the performative position (Schoonenboom, 2019), and the realist position (e.g. Maxwell & Mittapalli, 2010). However, there is still no consensus on the philosophical foundations of mixed methods research. What is worse, there is no systematic, critical examination of these positions, especially from a philosopher’s point of view. To a great extent, the debate over the philosophical foundations of mixed methods research is an in-house game among some philosophically minded mixed methods researchers. Very few philosophers of science has closely engaged in this important issue in the social sciences. This is very unfortunate. I contend that more dialogues between researchers and philosophers on this issue will lead to fruitful philosophical and methodological implications. This edited volume provides a comprehensive examination of the philosophical foundations of mixed methods research, contributed by both researchers and philosophers. Part I  offers new defences of seven main approaches, written by leading mixed methods researchers. Part II features critical reflections from philosophers’ point of view. It offers a platform to encourage a dialogue between mixed methods researchers and philosophers of science. 1.2

Thus Researchers Spoke

In a recent essay, I propose that there are three different senses of philosophical foundations of mixed methods research in the literature. a) Weak sense: Philosophical foundationsA allow the possibility of the integration of both quantitative and qualitative methods/data/designs. b) Moderate sense: Philosophical foundationsB provide a good reason to use mixed methods in (at least some) social scientific research. c) Strong sense: Philosophical foundationsC justify a normative thesis that mixed methods research should be encouraged in (at least some) social scientific research. (Shan, 2022, pp. 6–7) Martina Y. Feilzer argues for the pragmatist position as a ‘weak’ sense of philosophical foundations in ‘A Pragmatist Approach to Mixed Methods Research’. She highlights two distinctive features of pragmatist

Assessing Philosophical Foundations of Mixed Methods Research 3

thought: anti-representational and anti-dualist. Pragmatist thought is antirepresentational in the sense that it denies that research needs to represent reality in a corresponding way, while it is anti-dualist in the manner that it does not endorse the dualist perspectives on the research and its object or on positivism and constructivism. Feilzer argues that the pragmatist position provides a ‘weak’ sense of philosophical foundations of mixed methods research by justifying the possibility of the integration of quantitative and qualitative elements in research, while questioning the necessity of a search for a ‘strong’ sense of philosophical foundations of mixed methods research. In ‘The Philosophical Foundations of a Transformative Approach to Mixed Methods’, Donna Mertens defends the transformative position as what I called an ‘axiology-oriented’ philosophical foundation (Shan, 2022, p. 5). She maintains that all researchers need to pay attention to ethical issues and aim at an increase in justice ultimately. To this end, Mertens argues that researchers need to address factors that perpetuate discrimination, work in a culturally responsive manner, and promote sustainable actions for transformative change. This axiological assumption is coupled with the transformative ontological and epistemological assumptions, which motivate the use of mixed methods in practice. Mertens concludes that taking a transformative position will ultimately help to provide a basis for improving justice and contribute to a transformed society. In ‘Philosophical Underpinnings of Mixed Methods: Decolonizing Evaluation Practice Through Decolonizing Paradigms’, Bagele Chilisa argues for an indigenous position. It consists of a set of distinctive ontological, epistemological, and axiological assumptions, which radically differ from the EuroWestern ones. Chilisa argues that an indigenous position, by highlighting connectedness and relationality, promotes interaction of knowledge production structures and the importance of building relationships with and among participants and with the environment to improve the quality of data, and provides pathways towards equitable and sustainable futures. In ‘The Dialectic Stance: Navigating Difference’, Jori N. Hall argues that the dialectical position offers a guide to mixed methods research. She begins with a brief overview of the historical development of dialectics in philosophy and social science. Hall notes that there are four assumptions of the dialectic position: differences between philosophical positions exist and are important; these differences cannot be reconciled; all these positions represent legitimate but partial way to understand the world; and dialectical engagement between different positions and methods can result in new knowledge or better understanding. She argues that the dialectical position helps to navigate these differences by guiding mixed methods research design and data analysis. In ‘Dialectical Pluralism and Integration in Mixed Methods Research’, R. Burke Johnson further develops his dialectical pluralistic position, which was originally proposed to complement the dialectical position by articulating its

4

Yafeng Shan

philosophical assumptions (see Johnson, 2017). According to dialectical pluralistic position, there are multiple kinds of social reality, different epistemologies, and multiple ethical theories and values. Johnson argues that researchers should engage with these different ontologies, epistemologies, axiologies, methods, and methodologies dialectically and empirically. To some extent, this dialectical pluralism can be viewed as a synthesis of the pragmatist, dialectical, and pluralist positions. Johnson contends that the dialectical pluralist position is important for providing justification of and guidance for mixed methods research, especially equal-status mixed methods research. In ‘A Performative Approach to Mixed Methods Research’, Judith Schoonenboom argues for the performative position as a ‘strong’ sense of philosophical foundations of mixed methods research. She explores the foundational idea of the performative position. Schoonenboom maintains that different research worlds come into being through research methods and concepts. Accordingly, she argues that the aim of mixed methods research can be construed as an exploration, creation, and coordination of different worlds. Therefore, Schoonenboom concludes that mixed methods research should be encouraged in social scientific research from a performative point of view. Contra Schoonenboom, Joseph A. Maxwell argues that there is only one mind-independent world. He defends a particular realist position in ‘A Realist Approach for Mixed Methods Research’. Maxwell defines realism as the view that (1) entities exist in a mind-independent way and (2) our theories and perceptions of the world are inherently our fallible constructions. He argues that this realist position plays an important role in designing and conducting mixed methods research, especially in the studies of mind, culture, diversity, causation, and research design. In particular, Maxwell argues that a realist approach to causation well reflects the situationally contingent feature of causal mechanisms. 1.3

Thus Philosophers Spoke

Philosophers seem quite critical of these positions. In ‘Mixed Methods Research and Deweyan Pragmatism Reconsidered’, Gert Biesta critically examines the view that pragmatism is the most appropriate paradigm for mixed methods research. His central argument is twofold. On the one hand, Biesta challenges the view that pragmatism is ‘the best paradigm for mixed methods research’. He argues that the concept of paradigm is too ambiguous to be helpful in the discussion and problem-solving is not the only aim of social research. Thus, that ‘whatever works is the best methodology’ oversimplifies and distorts the logic behind mixed methods research. On the other hand, Biesta shows how Dewey’s pragmatism and perspectivalism make sense of mixed methods research. In summary, Biesta cracks down a particular version of the argument that pragmatism provides philosophical foundations of mixed methods

Assessing Philosophical Foundations of Mixed Methods Research 5

research, while argues for a new version of the argument that pragmatism provides philosophical foundations of mixed methods research. In ‘Mixed Methods Research and Critical Realism Reconsidered’, Rosa W. Runhardt challenges the tenability of Maxwell’s realism as a philosophical foundation of mixed methods research. She argues that there is a tension between the realists’ emphasis on the situational contingency of causal mechanisms and its proposed combination with population-level association studies in mixed methods research. In addition, Runhardt argues that Maxwell’s realism is incompatible with Evidential Pluralism, which has been regarded as a philosophical foundation of mixed methods research in causal enquiry (Shan & Williamson, 2023). She concludes that Evidential Pluralism provides a more promising foundation than Maxwell’s realism. In ‘Mixed Methods and Causal Ontology’, Christopher Clarke argues for the significance of the ontological issues in mixed methods research. He argues that different ontological assumptions about the nature of causation have different methodological consequences. In particular, Clarke argues that in political science, the methodological status of triangulation depends on one’s causal ontology. He suggests that the ontological issues ought to be taken seriously in mixed methods research in the social sciences generally. In ‘Evidential Partnerships and Multi-Method Research in Political Science: Methodological, Evidential, and Causal Pluralisms’, Sharon Crasnow suggests that various forms of multi-method research in political science should be regarded as examples of evidential partnership. She argues that evidential partnership can be motivated and justified by three types of pluralism: methodological pluralism, evidential pluralism, and causal pluralism, while admitting that causal pluralism leaves open the possibility that mixed methods research might not be always the best methodology in causal enquiry. 1.4

Remarks

As I have argued earlier (Shan, 2022), the pragmatist position offers a weak, axiology-oriented foundation in the sense that it merely justifies the possibility of the integration of quantitative and qualitative elements in a single study. It highlights the significance of the methodological need (viz., problemsolving) and downplays the significance of the ontological and epistemological commitments. However, Feilzer does not regard this as a weakness or a disadvantage of the pragmatist position. She doubts the necessity of a search for a strong philosophical foundation of mixed methods research. Moreover, Feizler warns that there is a danger of calling for a strong philosophical foundation, because it ‘may ignite a different set of paradigm wars by fighting over the most appropriate and coherent single paradigm for mixed methods research’. For Feilzer, mixed methods research may just need a weak foundation like the pragmatist position.

6

Yafeng Shan

The transformative position is clearly stronger than the pragmatist position. It provides good reasons to use mixed methods in social research rather than justify the possibility of the use of mixed methods. According to the transformative position, social research ultimately aims at a more just and democratic society, and mixed methods research is helpful to achieve this aim. In addition, the transformative position is stronger than the pragmatist position in another sense: the transformative position addresses all of the motivational, justificatory, and practical issues, while the pragmatist position is implicit on the practical issue. In other words, the pragmatist position only provides the justification of mixed methods research in a retrospective way: a given mixed methods research design is possible if it solves problems. However, the pragmatist position itself does not provide guidance on how to design and conduct mixed methods studies. By contrast, the transformative position, as Mertens shows, informs mixed methods research design. Thus, the transformative position provides a moderate, axiology-oriented philosophical foundation. The indigenous position is similar to the transformative position. Both highlight the diversity of culture-based social realities and pay attention to inequalities of knowledge systems and create strategies for knowledge systems to interact. Like the transformative position, the indigenous position also provides a moderate foundation, as it does not argue for a normative thesis that mixed methods research ought to be preferred to quantitative or qualitative research.2 However, the indigenous position is distinctive: it is based on distinctive indigenous relational ontological, epistemological, and axiological assumptions, while the transformative position is still largely framed by the Euro-Western concepts such as justice and democracy. In addition, the indigenous position differs from the transformative position in the way that the latter is an axiology-oriented position, while the former is fundamentally ontology-oriented. As the name suggests, the core of the transformative position is an axiological thesis with a transformative aim. The indigenous position stems from an indigenous theory of social ontology: relational ontology. Such a distinctive ontological theory leads to distinctive epistemological and axiological assumptions which motivate and justify the use of mixed methods. Thus, the indigenous position offers a moderate, ontology-oriented foundation of mixed methods research. By contrast, the dialectical positions provide a strong, axiology-oriented foundation by calling for close engagement of different methodologies in respectful dialogue. Hall shows that adopting a dialectical position may have fruitful consequences in research. However, there is a further metajustification problem: why ought one to take the dialectical position? In addition, the dialectical position does not say much about the practical issue. As Hall admits, ‘there is no prescription for employing the dialectic stance’. In short, the dialectical position is implicit on why one should adopt it and how one can apply it in research. Johnson’s dialectical pluralist position can

Assessing Philosophical Foundations of Mixed Methods Research 7

be viewed as a refined and strengthened version of the dialectical position by articulating the ontological, epistemological, and axiological assumptions and providing more explicit guidance for mixed methods researchers. In other words, the dialectical pluralist position addresses all the motivational, justificatory, and practical issues. Moreover, Johnson argues that the dialectical pluralist position motivates and justifies equal-status mixed methods research, which to a great extent responds to one of my earlier concerns, the problem of scope (Shan, 2022, p. 8): how widely should mixed methods research be encouraged? It seems that Johnson is optimistic about wide scope of mixed methods research, given that the ‘most challenging sort of’ mixed methods research can be motivated and justified by the dialectical pluralist position. Although the basic idea of the dialectical and dialectical pluralist positions is appealing, it is difficult to see how it can be applied in practice. Let us consider Johnson’s dialectical pluralist ontological principle: Recognise multiple kinds of reality and the presence of different ontologies and the tensions they produce as a strength to be embraced rather than a weakness that stunts growth. Researchers can produce from this a new, practical ontological mix or package of relevant ideas for each research study. It is unclear how one can embrace two conflicting ontologies in a single study. For example, in causal enquiry, as Clarke argues, ‘different ontological assumptions about the nature of causation entail different conclusions about what mixed methods research needs to do in order to deliver successful causal inferences’. One possible solution is, as Johnson and his associates (Johnson et al., 2019) suggest, to appeal to causal pluralism, which is the view that there are different types of causal relationships out there. By making a commitment to causal pluralism, mixed methods researchers may be able to accommodate different, conflicting ontological assumptions in their research design. That said, this poses a limit of the scope of mixed methods research. In causal enquiry, mixed methods research ought to be encouraged only in the cases that causal pluralism is assumed.3 Maxwell’s realist position provides a moderate, ontology-oriented foundation. It is ontology-oriented in the sense that the realist ontological assumptions play a key role in the motivation and justification of the use of mixed methods. It is moderate in the sense that it only provides some good reason to use mixed methods. As Maxwell himself emphasises: I’m not arguing that realism is the single ‘correct’ approach to mixed methods research, only that it has insights and advantages that other stances lack, and that it is thus a valuable conceptual tool in a researcher’s toolkit. My purpose . . . is to indicate areas where I think a realist perspective can be useful for mixed methods researchers.

8

Yafeng Shan

However, a main problem for the realist position is that it fails to justify the indispensability of quantitative methods, as Runhardts argues. Maxwell (2012) employed the realist position to motivate and justify qualitative research, but it is dubious that it can be equally applied to mixed methods research. Schoonenboom’s performative position is particularly interesting. It was originally somehow built upon Johnson’s dialectical pluralist position. As Schoonenboom (2019, p. 295) admits, ‘The ontology and epistemology of the performative paradigm stem from dialectical pluralism’. However, it is clearly different from the dialectical pluralist position: the performative position provides a strong, ontology-oriented foundation, whereas the dialectical position offers a strong, axiology-oriented foundation. Schoonenboom explicitly argues for a particular theory of social ontology: worlds are results of our research and do not exist mind independent. Accordingly, different research methods contribute to create different worlds. Thus, mixed methods research is well motivated and justified given that all research is assumed to create and coordinate worlds. It is clear that this volume does not provide the reader with the definite answers to the questions concerning the philosophical foundations of mixed methods research. Neither does it cover all the relevant issues. There is much more to explore in the future. For example, do we really need a strong sense of philosophical foundations of mixed methods research? Are ontological assumptions indispensable to motivate and justify the use of mixed methods? Is the tripartite analysis (by examining ontological, epistemological, and axiological assumptions) a useful tool to analyse and examine the philosophical foundations of mixed methods research? That being said, I contend that these issues can be more promisingly explored with the collaboration between researchers and philosophers. Notes 1 If not specified, mixed methods research refers to a methodology or a methodological orientation in this chapter. 2 See Chilisa and Phatshwane (2022), for a discussion of the indigenous position and qualitative research. 3 Jon Williamson and I argue that Evidential Pluralism, whose basic idea is that one ought to have both evidence of correlation and of mechanisms in order to establish a causal claim, motivates and justifies the use of mixed methods in causal enquiry (Shan & Williamson, 2023, chapter 4). Given that Evidential Pluralism assumes causal monism, it seems inconsistent with the implication of the dialectical pluralist position.

References Chilisa, Bagele. (2012). Indigenous Research Methodologies. Thousand Oaks, CA: SAGE Publications, Inc.

Assessing Philosophical Foundations of Mixed Methods Research 9

Chilisa, Bagele, & Phatshwane, Keneilwe. (2022). Qualitative research within a postcolonial indigenous paradigm. In Uwe Flick (Ed.), The Sage Handbook of Qualitative Research Design (pp. 225–239). London: Sage. Creswell, John W., & Plano Clark, Vicki L. (2018). Designing and Conducting Mixed Methods Research. London: Sage. Feilzer, Martina Yvonne. (2010). Doing mixed methods research pragmatically: Implications for the rediscovery of pragmatism as a research paradigm. Journal of Mixed Methods Research, 4(1), 6–16. Greene, Jennifer C. (2006). Towards a methodology of mixed methods social inquiry. Research in the Schools, 13(1), 93–98. Greene, Jennifer C., Caracelli, Valerie J., & Graham, Wendy F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255–274. Greene, Jennifer C., & Hall, Jori N. (2010). Dialectics and pragmatism: Being of consequences. In Abbas Tashakkori & Charles Teddlie (Eds.), Sage Handbook of Mixed Methods in Social and Behavioral Research (2nd ed., pp.  119–144). Thousand Oaks, CA: SAGE Publications, Inc. Guba, Egon G. (Ed.). (1990). The Paradigm Dialog. Newbury Park, CA: Sage. Johnson, R. Burke. (2017). Dialectical pluralism: A  metaparagidm whose time has come. Journal of Mixed Methods Research, 11(2), 156–173. Johnson, R. Burke,  & Onwuegbuzie, Anthony J. (2004). Mixed methods research: A research paradigm whose time has come. Educational Research, 33(7), 14–26. Johnson, R. Burke, Russo, Federica,  & Schoonenboom, Judith. (2019). Causation in mixed methods research: The meeting of philosophy, science, and practice. Journal of Mixed Methods Research, 13(2), 143–162. Johnson, R. Burke, de Waal, Cornelis, Stefurak, Tres, & Hildebrand, David L. (2017). Understanding the philosophical positions of classical and neopragmatists for mixed methods research. KZfSS Kölner Zeitschrift Für Soziologie Und Sozialpsychologie, 69, 63–86. Maxwell, Joseph A. (2012). A Realist Approach for Qualitative Research. Thousand Oaks, CA: SAGE Publications, Inc. Maxwell, Joseph A., & Mittapalli, Kavita. (2010). Realism as a stance for mixed methods research. In Abbas Tashakkori  & Charles Teddlie (Eds.), Sage Handbook of Mixed Methods in Social and Behavioral Research (2nd ed., pp. 145–168). Thousand Oaks: SAGE Publications, Inc. Mertens, Donna M. (2003). Mixed methods and the politics of human research: The transformative-emancipatory perspective. In Abbas Tashakkori  & Charles Teddlie (Eds.), Sage Handbook of Mixed Methods in Social and Behavioral Research (1st ed., pp. 135–164). Thousand Oaks, CA: Sage. Mertens, Donna M. (2007). Transformative paradigm: Mixed methods and social justice. Journal of Mixed Methods Research, 1(3), 212–225. Mertens, Donna M., Bledsoe, Katrina L., Sullivan, Martin,  & Wilson, Amy. (2010). Utilization of mixed methods for transformative purposes. In Abbas Tashakkori & Charles Teddlie (Eds.), Sage Handbook of Mixed Methods in Social and Behavioral Research (2nd ed., pp. 193–214). Thousand Oaks, CA: SAGE Publications, Inc. Morgan, David L. (2007). Paradigm lost and pragmatism regained. Journal of Mixed Methods Research, 1(1), 48–76. Romm, Norma R. A. (2018). Responsible Research Practice. Cham: Springer.

10

Yafeng Shan

Schoonenboom, Judith. (2019). A performative paradigm for mixed methods research. Journal of Mixed Methods Research, 13(3), 284–300. Shan, Yafeng. (2022). Philosophical foundations of mixed methods research. Philosophy Compass, 17(1), e12804. https://doi.org/10.1111/phc3.12804. Shan, Yafeng, & Williamson, Jon. (2023). Evidential Pluralism in the Social Sciences (1st ed.). Philosophy and Method in the Social Sciences. London and New York: Routledge. Teddlie, Charles,  & Abbas Tashakkori. (2009). Foundations of Mixed Methods Research. Thousand Oaks: Sage. Wilson, Shawn. (2008). Research Is Ceremony. Black Point and Winnipeg: Fernwood Publishing.

PART I

Thus Spoke Researchers

2 A PRAGMATIST APPROACH TO MIXED METHODS RESEARCH Martina Y. Feilzer

2.1

Origin of Pragmatist Thought

Pragmatism as a philosophical approach emerged in North America in the late 19th century in the work of Charles Sanders Peirce, a physical scientist focused on improving the accuracy of scientific measurement as well as the study and teaching of logic. Peirce developed his ‘scientific method’ from the view that reality exists only in relation to experience and social, historical, and economic context. He further realised through years of aiming to perfect the accuracy of measuring the intensity of gravitational fields that scientific and physical laws could not be proven through observation and that observations of physical laws displayed mere habits or regularities that could easily be disrupted by spontaneity. Hence, he started questioning scientific determinism and the dominant paradigm of positivism (Peirce, 1992–1998). Peirce’s early discussions of pragmatism or pragmaticism were developed by other ‘classical pragmatists’ such as William James – who mentioned the term pragmatism in a lecture in 1889 (Dickstein, 1998, p. 1) – and John Dewey before it is claimed that Richard Rorty and others created a second form or ‘new’ pragmatism. Pragmatism is marked by controversy, transformations, and a certain ‘muddledness’ (Rorty, 1991, p. 64), leading one commentator to suggest that there are ‘in effect, . . . two pragmatisms’ (Mounce, 1997, p. 2), if not more. Considering the various discussions and developments of the philosophies of pragmatists, it may be easier to consider pragmatism as a family of approaches ‘sharing a certain family resemblance’ (Suckiel, 1998, p. 308) or system of thought and ideas (Nisbet, 1993, p. 3) rather than a fully formed philosophical paradigm. Indeed, in one of his early lectures on pragmatism, James called it a ‘method only’ highlighting pragmatism’s empiricist DOI: 10.4324/9781003273288-3

14

Martina Y. Feilzer

attitude when looking for the practical or concrete consequences of research problems (James, 1906/1907, p. 40). Pragmatist thought made its significant contribution to philosophy and research in its challenge to the dualism(s) of the dominant paradigms of positivism and idealism/constructivism. Positivism underpinning virtually all scientific inquiry subscribing to the notion of a singular metaphysical reality, the one and only truth that is capable of being discovered by objective and value-free inquiry, entirely independent of the researcher. In positivism, a clear distinction – dualism – is established between reality and those who ‘happen’ to research it. This ontological and epistemological view is contrasted in idealism/constructivism with the idea that the researcher and the subject of the research co-create a temporarily located reality, that there is no such thing as a single objective reality, and that ‘subjective inquiry is the only kind possible to do’ (Creswell  & Plano Clark, 2007; Erlandson et  al., 1993, p. xi; Teddlie & Tashakkori, 2009, p. 88). Over time, specific research methods and the two main paradigms were linked and regarded as the only methods epistemologically capable of measuring their respective realities, each method – surveys, case studies, and biographical interviews – with their own disciplinary histories. In time, researchers were trained in the distinctive skill sets for quantitative and qualitative research leading to specialisation and researchers started defining themselves through their methodological orientation as quantitative, qualitative, or mixed methods researchers (Jick, 1979; Morgan, 2007, pp. 60–61). The alignment of quantitative and qualitative data with the two dominant paradigms and, at least in some cases, their disciplinary origin led to the notion that quantitative and qualitative methods are incompatible and should not be mixed and that the different paradigms and the resultant data are incommensurable and cannot be compared or even translated across. The so-called paradigm wars reflected these arguments but were seen as largely resolved by the late 1990s which also saw a resurgence of mixed methods research (Teddlie  & Tashakkori, 2009, p.  15). Howe (1988, p.  15) suggests that these debates are, to a large extent, a result of philosophers constructing ‘superconcepts such as “Truth”, “Reality”, . . .’ and ‘generating insoluble pseudoproblems’ in the process. Similarly, in the social sciences, Hanson (2008, pp. 103–106) argues that the distinctions between social phenomena as objective or subjective are primarily a result of political divisions between social scientists rather than philosophical or theoretical ones. Pragmatism challenged these dualist positions and accepted philosophically that there are both singular and, to some extent, external and independent realities as well as multiple transformations of these realities in subjective experiences. Dewey (1925, p. 40) uses the term ‘existential reality’ in referring to an experiential world which can hold both qualities of objective external layers and subjective elements – both the ‘stable and the precarious’ (Dewey,

A Pragmatist Approach to Mixed Methods Research

15

1925, p. 40), layers of ‘completeness, order, recurrences which make possible prediction and control, and singularities, ambiguities, uncertain possibilities, processes going on to consequences as yet indeterminate’ (Dewey, 1925, p. 47). Criticism of pragmatism is often levelled against a sense of relativism and anti-realism, whereas close reading of both Dewey (1925, p.  47) and later Rorty (1999, p. ixx) suggests that they are making a distinction between views of reality – some of it external and objective and some of it subjective – and the equation of such differing views of reality with a singular aim of truthful representation. Thus, a further major divergence in pragmatism from both positivist and constructivist paradigms comes from a rejection of the notion that research needs to represent reality ‘truthfully’. James (1906/1907, p. 34) suggested that pragmatism’s method and ambition started out as looking primarily at settling metaphysical disputes by trying to understand the practical consequences of the underlying questions and creating corresponding knowledge. Both objective and subjective realities remain open to empirical inquiry but require a purpose, a link, to a practical consequence to offer meaningful and usable knowledge. Pragmatists’ contention is that both positivism and constructivism seek truth in representation and thus derive from the same paradigm family, they are not dualist/dichotomous approaches at an ontological or epistemological level (Rorty, 1999, p. ixx; Hanson, 2008; Johnson & Onwuegbuzie, 2004). As a result, a measure of successful production of knowledge for positivists and constructivists alike is how closely research measures and findings represent reality whether an objective external one or multiple, relative, and subjective ones. This anti-representational perspective is linked to pragmatisms’ axiological position, the role of values, and what Rorty called the concept of utility, namely that, rather than aiming for research to provide an ‘accurate account of how things are in themselves’, it should ‘aim at utility for us’ (Rorty, 1999, p. xxvi). The concept of utility is contested, and it has been linked to utilitarianism, accused of risking conservatism in research as researchers might fall back on the consensus among their peers about the questions that are worth asking (Morgan, 2007, p. 66), and it has been dismissed as a simplistic justification of ‘what works’ and evaluation research (Denzin, 2012, p. 83). Arguably, the concept of utility goes back to the foundational critique of making a metaphysical distinction between objective and subjective realities instead of considering what the implications are of understanding a social phenomenon in its practical application, its consequences, and what actions should arise from such an understanding. In other words, knowledge should aim to respond to practical problems in the ‘real world’ (Creswell  & Plano Clark, 2007, pp. 20–28; Dewey, 1925; Rorty, 1999). Which values drive the research are not prescribed; pragmatism merely suggests that researchers ought to reflect on the purpose and practical application of the research which has

16

Martina Y. Feilzer

intrinsic value, in addition to how their values may affect the research and the interpretation of any research findings. This sits in contrast with the transformative perspective which implies a specific value set for research, that of addressing social injustice (Mertens, 2007, p. 212). So, pragmatism has an anti-dualist perspective on two levels, one in that is does not recognise the dualism of the researcher and its object, and another in that it rejects the dichotomy between constructivism and positivism at an ontological level – the compatibility thesis suggesting that quantitative and qualitative methods can be combined in research. Pragmatism also holds an anti-representational perspective of the production of research knowledge focusing on the importance of purpose and values in interpreting research results and the practical application of any research knowledge produced. In addition to the ontological and epistemological aspects of his thinking, Peirce’s interest in logic led to him developing a third form of inferential logic to his scientific or pragmatic method over the course of his lifetime. Rather than building a theoretical framework on the basis of prior knowledge, formulating hypotheses, and then testing these (deduction) – or reviewing prior knowledge, making observations, going through a process of checking whether observations match existing theories and if not suggesting propositions and then building theory (induction) – abduction is seen as a weaker form of inference of starting from observation to a basic explanation. Peirce (1903, p.  227 cited in Paavola, 2015, p.  233) called it an ‘act of insight, although of extremely fallible insight’. Abduction as part of a process of enquiry makes propositions after observation, a reasoning backwards which can then be subjected to testing, it involves moving from an observation to a possible explanation to then create opportunities for testing the explanation and creating new knowledge (Paavola, 2015, p. 232). The characteristic process is one of moving ‘back and forth between induction and deduction – first converting observations into theories and then assessing those theories through action’ (Morgan, 2007, p. 71). All three forms of inferences should be considered together in research analyses but abduction offers a logic that can explain surprising or unexpected events and initiate new theory (Mounce, 1997, p. 19; Teddlie & Tashakkori, 2009, p. 89). It will become clear later why its logic also aids analyses in mixed methods studies. Another significant contribution of pragmatist to research philosophy is the full commitment to the limitations of knowledge – fallibilism or fallibility – recognising that it is impossible to verify a theory or prove a causal proposition but also acknowledging the limitations created though the relationship between any knowledge produced and the researcher, as well as the specific spatial, temporal, and social context in which the knowledge has been created. There is a clear distinction here to philosophical scepticism which suggests that we cannot know anything. It reflects the pragmatist’s appreciation of

A Pragmatist Approach to Mixed Methods Research

17

the stability of relationships and structures while recognising that our knowledge of these stable features is always contextual and open to, or relative to, dynamic shifts and changes dependent on precarious and unpredictable occurrences and events (Mounce, 1997, pp. 99–101). Thus, I would suggest that there are some key contributions that pragmatist approaches bring to mixed methods research and social science more generally. Pragmatist approaches are anti-dualist, anti-representational, committed to the limitations of knowledge and, with that, aim to be self-correcting (Morales, 2003, p. xiii). In addition, pragmatism emphasises the use of a third logic to the solving of research problems and can be regarded as well suited to justifying mixed methods research and responding to the challenges of conducting mixed method studies and analysing research findings emerging from different types of enquiry. 2.2

Pragmatist and Other Approaches Relevant to Mixed Methods Research

In this chapter, the focus is on the elements of pragmatism that speak directly to ontological and epistemological questions relevant to mixed methods research, research using both quantitative and qualitative research methods in a single study in a variety of research designs. It argues for the rediscovery of pragmatism as a practically relevant philosophical approach for all types of research (Denscombe, 2008, pp. 273–275) while recognising its particular significance for mixed methods researchers. It suggests that a pragmatic approach to problem-solving in the social world offers a flexible, contemporary, and more reflexive guide to research design and grounded research without suggesting that it is the only approach that should be considered. The 21st century’s newfound enthusiasm for mixed methods research and its proclaimed importance as a third methodological movement (Doyle et al., 2009, p.  184; Hall, 2013, p.  2) have resulted in significant debates about the philosophical framework that can justify the use of different types of methods, range of research designs, and analytical approaches (Creswell & Plano Clark, 2007, pp. 26–28). At the level of paradigms, Teddlie and Tashakkori (2009, pp. 87–88) contrast five major paradigms to consider in these discussions – constructivism, the transformative paradigm, positivism, postpositivism, and pragmatism. Shannon-Baker (2016, pp. 323–324) would add critical realism and dialectics as important paradigmatic perspectives to this list. Shan (2022, p. 5) makes a distinction between philosophical foundations for mixed methods research that are ontology-oriented and those that are axiologically oriented and suggests that the call for philosophical foundations ranges from calling for weak foundations – offering a justification and permission for mixing qualitative and quantitative methods/data and design; moderate foundations – providing ‘a good reason for using mixed methods’;

18

Martina Y. Feilzer

or strong foundations of mixed methods research – offering a ‘normative thesis that mixed methods should be encouraged’ (Shan, 2022, pp. 6–7). Shan’s (2022, pp. 6–7) discussion of what type of philosophical foundation proponents of mixed methods research are looking for is tied to the concept of paradigms. The term paradigm has its foundation in Kuhn’s work where he describes paradigms as an ‘accepted model or pattern’ (Kuhn, 1962, p. 23), an organising structure that links epistemological approaches directly to deeper philosophical positions relating to the nature of social phenomena and social structures. In Kuhn’s original framing of paradigms, paradigms directly determine and direct research – questions and methods – and thus become prescriptive. In that sense, looking for a paradigm that underpins mixed methods research means looking for a strong philosophical foundation that sees the use of mixed methods research as linked directly to the underpinning ontological worldviews, in the same way that positivist consider the use of quantitative methods as a logical response to their understanding of social phenomena and constructivists see qualitative methods as the only ones capable of measuring multiple, subjective realities. However, in the efforts of situating mixed methods research in a research paradigm, debates have erupted around the use and conceptualisations of paradigms ranging from interpreting paradigms as tools useful for the research process or fluid constructed entities to a wholesale rejection of the concept and a consideration instead of stances towards research or mental models (for a brief summary, see Shannon-Baker, 2016, p. 319). Thus, maybe unsurprisingly, there is no consensus among mixed methods researchers as to the nature of the philosophical foundation for mixed methods research. Does mixed methods research require a single underpinning worldview or can mixed methods research be a-paradigmatic/non-paradigmatic, in other words, a link between ontology, epistemology, and research methods is unnecessary. Alternatively, proponents of the multiple paradigms thesis and dialectics argue that researchers can choose from a range of paradigms appropriate for mixed methods research by recognising and respecting their underlying differences, offering an opportunity for pragmatist thought to be fully included in mixed methods design where such inclusion has utility (Tashakkori & Teddlie, 1998, pp. 96–100). As already mentioned, the distinctions between paradigms, approaches, or stances relate to the extent to which their proponents insist that research methods need to be consistent and internally coherent with their underlying philosophical foundations requiring articulation and justification of methodological and analytical choices. In terms of mixed methods research, the approach most commonly listed as a philosophical foundation or orientation is pragmatism (Teddlie & Tashakkori, 2009, p. 7), an alternative to the worldviews of positivism/postpositivism and constructivism with a focus on the problem to be researched, or research question to be answered, and the

A Pragmatist Approach to Mixed Methods Research

19

impact and consequences of the research findings in the real world (Brewer & Hunter, 1989, p. 74; Creswell & Plano Clark, 2007, p. 26; Miller, 2006; Tashakkori & Teddlie, 1998, pp. 29–30). The different versions of pragmatist thought and the need to further develop the philosophical coherence of pragmatism make it difficult to argue that pragmatism is a coherent and fully fledged philosophical worldview. Additionally, and most importantly in relation to the aforementioned discussion, pragmatism does not require a link between its philosophical foundation and the chosen method – pragmatism does not call for mixed methods research, it only asks that the methods chosen in research are capable of answering the research questions asked. So rather than presenting a full paradigm, Morgan (2007) and Shannon-Baker (2016) suggest that pragmatist approaches to research are a more accurate description of what is in evidence in both pragmatist philosophy and its application to research practice and mixed methods research. Nevertheless, the views as to which foundation should underpin mixed methods research are important. Creswell and Plano Clark (2007, p.  26) describe three alternative stances on the paradigm issue; Greene et al. (2001, p. 28) list four frameworks for mixing methods – pragmatism, dialectics, substantive theory, and an alternative paradigm – and in 1998, Tashakkori and Teddlie discuss one framework – pragmatism – in detail, but include another framework, the transformative perspective, ten years later (Teddlie & Tashakkori, 2009, p. 87). In 2017, Johnson suggests that dialectical pluralism should be considered a meta-paradigm for mixed methods research – ‘an approach to research and policy that carefully, “fully,” and respectfully listens to two or more paradigms’ (Johnson, 2017, p. 159). It is clear from this short discussion that the search for a philosophical foundation for mixed methods research continues and that agreement as to what type of philosophical foundation is desired – weak, moderate, or strong – may be required as an initial step forward. A focus on these discussions and debates should not detract from the recognition that there are some important connections between the different frameworks put forward as supporting mixed methods research. Some of these connections are at the axiological level – the importance of the role of values links transformative, dialectical, and pragmatist positions – or at the ontological level where connections can be made between critical realism and pragmatism, both adopting the compatibility thesis and accepting social structures and individual agency – layers of the stable and the precarious. But, of course, all these different paradigms, approaches, and stances also differ in important ways. Rather than spending time here on delineating the pragmatist approach against some of the others listed here, I would refer the reader to the work by a number of excellent commentators who have already done so through setting out very helpful paradigm contrast tables – for

20

Martina Y. Feilzer

example, Teddlie and Tashakkori (2009, p. 88) who contrast constructivism, transformative, pragmatism, postpositivism, and positivism – or discussing the main differences between perspectives for mixed methods research – for example, Shannon-Baker (2016, pp.  323–324) who looks at pragmatism, transformative-emancipation, dialectics, and critical realism. In the previous section, I have set out some of the major contributions of pragmatism to the debates linking questions of ontology, epistemology, and research practice, and in the next section, I will suggest why these contributions make pragmatism such ‘an attractive philosophical partner to mixed methods research’ (Johnson  & Onwuegbuzie, 2004, p.  14). This may well be regarded only as a weak philosophical foundation for mixed methods research, but I would argue an important one and one that is uniquely placed to support mixed methods researchers in overcoming the practical challenges of using mixed methods research. 2.3

Pragmatist Approaches and Mixed Methods – From Research Design to Analyses and Beyond

In 2013, Hall wrote a paper titled Mixed Methods: In Search of a Paradigm setting out the reasons why a single paradigm is needed to support the development of mixed methods research as a third methodological movement. In this chapter, reference is made to the paradigm wars of the 1970s and 1980s, and while many commentators agree that these ‘wars’ have now been laid to rest, the two main paradigms are still dominating methodological textbooks, epistemological debates, and teaching of new researchers in social sciences (Hughes & Sharrock, 2007; Teddlie & Tashakkori, 2009, pp. 14–16; Morgan, 2007, pp. 60–61). Thus, in some respects, there is a danger that calling for a strong philosophical foundation (Shan, 2022, p. 9) may ignite a different set of paradigm wars by fighting over the most appropriate and coherent single paradigm for mixed methods research. Rather than adding to this debate, I will set out, in this section, why pragmatism is a coherent philosophical partner to mixed methods research and that if carefully considered it could be considered a strong philosophical foundation that encourages the use of mixed methods in response to most research questions that are designed based on a pragmatist view of the social world. I will conclude this chapter by arguing that there is room for further development of pragmatism as a coherent approach and emphasise its value in considerations of mixed methods research and wider social research practice. Mixed methods designs can take many different forms and respond to different research settings – theory driven or applied research – but it has been used most widely in applied research in health and education, and in evaluation research. It appeals to researchers who deal with the ‘complexity of social phenomena’ in which individuals live in ‘tangled and everchanging

A Pragmatist Approach to Mixed Methods Research

21

contexts’ and recognise ‘the limitations of using just one research approach for capturing this complexity’ (Greene et al., 2001, pp. 25–26; see also Doyle et al., 2009, pp. 175–176). Reviews of mixed methods research have identified a number of key strengths, namely triangulation, greater validity of research through the use of quantitative and qualitative data; completeness, using different methods leads to a more comprehensive picture of the social phenomenon in question; offsetting weaknesses by overcoming the limitations of particular methods; answering different research questions by offering different perspectives; explanation of findings, using one method to explain the findings from another methods; illustration of data, using one method to help paint a picture of the findings of another; hypotheses development and testing, using one method to develop hypotheses to be tested; instrument development and testing, to use one method to design the other to be used in a later stage of the research (Doyle et al., 2009, pp. 178–179). The strengths attributed to mixed methods research resonate strongly with the key contributions of pragmatism as a philosophical approach, and these will now be discussed in relation to the different stages of mixed methods research. Some of the strengths identified are linked to the very beginning of a research project, the set of research questions to be answered, and one of the core principles of pragmatism, the ‘dictatorship’ of the research question (Tashakkori & Teddlie, 1998, p. 20). At the outset, researchers should ask the question of ‘what the research is for’ and ‘who it is for’ and reflect on how their values might influence the research. Some of the discussions in the methods literature suggest that pragmatists subordinate ontological and epistemological concerns to the research questions suggesting a non-paradigmatic stance. This view neglects the influence of a pragmatist perspective on how research questions are generated as well as how particular disciplines situate research and relevant research questions within an accepted ontological frame or a shared ‘research culture’. So, for example, health research is shifting from a dominant positivist tradition of assessing effectiveness of clinical interventions solely through quantitative methods of random control trials to a mixed methods approach recognising the important role patients and their families play in mediating effectiveness and acceptability of clinical interventions and the need to assess their perspectives. The move from evidence-based medicine relying on traditional quantitative and experimental methods – randomised control trials (RCTs), systematic reviews, and meta-analyses – ‘designed to answer simple, focused questions in a stable context’ (Greenhalgh et al., 2022, p. 253), reflects a recognition of the complex social systems and real-world uncertainties that can lead to dynamic change and undermined the traditional biomedical (or positivist) paradigm (Greenhalgh et al., 2022, p. 256). This shift could give rise to the use of a number of methodological approaches and philosophical justifications including pragmatism, critical realism, and dialectics.

22

Martina Y. Feilzer

However, in their discussion of EBM+, Greenhalgh et al. (2022, p. 256) specifically highlight the use of a mixed-methods design which allows for ‘rich description of a phenomenon in context’, thus recognising the need to bring together research looking at an issue from a variety of perspectives in order to account for the many precarious influences on human behaviour. Indeed, the UK’s Medical Research Council most recent guidance of complex interventions now states explicitly that quantitative methods alone are rarely adequate for the assessment of complex interventions and that qualitative and mixed methods designs need to be considered (Skivington et  al., 2021, p.  7). An acknowledgement of the limitations of mono-method research in understanding social phenomena in the health context has led to the development of the complex systems paradigm as set out by Greenhalgh et al. (2022, p. 258). Not only does this paradigm call for the use of mixed methods research but also the discussions around the consideration of a wider range of methods and approaches in health research resonate strongly with pragmatist thought – see for example the reference to the need for research to ‘maximise usefulness for decision makers’ (Skivington et al., 2021, p. 7) and the acknowledgement that ‘data will never be complete’ (Greenhalgh et al., 2022, p. 258). It stands to reason that pragmatists who see social phenomena as multidimensional and as consisting of layers of stable patterns and structures affected by precarious, subjective, and agentic activity, would generate research questions that address these multiple dimensions through the use of different methods. The answers to such questions would offer insights into multi-dimensional phenomena and knowledge that would allow for practical consequences and actions to be taken in relation to the phenomena observed. While researchers do not need to be constrained by the use of a specific method (Robson, 1993, p. 291), mixed methods designs can respond effectively to that challenge by allowing researchers to looking at phenomena from different perspectives, to test the objective or stable aspects of a phenomenon through quantitative methods and the agentic one through qualitative methods, thus providing an enriched understanding (Jick, 1979, pp. 603–604). Of course, pragmatists recognise that both quantitative and qualitative methods have the potential to elucidate both stable and precarious elements of the social phenomena in their own right. Nevertheless, while pragmatists do not stipulate a particular method or method mix, mixed methods designs seem best suited to answering the types of questions pragmatist researchers would ask. It appears that a number of disciplines – for example, health as discussed earlier – are now ready to embrace and acknowledge the complexity of the systems in which social phenomena are situated and with that are developing a different set of research questions better suited to the use of mixed methods research. The development of mixed methods research has brought with it a multitude of typologies and classifications of different research designs involving

A Pragmatist Approach to Mixed Methods Research

23

different methods; different ordering of methods; subordination of one method to the other or weighting of methods; and the stage of mixing methods and data, that is, during analysis or interpretation. Whether or not different methods should have equal status or one should dominate, whether they should be deployed sequentially or concurrently, and whether or not data are transformed – that is, whether qualitative data are quantified or quantitative data are qualified – depends on the specific research questions asked (for a discussion of the different typologies and models of mixed methods research design, see Doyle et al., 2009, pp. 179–183). From a pragmatist perspective, the main requirements of mixed methods design are the recognition of the compatibility of both methods – the convergence of both qualitative and quantitative methods which supports data transformation; and the preference for moving past a presentation of the findings from the different data types by juxtaposition, as ‘totally or largely independent of each other’ (Bryman, 2007, p. 8). This reiterates the argument that quantitative and qualitative methods are not different at an epistemological or ontological level and share many commonalities in their approaches to inquiry (Hanson, 2008; Johnson & Onwuegbuzie, 2004). This acceptance of methodological compatibility also allows researchers to recognise that quantitative methods can generate qualitative data even if that is not desired or part of the data collection design. One example of this is the marginalia created on survey research, the spontaneous notes or comments offered by participants ‘speaking back’ which has been largely ignored as undesired and unanticipated communication (Muddiman et  al., 2018, p.  294; also see Feilzer, 2010, p.  11). It highlights a key advantage of pragmatism as a philosophical foundation, as it offers a chance but also a requirement to consider data regardless of methodological or ontological constraints as it is simple regarded as research participant generated data and should thus be fully considered in analysis and the interpretation of findings. In this sense, it may answer some of the critics of recent conceptions of mixed methods research who consider the dominance of quantitative methods in mixed methods research designs a threat to the full inclusion of stakeholders into the research process (Denzin, 2012, p. 81). Pragmatism makes a commitment to asking questions that involve the different perspectives of a social phenomenon which calls for all stakeholders to be heard, in whichever form they wish to speak. Its ‘acknowledgement of the unpredictable human element forces pragmatic researchers to be flexible and open to the emergence of unexpected data’ (Feilzer, 2010, p. 14). Earlier in this chapter, the discussion of pragmatism as a full philosophical paradigm versus considering it just an approach suggested that it cannot be regarded as a full paradigm. The only requirement that pragmatism makes of researchers is a ‘duty’ to be curious, adaptable, and open to new data, experiences, and insights.

24

Martina Y. Feilzer

Additionally, the integration of different data methods at the stage of analysis speaks directly to the application of the third logic of abduction to mixed methods research but of course, abductive reasoning is also employed in the analyses of mono-method quantitative and qualitative research. Integration in mixed methods research is defined as ‘linking of qualitative and quantitative approaches and dimensions together to create a new whole or a more holistic understanding than achieved by either alone’ (Fetters & Molina-Azorin, 2017, p. 293). Considering pragmatism as a feature of mixed methods research in this context means that abductive reasoning can offer an analytical framework for transforming data and gaining new insights and explanatory value from the data, for allowing different data to speak to one another, to move between data sets, to analyse datasets abductively as well as deductively and inductively, separately at first, then moving back and forth between the data sets with the knowledge produced by each one. This approach underlies the key strengths identified earlier for mixed methods research (p.  21). In this way, methods and data are fully integrated, allowing for multi-dimensional perspectives to emerge, and the interpretation of each dataset is informed and enhanced by the other. It represents the separate analytical steps of ‘betweenmethod’ triangulation and truly offers the added value of using more than one method to address the research questions (Jick, 1979). Asking research questions in the context of complex systems with practical utility will often require complex research designs and in turn, much care and reflexivity throughout all research stages (Greene et al., 2001, p. 41), as well as a comprehensive set of research skills and confidence of using both quantitative and qualitative data either in individual researchers or within a research team. The use of reflexivity in research is a feature that could be argued to be core to pragmatic thought, and in particular Dewey’s thinking on reflective inquiry (Dewey, 1933). The notion of reflection as meaning-making, understanding the social world by gaining a greater understanding of relationships, connections between data, considering researchers and the people they interact with as connected, provides a coherent intellectual stance for pragmatic mixed methods research. Criticisms have been levelled against the concept of utility in pragmatist thought and its link to utilitarianism, as it might allow for a simplistic ‘what works’ pragmatism (Denzin, 2012, pp.  82–83), suggesting that pragmatists may support a methodological approach of ‘whatever gets results’. However, pragmatism is not an excuse for expedient or poorly designed research (Denscombe, 2008, p.  274) but needs to be transparent, and clearly pragmatic research needs to follow guidance on good-quality research including mixed methods research (Hammersley, 2008, p.  177). Quality criteria for mixed methods research have been the subject of extensive discussions over the past decades, and the most recent contribution on this issue is from Hirose and Cresswell (2023), who summarise the literature on mixed methods

A Pragmatist Approach to Mixed Methods Research

25

quality checklists and highlight six criteria for mixed methods researchers to adhere to. These criteria reflect standards for mixed methods research set by American Health and Psychology associations, in recognition that research quality standards need to speak to a number of stakeholders – at a minimum academic and research users – and be built on a widely accepted framework that resonates with current training practices (Hirose  & Cresswell, 2023, pp. 13–15). Hirose and Cresswell (2023, pp. 15–17) establish six core criteria – making explicit the reason for the appropriateness of mixed methods; write quantitative, qualitative, and mixed methods questions and aims; report quantitative and qualitative data separately; name and identify the type of mixed methods design; state the use of integration; and discuss meta-inferences and value resulting from the integration of the research findings. The value of quality criteria when narrowly conceived and seen as a constraint is contested and in Hirose and Cresswell’s (2023) version does not cover questions of the mixed methods research’s underpinning philosophical foundation. Criteria seem restricted mainly to good-quality presentation of mixed methods research rather than epistemological and ontological coherence and quality, and this is acknowledged in the discussion of this chapter (Hirose & Cresswell, 2023, p. 25). Pragmatist researchers need to be responsive to tensions between the questions asked and what emerges in the field and during data collection – the messiness of mixed methods research in terms of the empirical mess that can be encountered and its potential implications for the researcher’s paradigmatic stance (Sakata, 2022, p. 10). Research designs can be affected by changes in the fieldwork environment, institutional settings, and wider social and political field. Of course, this does not suggest that researchers change their original research questions but that any discussion of research findings need reflection and that the theoretical frames underpinning the research may need sharpening or rethinking abductively. Indeed, Sakata (2022, pp. 16–17) revised her entire research design, conceptual framework, and paradigmatic understanding of the research as a result of the research findings encountered in the field. However, this did not lead to an approach of ‘anything goes’ but rather an in-depth exploration as to how researchers negotiate and theorise their necessary adaptation to research in complex and dynamic social contexts. The discussion of what constitutes good mixed methods research needs to engage with the contention that mixed methods findings lack validity as conceived by both quantitative and qualitative mono-method proponents, although the approach to validity taken is much closer to those of qualitative researchers (see e.g. the discussion on validity by Maxwell, 1992). Here pragmatism differs also from dialectical pluralism as dialectical pluralists would follow the original paradigmatic conception of validity. Pragmatism based on its anti-representational view considers validity differently to other dominant

26

Martina Y. Feilzer

paradigms in the sense of the relationship between the problems or questions posed and the answers received, the ‘relationship between theory and method’, ensuring the research has delivered what the researcher wanted to know (Hanson, 2008, pp. 107–109). Validity is not considered as best ‘correspondence to reality’ (Rorty, 1991, p. 64) and is in clear contrast with the view of validity considered by quantitative positivist researchers. It further highlights the importance of reflective practice throughout the research process, a constant assessment of how the research data relates to the problems set and recognising that ‘all knowledge is knowledge from some point of view’ (Fishman, 1978, p. 531). Clearly, the overall question of how validity is defined also impacts on questions of generalisability and pragmatists’ views on the fallibility of research knowledge produced. Properly understood research knowledge is contextual and time-bound, and as such, all findings need to be situated in their spatial, temporal, and social context. Rather than regarding this as a limitation, such contextualisation allows for a better understanding of the processes and structures involved in creating certain findings, and it increases the meaningfulness of any research (Erlandson et al., 1993, pp. 14–19; Tashakkori & Teddlie, 1998, p. 10). It also provides research knowledge that lends itself to actions; in other words, if the research addressed specific social problems, recommendations and solutions can be implemented as the research is conducted within a specific institutional and social context and geographic and cultural setting. 2.4

Where Next for Pragmatist Approaches and Mixed Methods Research?

The previous section has highlighted how pragmatism and mixed methods research have affinity and how pragmatism can provide a weak to moderate foundation for mixed methods research by offering a justification and a good reason for mixing qualitative and quantitative methods/data and design (Shan, 2022, pp. 6–7) in order to better understand complex social phenomena in their appropriate and dynamic contexts and settings. A number of key elements of pragmatist thought such as anti-dualism, anti-representational forms of validity, and a commitment to the limitations of knowledge provide a clear link between pragmatism as a philosophical approach to research and mixed methods research designs. For many researchers, such a foundation is sufficient; however, some of course are looking for a single paradigm to underpin mixed methods research. To provide for such a strong paradigmatic foundation and offering a ‘normative thesis that mixed methods should be encouraged’ (Shan, 2022, p.  8) through a pragmatist paradigm, two additional steps would need to be taken. First, pragmatism as a philosophical framework for research needs to be further refined and develop

A Pragmatist Approach to Mixed Methods Research

27

an improved internal coherence, so it can live up to Morgan’s desire for pragmatism to produce a ‘properly integrated methodology for the social sciences’ (Morgan, 2007, p. 73). Second, the internal tension in pragmatism between research questions as the main drivers of methodological choices and, as some suggest, a requirement for a paradigm to require the use of a specific set of mixed methods research designs needs to be acknowledged and addressed. This still leaves the question of whether such a strong foundation is desirable and necessary, and a consensus among researchers on this has yet to be reached. Reflecting on decades of research experience and the use of mono-method and mixed method designs and research, it is important for research paradigms to offer support to researchers aiming to make sense of the complexity and messiness of social life through research. Imposing requirements for specific research designs or methods to fulfil philosophical musings does not necessarily make for good-quality research and can lead to ‘misleading findings’ (Greenhalgh et al., 2022, p. 257). From this perspective, reflections on the paradigm wars, and recent advances in a number of disciplines such as health, mixed methods researchers should consider carefully the need for a single paradigmatic foundation. Pragmatism, offers the advice of asking what methodological approach is required to answer a specific set of research questions, while acknowledging the value of both quantitative and qualitative research methods and of bringing them together to produce knowledge that is well suited to furthering our understanding of complex contemporary social phenomena. In this spirit of having a good reason to conduct mixed methods research, pragmatism enables researchers to enjoy the complexity and messiness of researching social life and reviving a flagging sociological imagination without artificially constraining researchers’ methodological choices. References Brewer, J., & Hunter, A. (1989). Multimethod Research: A Synthesis of Styles. Newbury Park, CA: SAGE. Bryman, A. (2007). Barriers to integrating quantitative and qualitative research. Journal of Mixed Methods Research, 1, 8–22. Creswell, J. W., & Plano Clark, V. L. (2007). Designing and Conducting Mixed Methods Research. Thousand Oaks, CA: SAGE. Denscombe, M. (2008). Communities of practice. Journal of Mixed Methods Research, 2, 270–283. Denzin, N. K. (2012). Triangulation 2.0. Journal of Mixed Methods Research, 6(2), 80–88. Dewey, J. (1925). Experience and Nature. Whitefish, MT: Kessinger. Dewey, J. (1933). How We Think: A Restatement of the Relation of Reflective Thinking to the Educative Process. Boston, MA: D.C. Heath & Co Publishers. Dickstein, M. (1998). Introduction: Pragmatism then and now. In M. Dickstein (Ed.), The Revival of Pragmatism, (pp. 1–18). Durham and London: Duke University Press.

28

Martina Y. Feilzer

Doyle, L., Brady, A-M., & Byrne, G. (2009). An overview of mixed methods research. Journal of Research in Nursing, 14(2), 175–185. Erlandson, D. A., Harris, E. L., Skipper, B. L., & Allen, S. D. (1993). Doing naturalistic inquiry: A guide to methods. Newbury Park, CA: SAGE. Feilzer, M. Y. (2010). Doing mixed methods research pragmatically – implications for the rediscovery of pragmatism as a research paradigm. Journal of Mixed Methods Research, 4(1), 6–16. Fetters, M. D., & Molina-Azorin, J. F. (2017). The Journal of Mixed Methods Research starts a new decade: The mixed methods research integration trilogy and its dimensions. Journal of Mixed Methods Research, 11(3), 291–307. Greene, J., Benjamin, L.,  & Goodyear, L. (2001). The merits of mixing methods in evaluation. Evaluation, 7, 25–44. Greenhalgh, T., Fisman, D., Cane, D. J. et al. (2022). Adapt or die: How the pandemic made the shift from EBM to EBM+ more urgent. BMJ Evidence-Based Medicine, 27, 253–260. Hall, R. F. (2013). Mixed methods: In search of a paradigm. In T. Le & Q. Le (Eds.), Conducting Research in a Changing and Challenging World. New York: Nova Science Publishers. Hanson, B. (2008). Wither qualitative/quantitative? Grounds for methodological convergence. Quality & Quantity, 42, 97–111. Hirose, M., & Cresswell, J. W. (2023). Applying core quality criteria of mixed methods research to an empirical study. Journal of Mixed Methods Research, 17(1), 12–28. Howe, K. R. (1988). Against the quantitative-qualitative incompatibility thesis or dogmas die hard. Educational Researcher, 17, 10–16. Hughes, J. A., & Sharrock, W. W. (2007). Theory and Methods in Sociology. Basingstoke, UK: Palgrave Macmillan. James, W. (1906/1907). Pragmatism – a new name for some old ways of thinking. Project Gutenberg eBook of Pragmatism. www.gutenberg.org/files/5116/5116-h/5116h.htm#link2H_4_0004. Jick, T. D. (1979). Mixing qualitative and quantitative methods: Triangulation in action. Administrative Science Quarterly, 24, 602–611. Johnson, R. B. (2017). Dialectical Pluralism: A metaparadigm whose time has come. Journal of Mixed Methods Research, 11(2), 156–173. Johnson, R. B.,  & Onwuegbuzie, A. J. (2004). Mixed methods research: A  research paradigm whose time has come. Educational Researcher, 33(7), 14–26. Kuhn, T. S. (1962). The Structure of Scientific Revolutions (1st ed.). Chicago: University of Chicago Press. Maxwell, J. A. (1992). Understanding and validity in qualitative research. Harvard Educational Review, 62, 279–300. Mertens, D. M. (2007). Transformative paradigm: Mixed methods and social justice. Journal of Mixed Methods Research, 1(3), 212–225. Miller, S. (2006). Mixed methods as methodological innovations: Problems and prospects. Methodological Innovations Online, 1, 1–7. Morales, A. (2003). Introduction. In A. Morales (Ed.), Renascent Pragmatism. London: Routledge. Morgan, D. L. (2007). Paradigms lost and pragmatism regained. Journal of Mixed Methods Research, 1, 48–76. Mounce, H. O. (1997). The Two Pragmatisms: From Peirce to Rorty. London: Routledge.

A Pragmatist Approach to Mixed Methods Research

29

Muddiman, E., Lyttleton-Smith, J.,  & Moles, K. (2018). Pushing back the margins: Power, identity and marginalia in survey research with young people. International Journal of Social Research Methodology, 22(3), 293–308. Nisbet, R. (1993). The Sociological Tradition. New York: Taylor and Francis. Paavola, S. (2015). Deweyan approaches to abduction. In U. Zackariasson (Ed.), Action, Belief and Inquiry – Pragmatist Perspectives on Science, Society and Religion (pp. 230–249). Nordic Studies in Pragmatism 3. Helsinki: Nordic Pragmatism Network. Peirce, C. S. (1992–1998). The Essential Peirce: Selected Philosophical Writings (The Peirce Edition Project, Ed., vol. 2, 2 vols). Bloomington and Indianapolis: Indiana University Press. Robson, C. (1993). Real World Research. Oxford, UK: Blackwell. Rorty, R. (1991). Objectivity, Relativism and Truth: Philosophical Papers (Series – Philosophical Papers, Vol. 1). Cambridge, UK: Cambridge University Press. Rorty, R. (1999). Philosophy and Social Hope. London: Penguin Books. Sakata, N. (2022). Embracing the messiness in mixed methods research: The craft attitude. Journal of Mixed Methods Research, online first. https://journals.sagepub. com/doi/epub/10.1177/15586898221108545. Shan, Y. (2022). Philosophical foundations of mixed methods research. Philosophy Compass, 17(1), 1–12. Shannon-Baker, P. (2016). Making Paradigms Meaningful in Mixed Methods Research, 10(4), 319–334. Skivington, K. et al. (2021). A new framework for developing and evaluating complex intervention: Update of medical research council guidance. BMJ, 374, 1–8. Suckiel, E. K. (1998). Book review: The two pragmatisms: From Peirce to Rorty. Transactions of the Charles S. Peirce Society, 304–312. Tashakkori, A., & Teddlie, C. (1998). Mixed Methodology: Combining Qualitative and Quantitative Approaches. Thousand Oaks, CA: Sage. Teddlie, C., & Tashakkori, A. (2009). Foundations of Mixed Methods Research. Thousand Oaks, CA: Sage.

3 THE PHILOSOPHICAL FOUNDATIONS OF A TRANSFORMATIVE APPROACH TO MIXED METHODS Donna Mertens

3.1

Introduction

The transformative paradigm is one of the major philosophical frameworks that is strongly associated with mixed methods research (Mertens, 2023, 2018a, 2020; Molina-Azorin  & Fetters, 2019, 2020; Shannon-Baker, 2016; Tashakkori et al., 2021). The transformative paradigm emerged because members of marginalised and vulnerable groups voiced their concerns that their communities were either unrepresented or misrepresented in research, resulting in policies and programmes that either harmed them or left them no better off (Cram et al., 2013; Chilisa & Mertens, 2021; Harris et al., 2009; Mertens, 2020). Mertens (1999) first introduced the concept of the transformative lens for research and evaluation in her presidential address to the American Evaluation Association in 1998. The international community has heightened awareness of the need for research that supports transformative change as a strategy for attaining the Sustainable Development Goals, because they see the need to incorporate understanding of the complexities in communities that have been historically left behind (Stephens et al., 2018). Many scholars in specific disciplines have recognised the importance of using a transformative framework, including Poradzisz and Florczak (2018) in the field of nursing, Marra (2015) in the field of policy analysis, Cook (2014) in the field of community psychology, Farias et al. (2017) in the health sciences, Habashi and Worley (2009) in international peace studies, Bolinson and Mertens (2019, 2021) in impact investing, and Sullivan et  al. (2014) when working with the disability community. The transformative framework prioritises the human rights that are recognised in the United Nations’ conventions and declarations that call for respect DOI: 10.4324/9781003273288-4

The Philosophical Foundations of a Transformative Approach

31

for the rights of women, people with disabilities, Indigenous communities, and refugees and immigrants. With the transformative framework, research begins with a critical examination of conditions that sustain an oppressive status quo and works to provide support for the changes that are needed to enhance human rights. Marra (2015) argues that transformative mixed methods research needs to take into account the complexity of systems in terms of power relationships and how dominant groups may ‘benefit from denying others access to material and social resources, such as adequate child care, paid work opportunities, or political representation’ (Marra, 2015, p.  32). Thus, transformative mixed methods researchers include a critical look at strategies that can shift ‘the balance of power such as access to mental health services, stable housing, provision of a living wage, affordable child care, and universal health care’ (Mertens, 2018b, p. 2). In this chapter, the structure of paradigms in social research is briefly presented. This is followed by an explanation of the transformative paradigm’s philosophical and theoretical roots and an in-depth description of the assumptions that constitute the transformative paradigm. Implications for methodology are integrated into the sections describing the assumptions, along with a discussion of the permeability of borders across paradigms. The final section provides reflections on the philosophical assumptions of the transformative paradigm and implications for mixed methods researchers who situate themselves in that paradigm. 3.2

The Structure of Paradigms

The philosophical foundations for the transformative approach to research builds on the characterisation of research paradigms developed by Guba and Lincoln (1994, 2005). Guba and Lincoln based their discussion of paradigms in the social sciences on Kuhn’s (1970) earlier work that focused mainly on the role of paradigms in research in the natural sciences. Kuhn argued that a new paradigm emerged and replaced older paradigms when discrepancies arose in the natural sciences that could not be fully explained by the paradigm in use at the time (e.g. Newtonian mechanics vs. relativity and quantum theories). Guba and Lincoln put forth the position that in the social sciences one paradigm did not replace the other; rather, the social sciences community operated with several different paradigms, because their methodological decisions were based on a set of philosophical assumptions that guided methodological choices. Thus, Guba and Lincoln helped the research community get past the argument of which is better, quantitative or qualitative methods, and opened up possibilities for the integration of quantitative and qualitative methods, now known as mixed methods (Tashakkori et al., 2021). Guba and Lincoln identified four types of philosophical assumptions that defined research paradigms: axiology (the nature of ethics and values), ontology (the

32

Donna Mertens

nature of reality), epistemology (the nature of knowledge and the relationship between the researcher and stakeholders/participants), and methodology (the nature of systematic inquiry). 3.3

Transformative Philosophical and Theoretical Roots

The roots of the transformative paradigm are quite eclectic and include theories that focus on inequities on the basis of identity characteristics (e.g. feminist theory; critical race theory), as well as thought leaders in the activist and philosophical communities (Mertens, 2020). For example, the transformative paradigm is commensurate with the teachings of the social activist and scholar Paulo Freire and his ‘dialogical conscientization’ model in Brazil (Freire, 1970, 2006). Freire’s concept of conscientisation has relevance, because it consists of an iterative process of critically reflecting on social, political, and economic realities and taking action to challenge oppressive structures to create a just society. This critical reflection occurs through dialogues between participants who are considered to be equal, each bringing valuable knowledge and experiences that can be used to challenge dominant social myths that are oppressive. The outcome of dialogical conscientisation is increased awareness of social realities that have historically been used to oppress people and provision of a basis for action that can lead to a more just world. Other relevant philosophical basis come from Habermas’s communicative action theory and Foucault on the academic rhetoric supportive of institutional forms of domination and control (Christians, 2005). The Theory of Communicative Action (Habermas, 1984) holds that humans are rational beings and that language is the form in which rationality is expressed. Greater understanding can be reached through noncoercive dialog that encompasses subjective, objective, and social experience. Similar to Freire, Habermas held that greater understandings should lead to communicative action when the process of communication is structured to be between equals. This type of action provides the basis for societies to be more egalitarian and inclusive. His work has relevance as a basis for the transformative paradigm through his argument that many truths can be claimed, but it is through interactive communications as equals that we reach greater understanding (Howell, 2013). Understanding is not the final goal; the final goal is to move towards a more just society. Foucault (1981, 2002, 2003) situated his philosophical writings as a critique of social science research that, under post-positivism, had held that a reality exists that we discover through use of the scientific method. In contrast, he set the stage for both constructivist and transformative assumptions about the nature of reality, in that he argued that reality is constructed through the lens of social discourse. This assumption that reality is socially constructed reflects

The Philosophical Foundations of a Transformative Approach

33

a constructivist paradigm. The assumption appears in a slightly altered state in the transformative paradigm’s ontological assumption. The transformative paradigm pushes the concept of reality to challenge those that support the continuance of an oppressive status quo and highlight those versions of reality that lead to increased justice. Foucault also recognised the political nature of research and the importance of considering issues of power inequality and social structures (Howell, 2013). He challenged existing structures of authority and interrogated their role in continuing an oppressive status quo. This stance is commensurate with the transformative epistemological assumption. His emphasis on understanding historical and contextual factors is adopted in the transformative paradigm’s methodological assumptions. When phenomenon are understood through a historical lens, it can lead researchers to see that present-day assumptions about past phenomenon may have been misunderstood because of versions of reality imposed by the powerful of the time. He emphasised the importance of illuminating ‘plurality and the extent power has influenced and or determined our understanding of truth’ (Howell, 2013, p. 174). Foucault’s study of power is particularly relevant for the transformative paradigm. He saw power as involving social relationships that was not limited to the oppressor having power over the oppressed. Rather power permeates all aspects of everyday life in personal and institutional structures. ‘Power relations are concealed within society and to understand these we must excavate and uncover these concealed relationships that exist within the economic system, governmental structures, and everyday existence’ (Howell, 2013, p. 176). The issue of power is critically important in the transformative paradigm’s epistemological assumption in that questions are raised about the relationship between researchers and those who are engaged with them or impacted by the research (stakeholders). It also raises questions about whose knowledge is given privilege and the historical origins of that knowledge. The specific nature of the transformative paradigm’s assumptions is addressed in the next section. 3.4

Transformative Axiological Assumption

All researchers, no matter what paradigm they use to frame their studies, need to pay attention to ethical issues. The traditional approach to ensuring ethical research is embodied in institutional ethical review boards (Mertens, 2020). Transformative researchers also must obtain approval from such review bodies. However, the transformative axiological assumption reflects values and ethics that direct researchers to design their studies with a conscious attempt to contribute to social, economic, and environmental justice (Mertens, 2020; Mertens & Wilson, 2019). To some, this is a huge and lofty goal that is considered to be unachievable by some researchers. However, members

34

Donna Mertens

of marginalised communities contend that researchers who avoid addressing these three types of justice risk being complicit in the continuation of an oppressive status quo (Neubauer et al., 2020). In order to pursue this increase in justice, the transformative axiological assumption includes other elements: the need to address structural and systemic factors that perpetuate discrimination; working in a culturally responsive manner; the promotion of sustainable actions needed for transformative change; providing for reciprocity; and recognising the resilience in communities and the interconnectedness of all human and natural things. Because of the complexity of these values, researchers who want to design their studies in ways that uphold these values tend to use mixed methods designs in order to engage with the contextual and cultural factors implicit in this approach. Based on the transformative axiological assumption, researchers consciously explore how social, economic, and environmental justices are connected. This exploration is based on the belief that an interconnection exists between all human and natural living things and objects. If an intervention is introduced to improve economic development, but it has a negative impact on the environment or on the living conditions of the humans, animals, and plants, then data on these potential impacts needs to be considered before interventions are implemented. Collection of quantitative and qualitative data provides an opportunity to create understandings of these different types of justice. Indigenous communities have contributed greatly to the understanding of the interconnectedness through their understanding of the spiritual nature of the earth, the sky, the waters, and those who inhabit this planet (see Chilisa this book). Addressing structural and systemic factors that perpetuate discrimination is a fundamental part of the assumption of what makes for an ethical study within the transformative paradigm. This is not an assumption that is explicitly articulated under other paradigms that are used in the mixed methods community. Yet barriers to transformative change in the form of increased equity and justice are commonly rooted in oppressive cultural beliefs and historical structures that perpetuate oppression. Thus the transformative axiological assumption leads researchers to ask questions about the nature of societal structures such as those raised by Foucault (2002) that sustain oppression, such as governmental and economic structures. Cultural responsiveness is a value that is also integral to the conduct of an ethical transformative mixed methods study (Chouinard & Cram, 2020). The concept of cultural responsiveness was articulated by a team of researchers of colour including, but not limited to, Stafford Hood (2005, 2015). Their stance, which elucidates the need for cultural responsiveness as a determinant of an ethical study, can be summarised in one of their core principles: ‘without the nuanced considerations of cultural context in evaluations conducted within diverse ethnic, linguistic, economic and racial communities of colour there

The Philosophical Foundations of a Transformative Approach

35

can be no good evaluation’ (p. ix). Their work on being culturally responsive focused on work in the evaluation of programmes in communities of colour. However, this concept of cultural responsiveness has been broadened to include many different marginalised and vulnerable communities, such as women, people with disabilities, sexual minorities, and indigenous communities (Cram & Hopson, 2018; Mertens & Boland, 2018; Miller, 2018). The transformative axiological assumption also has an action-orientation. If research is done to ‘once again’ tell communities how bad-off they are or to merely ‘give voice’ to marginalised communities, then it does not reflect the ethical principles of the transformative paradigm. As was mentioned at the beginning of this chapter, communities are tired of having researchers come into their community and conduct studies that leave them no better off or worse off than before. Hence, researchers who situate themselves in this paradigm are likely to structure the studies to include the collection of both quantitative and qualitative data that can be used as a basis for action that leads to positive transformative change. In this respect, this assumption is commensurate with some action-research approaches. However, all action research does not use a transformative lens; some of it focuses only on the needs of a small proportion of the people who are affected by the research without consideration for inclusion of the full range of stakeholders in culturally responsive ways. This principle ties in with the ideas of reciprocity and recognition of community resilience. Reciprocity means that communities do receive valuable outcomes as a result of participating in the research; this can come in the form of increased capacity to conduct research themselves or the ability to use the results of the research to advocate for their own priorities. Communities are viewed as having strengths that result from their lived experiences; they have knowledge and experiences that a researcher from the outside does not have. 3.5

Applying the Transformative Axiological Assumption

The following example provides insights into how mixed methods researchers can act upon these ethical principles and operationalise their values. The area of West Java in Indonesia is a high-poverty area that has been targeted by government and international donor agencies for economic development (Widianingsih & Mertens, 2019). Researchers at the Padjadjaran University in West Java used a transformative mixed methods approach to study the effects of the economic development policies to reveal the contextual factors that led to disasters based on the projects that were implemented, and to support changes in policies that would lead to increased justice. Two of the strategies they used – building inclusive, culturally responsive relationships and conducting a contextual analysis – reflect the application of the transformative axiological assumption.

36

Donna Mertens

The Padjadjaran University in West Java, Indonesia, where Widianingsih is a faculty member, began by building relationships across the full spectrum of stakeholders (Widianingsih & Mertens, 2019). They were aware that different strategies would be needed to have culturally responsive relationships with the different stakeholder groups. For example, the researchers met with farmers, their wives, and young people in their local communities. The farmer contingent formed a coalition where they could meet and systematically create data to encourage government to respond to their needs in ways that they perceived as having value. The data collection from the coalition was largely qualitative and was collected by means of focus groups and town meetings. However, this was supplemented with quantitative data on agricultural production and revenues. For government representatives, the researchers provided programmes to build their capacity to use data in their decision-making and policy development. The government created a process for community members to bring their concerns to the appropriate department’s attention. Mixed methods were used to collect data via surveys, interviews, and document reviews to determine the efficacy of the university programmes and the government processes. For students at the university, the curriculum was changed, so students took a service course where they went into the poor communities to engage in participatory data collection to determine the needs of farmers and other local groups. Data from the students were both qualitative and quantitative and were reported via presentations at the university. The contextual analysis was conducted in ways that reflected the transformative axiological assumption, because it focused on historical data that led to the current conditions of inequity, along with limited economic opportunities and environmental destruction (Widianingsih  & Mertens, 2019). The government encouraged the production of palm oil and supported policies that led to large corporate buy-outs of small farmers’ lands. These policies led to an increase in social, economic, and environmental injustice, because the internally displaced people moved from their rural areas and found it difficult to find work in the cities. Some turned to illegal activities, and some were trafficked to other countries where they were subjected to abuse. Extant quantitative data supported the degree of displacement and the lack of jobs; interview data supported the negative impacts of human trafficking. The contextual analysis further reflects the transformative axiological assumption by critically examining the solutions implemented by the government for job creation (Widianingsih & Mertens, 2019). One of the government’s solutions to job creation was to build a textile factory that polluted the Citarum River; this is the river that the people used for drinking and bathing. Mixed methods data from quantitative analysis of ecological and health data

The Philosophical Foundations of a Transformative Approach

37

supported the ecological disaster and its impact on human rights. The Citarum River is extremely polluted with levels of lead that are 1,000 times worse than the U.S. standard for drinking water. Yet, 25 million people depend on it for drinking water, irrigation of crops, and energy production. The result is that many people who use this heavily polluted water and breath the contaminated air now suffer from health problems such as scabies, infections, and respiratory distress. (Tarahita & Rakhmat, 2018, cited in Widianingsih & Mertens, 2019, p. 31) Thus, this example provides insights into the meaning of the transformative axiological assumption because of its conscious attention to issues of environmental, social, and economic justice, and by the employment of strategies to build culturally responsive relationships that recognise the interconnectedness of humans and natural resources. 3.6

Transformative Ontological Assumption

The transformative ontological assumption holds that there are different versions of reality, some that sustain an oppressive status quo and others that promote justice (Mertens, 2018a, 2020, 2023a; Mertens & Wilson, 2019). This contrasts with the ontological assumptions of the post-positivist, constructivist, pragmatic, and Indigenous paradigms. In post-positivism, the ontological assumption asserts that there is one reality that can be measured with a certain degree of error. It does not raise questions about whose version of reality is given privilege nor to the consequences of accepting one version of reality over another. The constructivist ontological assumption holds that there are multiple socially constructed realities; however, this assumption does not push this concept to recognise that realities are constructed by persons with different social positionalities and that a version of reality constructed by the most powerful will most likely not be the same version of reality created by members of marginalised communities. The version of reality constructed by the most powerful has been used to oppress members of marginalised communities throughout history; hence, it is the transformative researcher’s ethical responsibility to critically interrogate the versions of reality that emerge in a study. Pragmatism has a very different assumption about reality: what is considered to be real comes from common agreement about what is found to be real; this can be achieved through the use of the scientific method (Grayling, 2019). The key criterion is whether or not the version of reality is viewed to be useful and whether or not there is convergence based on evidence.

38

Donna Mertens

Transformative researchers would interrogate the possibility of having a version of reality that emanated from a select group without consideration of representation of marginalised voices and mechanisms for insuring that their perspectives were included. The Indigenous ontological assumption aligns somewhat with the constructivist and transformative assumptions; however, it extends the idea of multiple socially constructed realities to include the grounding of those realities in ‘material, social, and spiritual context and marked by the interconnectedness of the living and the nonliving and relational existence’ (Chilisa  & Mertens, 2021, p.  246). The uniqueness of the Indigenous ontological assumption is the inclusion of spirituality, interrelatedness, and relational existence. The transformative ontological assumption is enhanced by consideration of these Indigenous concepts. In the transformative ontological assumption, versions of reality are seen to emanate from different social positionalities; some versions of reality are accorded greater privilege because of extant power relations. For example, the USA was founded with the version of reality that White Europeans and their descendants were worth more than people of colour and Indigenous people, and that the colonisers had a right to take the land from the Indigenous people and to enslave people of colour (Wilkerson, 2020). The consequence of accepting this version of reality was centuries of oppression of people of colour and Indigenous communities; it continues to have repercussions today. If this version of reality was allowed to continue unchallenged, there would be no road to increased justice. Transformative mixed methods researchers hold that they have a responsibility to make visible the different versions of reality that are operating in a particular context, along with the origins of those versions of reality. They also need to interrogate those versions of reality that sustain oppression and those that provide a pathway towards increased social, economic, and environmental justice. 3.7

Application of the Transformative Ontological Assumption

The work of Miller et al. (2021; Miller, 2020) provides an example of how dominant versions of reality sustain oppression against members of sexual minority communities. In some countries, laws and policies state that engaging in same-sex behaviours is a crime, punishable by jailtime or even death. This version of reality comes from a heterosexual majority who use religion to justify their position. Miller and colleagues set out to understand how this version of reality was manifest in these countries and to challenge it by examining how to provide safe health services for gay and bisexual men, and transgender women. They described their study as: the project was a human rights advocacy demonstration project to dismantle structural and social barriers to HIV care for gay and bisexual men and transgender

The Philosophical Foundations of a Transformative Approach

39

women . . . in five countries in Africa (Burundi, Cameroon, Côte d’Ivoire, Ghana, Zimbabwe) and two Caribbean countries (Dominican Republic and Jamaica). (Miller et al., 2021, p. 3) Miller and colleagues (2021) used both quantitative and qualitative methods to reveal the oppressive versions of reality and their consequences. They conducted a document review of legal provisions in each country, along with a review of policies and practices related to the provision of health care. The results of a survey and a historical analysis confirmed that gay and bisexual men experienced alienation from their families, homelessness, and denigration by others. Their study focused on making visible a version of reality in which gay and bisexual men and transgender women could receive appropriate healthcare in a respectful manner. More detail about their methods is discussed in the next section on the application of the transformative epistemological assumption. 3.8

Transformative Epistemological Assumption

The transformative epistemological assumption places value on the knowledge of community members and asks the researcher to form relationships with stakeholders to explicitly address power differentials (Mertens, 2018a, 2020; Mertens & Wilson, 2019). The transformative epistemological assumption aligns with the transformative axiological assumption by holding that relationships need to be formed across the full spectrum of stakeholders in culturally responsive manners in order to provide a safe and supportive environment during the research. This contrasts with the post-positivist assumption that prioritises ‘objective’, distant relationships between researchers and participants to prevent bias. The transformative epistemological assumption holds that researchers need to cultivate relationships of trust in order to prevent their own cultural biases from influencing the research. Building these relationships allows for researchers to be challenged about their own assumptions and gives them the opportunity to understand the context, problem, and appropriateness of interventions from the community’s perspective. In line with the transformative ontological assumption, the relationships between researcher and stakeholders need to be structured in ways that permit the knowledge from the community to be shared and valued. In order for this to happen, the relationships need to acknowledge existing power inequities and address those by inclusion of stakeholders, including those from marginalised and vulnerable groups, in ways that inform the process and use of the research. This view of epistemology is necessary in order to create the conditions where versions of reality can be made visible that lead to increased justice. Traditional power relationships need to be challenged and structural

40

Donna Mertens

changes introduced that provide power to communities. Community-based knowledge is valued as a basis for understanding history, context, problems to be addressed, strategies for engagement with stakeholders, and development and implementation of interventions. Community members also contribute to the understanding of viable research methods, data collection instruments, and analysis, interpretation, and use of data. 3.9

Application of the Transformative Epistemological Assumption

Miller et al.’s (2021) work with members of sexual minority communities illustrates the transformative epistemological assumption. The title of the project, the Advocacy and other Community Tactics Project (ACT), indicates that the researchers situated their position as one in which they would explicitly advocate for community involvement at all stages of the study. An external researcher (Miller) worked with a collaborative that included a lead activist agency and nine collaborating identity groups. The collaborative provided a structure for relationships to develop that were based on valuing the voices of members of the sexual minority community throughout the study. The researcher and the collaborative were aware of the need to provide safe and supportive places for the community representatives to meet. Decisions about data collection were made based on discussions with the community. Community members wanted to use a mystery shopper approach, that is, they would visit health clinics as patients and collect data about their experiences. The mystery shoppers were chosen by their identity groups to receive training in how to collect the data. The community members wanted to use a quantitative checklist to record their experiences. The identity groups developed a standardised checklist to ensure that it included indicators of importance to their community. As the data collection process got underway, the researcher did not assume that the quantitative data would capture the full knowledge of the community members who served as mystery shoppers (Miller et al., 2021). Rather, qualitative data were collected from the mystery shoppers when debriefing them on their experiences. The data collectors indicated that they needed additional training in order to be able to more effectively deal with the hostility they encountered at the health clinics. The mystery shoppers’ knowledge of the situation and its effects on them as data collectors were given credence and acted upon by the project. The advocacy organisation provided additional training for the data collectors on how to respond when they encountered hostile and discriminatory behaviours. They also provided support meetings for the data collectors so they could deal with the stress involved in the work they were doing. The advocacy organisation’s response reveals both the value placed on the community members’ knowledge and the respectful relationships that had developed.

The Philosophical Foundations of a Transformative Approach

3.10

41

Transformative Methodological Assumption

The first three assumptions (axiology, ontology, and epistemology) in the transformative paradigm have implications for the transformative methodological assumption. The methodological assumption derives from these three previous assumptions and calls for the use of mixed methods in order to be responsive to the multiple constituencies who are involved in and affected by the research. Many different designs can be used in a transformative mixed methods study; the crucial factor is using a transformative lens throughout the process of the study. Inclusion of the full range of stakeholders is important and this needs to be done in ways that are culturally responsive and that challenge existing power structures. The design needs to consciously address issues of inequity and discrimination in order to provide a pathway towards increased justice. Transformative mixed methods have an action orientation by virtue of their commitment to supporting transformative change. The transformative methodological assumption also recognises that mixed methods can be used to conduct a contextual analysis to reveal cultural, historical, political, and economic conditions that influence the possibility of transformative change. Data collected through mixed methods provides the opportunity to better understand the nature of the problem through a critical lens and to contribute to the development and determination of the effectiveness of culturally responsive interventions. The transformative axiological assumption supports the need for mixed methods designs to identify who needs to be involved, how best to involve them in culturally respectful ways, and how to facilitate the use of data to sustain transformative change. Mixed methods can also be used to evaluate the quality of the relationships formed during the research study. The methodological implications from the transformative ontological assumption are that data needs to be collected that reveals the different versions of reality and how these either sustain oppression or increase justice. The use of mixed methods to accomplish this purpose is recommended in order to capture the differences in perspectives and lived experiences of the full range of people impacted by the research. The methodological implications of the transformative epistemological assumption are that community members need to be included in ways that value their knowledge and provides a safe space for sharing their experiences. Mixed methods contribute to this goal by developing data collection that is responsive to the cultural diversity in the population.

3.11

Application of the Transformative Methodological Assumption

The two examples used to illustrate the application of the axiological, ontological, and epistemological assumptions of the transformative paradigm

42

Donna Mertens

have already partially revealed methodological implications for this paradigm. Both the West Java example (Widianingsih & Mertens, 2019) and the African/Caribbean example (Miller et al., 2021) highlighted the importance of directly addressing issues of justice and discrimination, building culturally responsive and equitable relationships with the full range of stakeholders, and valuing community-based knowledge. A generic design of a transformative mixed methods study provides further insights into the application of the methodological assumptions for this paradigm (see Figure 3.1). All transformative mixed methods studies do not follow this specific design; however, it does provide insights into operationalising the transformative methodological assumption throughout the research process. As can be seen in Figure 3.1, the transformative methodological assumptions lead to a dynamic iterative cycle for the mixed methods design in order to incorporate the implications of the axiological, ontological, and epistemological assumptions. A  research study generally starts with some felt need; however, a transformative researcher would not accept the articulation of the need uncritically. In order to gain a better understanding of what is needed, a period of time needs to be devoted to forming relationships with the full range of stakeholders with a conscious awareness of challenging traditional power structures. The transformative mixed methods cyclical approach is not linear;

FIGURE 3.1

Transformative Mixed Methods Cyclical Approach

Source: Adapted from Mertens, 2023b in Tierney, Rizvi, Ercikan and Smith (Eds.)

The Philosophical Foundations of a Transformative Approach

43

data from each phase informs the next phase. For example, forming relationships occurs as a starting point, but it is a process that continues throughout the course of the study. Data from these relationships can be used to critically assess the methodologies used to create research questions, develop data collection instruments, or determine next steps in intervention. Methodologically, the transformative paradigm is very eclectic and can incorporate many of the research designs found in other paradigms. However, the development and implementation of the designs would be framed through the transformative lens. For example, community-based participatory research (CBPR) is not a new approach to research; however, transformative researchers who use this approach would start with the assumptions commensurate with this paradigm. They would ask questions about the full range of stakeholders and how members of traditionally less powerful communities could be appropriately identified and included. One of the strategies that has been employed in transformative CBPR mixed methods studies is to include community members as co-researchers (DeJonckheere et al., 2019). In order to align with the assumptions of the transformative paradigm, CBPR needs to include community members in an equitable manner provide for capacity building as necessary and develop coalitions that can inform the research process itself and sustain efforts when the study ends. Based on DeJonckheere et al.’s (2019) literature review of CBPR and mixed methods, they concluded that the use of mixed methods in this type of study increased participation and collaboration and resulted in the development of interventions that were more responsive to community needs. An example from the education sector is now presented to illustrate the application of these assumptions. Educational inequities exist throughout the world; they manifest in different ways in different countries and cultures. In the USA, school disparities in the form of achievement, school engagement, and access to supportive services are well documented on the basis of race/ ethnicity, ability, gender, sexual identity, and language. A group of universitybased researchers in the Northeastern part of the USA was contacted by a school system in a nearby state that recognised its ‘zero tolerance’ disciplinary system was resulting in the exclusion of a disproportionate number of students of colour (Garnett et al., 2019). A zero tolerance policy means that the school does not tolerate any breach of the rules; students who do so are suspended from school. Rather than simply accepting the description of the problem that was put forth by the school, the researchers decided to take a transformative approach based on their thinking that school reform has historically fallen short of its objectives, and a different approach would be warranted. The discipline policy may well have contributed to the disproportionate exclusion of students of colour but Garnett and colleagues wanted to gain a more complete understanding of the problem. They chose a transformative mixed methods approach and justified their choice because they

44

Donna Mertens

wanted to be able to support transformative change in the school climate, power hierarchies, and an ‘entrenched white supremacy in exclusionary discipline’ (p. 3). Thus, the researchers started by being explicit about the need to address systemic discrimination as a conscious part of their design in keeping with the transformative axiological assumption. The transformative methodological assumption prioritises forming culturally responsive relationships that challenge traditional power hierarchies. Relationships were formed on multiple levels in the Garnett et al. (2019) study. As expected, the university researchers formed a multi-year partnership with the school system. However, the university researchers included provisions in the partnership that specified that a transformative mixed methods action research project would be central to their work and that a Youth Participatory Action Research strategy would be used. The more challenging relationships that were built were between the researchers and the students. Traditionally, researchers control decisions about methodology and other aspects of the studies they conduct. However, these researchers consciously sought ways to address the power differences so that they could build trusting relationships with the youth with a goal to ensure that the ‘research topic, question and identification of the problem/asset is truly youth driven’ (Winn & Winn, 2016, cited in Garnett et al., 2019, p. 8). Garnett and colleagues (2019) recognised that forming the relationships was an on-going process that required critical reflection throughout the study. They had tried to live up to the goal of having a youth-driven study; however, they found that their strategies had fallen short. During one session when students were discussing the results of an early survey on perceived gender, religious, and racial discrimination in their school, the students expressed concerns about the direction that the study was taking and who was making decisions. The students had learned that the research questions had indeed been decided by the adult facilitators. The adults came to realise that they had violated their own principles of shifting traditional power roles. They then revised the research questions based on the input from the students who felt the study needed to more explicitly focus on the injustices experienced by marginalised students in the school. As the researchers, school personnel, and students were building their relationships, they were also aware of the need to understand the historical and current contextual factors that were influencing school inequities (Garnett et al., 2019). They used a mixed methods approach to collect data from existing databases and qualitative data through surveys and meetings with the students and staff. The quantitative data revealed two things: first, the demographics of the area had changed considerably over the previous decade with many more families from communities of colour, immigrants, and students with multi-lingual backgrounds. Second, the number of suspensions of students from school was disproportionately high for students from these

The Philosophical Foundations of a Transformative Approach

45

families. They argued that the White supremacy and ableist legacy combined with the ‘zero tolerance’ disciplinary policy resulted in a disproportionate number of minorities and children with disabilities being suspended from school. Furthermore, the data indicated that Black and Brown students or those with emotional or behavioural problems were more frequently involved in the juvenile justice system when they were excluded from school. The qualitative data supported that Black and Brown students and those with disabilities experienced harsher disciplinary practices than their White or nondisabled peers. Before engaging with the research team, school personnel had already identified an intervention called restorative practices as a programme that could reduce the disproportionate suspension of students of colour and those with disabilities (Garnett et al., 2019). The researchers did not accept the proposed intervention uncritically based on the assumption that the knowledge and experiences of the students needed to be considered before an intervention was confirmed. Based on the quantitative and qualitative data that were collected prior to implementation of the programme and the feedback from students about what was needed, the intervention was substantially changed to be more culturally responsive and fair. Thus, the transformative methodological assumptions were evidenced in this study by careful attention to establishing culturally responsive relationships with the full range of stakeholders, consciously addressing traditional power inequities, being explicit about addressing issues of discrimination, and valuing the knowledge of those who typically hold the least power in school systems. The data were actively used to revise the research questions, the methods used to collect data, and the intervention that was implemented. The use of mixed methods allowed the researchers to examine cultural and contextual issues as well as incorporate changes that were needed in the disciplinary policies and practices. 3.12

Permeability of Borders Across Paradigms

The transformative paradigm is made up of a unique set of philosophical assumptions that differentiates it from other paradigms that are used in the mixed methods community. However, it shares common ground with other paradigms, thus allowing for opportunities for dialectical pluralism and integration across paradigm borders (Greene & Caracelli, 1997; Johnson, 2012; Johnson & Schoonenboom, 2015). Chilisa and Mertens (2021) highlight the similarities across the transformative and Indigenous paradigms in that both are concerned with engaging with the full range of stakeholders in culturally responsive ways with the goals of increasing social, economic, and environmental justice (Cram  & Mertens, 2015; Mertens  & Wilson, 2019). .  .  . The social justice dimension of the transformative paradigm

46

Donna Mertens

also brings an essential language that evaluators can use to address power differentials among the diverse stakeholders ranging from donors with conflicting interests to the competing interests of stakeholders, governments, and beneficiaries. (Chilisa & Mertens, 2021, p. 250) When working in Indigenous communities, it is critical that the assumptions of the Indigenous paradigm (see Chilisa, this volume) are prioritised in order to be responsive to their history of colonisation and diverse cultures of these communities, and to re-interpret transformation based on the community’s needs. 3.13

Application Illustrating Permeable Borders Across the Transformative and Indigenous Paradigms

An Indigenous South African, Lesego Serolong, returned to her homeland in the North West Province after years abroad studying economic development (Arko-Achemfuor et al., 2019; McIntyre-Mills et al., 2019). She had the idea to improve economic conditions in that high-poverty area by teaching farmers sustainable agricultural methods and subsequently established Bokamoso Impact Investing. The process that she undertook illustrates the permeable borders between the transformative and Indigenous paradigms in a study of impact investing and agriculture. Serolong partnered with researchers from South African universities, and they began by building relationships and collecting contextual data. The team explicitly valued Indigenous knowledge and protecting the environment and reflected the Indigenous ethical values of respect, appreciation, resilience, and spirituality. In keeping with the Indigenous paradigm, relationship building initially focused on the village chief, his counsellors, and the farmers, interacting through established Indigenous traditions. Qualitative data were collected using the community-based practice of having open forums to discuss problems in the community and possible solutions. Quantitative data were collected from extant data bases about unemployment, poverty, literacy levels, climate conditions, economic levels, and malnutrition. Historical analysis revealed that multiple projects had been introduced into the region by government agencies, NGOs, and other donors, promising life-changing interventions, but resulting in no substantial changes. Hence, the research team and community leaders did not want to make assumptions about the nature of the intervention in order to be responsive to the contextually specific needs of the farmers. In keeping with the transformative ontological assumption, ArkoAchemfuor and colleagues (2019) challenged the oppressive version of reality that wrote off this area as having no economic viability. Rather, they wrote: These villages and their surrounding areas have vast arable lands that are not being fully utilized due to lack of adequate public investments. This can

The Philosophical Foundations of a Transformative Approach

47

be regarded as presenting a great opportunity for growth and economic development within these communities. Hence, instead of reproducing ways of speaking which point to the non-viability of these communities in terms of economic thriving, Bokamoso is intent on forwarding a ‘version of reality’ based on seeing (and activating) untapped potential in recognition that, as Mertens (2017, p. 21) reminds us, there are consequences associated with accepting one version of reality over another. (Arko-Achemfuor et al., 2019, p. 8) Based on the quantitative and qualitative data, the research team chose to focus on improving agricultural methods and entrepreneurship. However, the contextual data on literacy levels indicated that the farmers needed to develop skills in reading, writing, and numeracy before advancing to teaching agriculture and entrepreneurship. This led the researchers to widen their relationship circle to include experts in adult literacy. In keeping with the transformative paradigm’s assumption of the need for attention to those who are marginalised in communities, the researchers asked that the literacy classes be offered to men and women in equal numbers. The community agreed with this and the training was implemented. Quantitative data were collected on the literacy and numeracy classes using standard testing required by the national government. Once the necessary literacy skills were attained, transformative change was evidenced as a result of shifting the intervention to training in sustainable agriculture, enabling the farmers to grow vegetables on a commercial scale, market their wares, and develop a cooperative. Thus, this example provides evidence of how the borders of paradigms can be crossed. Indigenous communities need to engage through an Indigenous lens; however, researchers can bring in awareness of different types of issues by combining that with a transformative lens. The philosophical assumptions associated with these paradigms present challenges and opportunities when applying them in practice and when contrasted with the assumptions of the other mixed methods paradigms. Reflections on the philosophical assumptions of the transformative paradigm and its place in the mixed methods community are discussed in the next section. 3.14

Reflections on the Transformative Paradigm’s Philosophical Assumptions

The presence of multiple paradigms in the mixed methods community provides opportunities and challenges, as well as fertile ground for innovative practice to address complex problems. The Future of Mixed Methods Task Force, under the auspices of the International Mixed Methods Research Association, identified several major themes for the future of mixed methods (Mertens et  al., 2016). Two of those themes have particular relevance for the transformative paradigm: advancing understanding of philosophy and methodology; and increasing responsiveness to complex societal problems.

48

Donna Mertens

Molina-Azorin and Fetters (2018) also recognised the importance of improving understandings of paradigms and using mixed methods research for social change. The transformative paradigm is not without its critics and its challenges. Sweetman et  al. (2010) criticised the transformative paradigm, claiming it is too vague and open to multiple interpretations. Researchers who situate themselves in this paradigm see the ‘vagueness’ as a strength because it allows them to address the complexity of contexts and problems that cannot be studied and remedied using a step-by-step cookbook approach. ‘The intentional flexibility inherent in this paradigm creates space for interdisciplinary thinking, integration of various ways of knowing and questioning the processes of mixed methods research’ (Garnett et al., 2019, p. 8). This approach allowed Garnett and colleagues to adapt their research to meet the needs of the students who experienced discrimination in the school system. One of the strengths of the transformative paradigm’s assumptions is that they prompt mixed methods researchers to critically reflect on the origins of the research questions as a means to ensure that the concerns of the communities are prioritised. Transformative mixed methods researchers reject the authority of the researcher to determine the research questions without consultation with the communities impacted by the research. This paradigm also asks questions about whose knowledge is viewed as legitimate and whose purposes are being served by the research. Exemplary questions include: Whose definitions of the research problem are given priority? How are relationships developed and sustained in culturally responsive ways that allow for the integration of community-based knowledge? How can mixed methods researchers adapt their data collection instruments and strategies to ascertain a better understanding of the context, problems, development of interventions, and determining the effects of those interventions? While such questions can be asked by researchers positioned in other paradigms, these questions are integral to using the transformative paradigm to frame a mixed methods study. The transformative axiological assumptions hold that researchers should design their studies to consciously address inequities and discrimination and to provide a basis for improving social, economic, and environmental justice. As noted previously, these are lofty goals that are viewed by some who align themselves with other paradigms as unattainable. Transformative researchers make the argument that the lack of progress on many social issues with many marginalised and vulnerable populations is rooted in inequities and discrimination; therefore, lack of attention to these issues serves to sustain an oppressive status quo. A limitation of transformative mixed methods research is that many factors needed to increase social, economic, and environmental justice are outside of the researcher’s control. Transformative researchers need to operate with full transparency about the potential limitations of the research and not

The Philosophical Foundations of a Transformative Approach

49

make promises they cannot keep to communities that have historically felt betrayed by researchers. For example, in the research with the South African farmers, a severe drought, broken down trucks, and lack of access to financial resources, conditions beyond the control of the researchers (or anyone for that matter), caused delays in expected benefits (Arko-Achemfuor et al., 2019). The farmers were discouraged and complained to the chief. Fortunately, the researchers had formed a strong relationship with the chief and the farmers. The chief was able to mediate with the community and assure them of Bokamoso Impact Investments commitment to the village. The world faces a plethora of urgent challenges including the climate crisis; the status of refugees and immigrants; conflicts and violence; a global pandemic; and on-going disparities in economic, educational, and health systems; and violations of the rights of racial and ethnic minorities, women and girls, and people with disabilities. Mixed methods researchers have the potential to be part of the solution, if they choose to do so. The use of mixed methods allows for collection of different types of data and the integration of those data that are needed to shed light on these complex issues. Mixed methods researchers from any paradigm can make the choice to be part of the solution; however, the assumptions of the transformative paradigm are quite explicit about the importance of adopting this role. These assumptions ask researchers to consciously address discrimination and oppression and inequitable power relationships. Thus, design choices in the transformative paradigm focus on community engagement and an action orientation towards increased justice. This might mean borrowing ideas from social change agents and social activists. The adoption of this role is challenging, but it also provides an opportunity for researchers to disrupt historical discrimination and contribute to a transformed world (Hall, 2020). Mertens (2010) argued that the mixed methods community benefits by having multiple frameworks that guide methodological decisions. As the then editor of the Journal of Mixed Methods Research, she wrote: ‘A priority at this point, from our editorial perspective, is to keep the spirit of divergence alive and well’ (p. 2). Innovations in mixed methods approaches can be found in research studies based in all of the paradigms, and the mixed methods community benefits by learning from these different approaches. The unique contribution of the transformative paradigm is employment of divergent thinking to address the global challenges rooted in discrimination and inequitable power relationships. Transformative mixed methods supports divergence in thinking about methodologies based on historical and cultural factors, as well as on engagement with communities. Rather than having a predetermined mixed methods design, the transformative paradigm encourages researchers to be flexible and responsive to the context. There is much room to understand better how to do this, but the transformative assumptions provide a framework that holds promise for contributing to a more just world.

50

Donna Mertens

References Arko-Achemfuor, A., Romm, N.,  & Serolong, L. (2019). Academic-practitioner collaboration with communities towards social and ecological transformation. International Journal for Transformative Research, 6(10), 1–9. Bolinson, C., & Mertens, D. M. (2019). Transformative Evaluation Toolkit for the Impact Investing Sector. Toronto, Canada: Engineers without Borders Canada. Bolinson, C., & Mertens, D. M. (2021). Transformative evaluation and impact investing: A fruitful marriage. In R. P. Herman & E. de Morais Sarmento (Eds.), Global Handbook of Impact Investing. Hoboken, NJ: Wiley. Chilisa, B.,  & Mertens, D. M. (2021). Indigenous made in Africa evaluation frameworks: Addressing epistemic violence and contributing to social transformation. American Journal of Evaluation, 42(2), 241–253. Chouinard, J. A., & Cram, F. (2020). Culturally responsive approaches to evaluation: Empirical implications for theory and practice. SAGE Publications. Christians, C. (2005). Ethics and politics in qualitative research. In N. Denzin & Y. S. Lincoln (Eds.), Handbook of Qualitative Research (3rd ed., pp.  139–164). Thousand Oaks, CA: Sage. Cook, J. R. (2014). Using evaluation to effect social change: Looking through a community psychology lens. American Journal of Evaluation, 36(1), 107–117. Cram, F., Chilisa, B., & Mertens, D. M. (2013). The journey begins. In D. M. Mertens, F. Cram & B. Chilisa (Eds.), Indigenous Pathways into Social Research (pp. 11–40) Walnut Hills, CA: Left Coast Press. Cram, F., & Hopson, R. (2018). Digging deeper to engage wicked problems through evaluation. In R. Hopson & F. Cram (Eds.), Tackling Wicked Problems in Complex Ecologies. (pp. 234–255). Stanford, CA: Stanford Business Books. Cram, F.,  & Mertens, D. M. (2015). Transformative and indigenous frameworks for multimethod and mixed methods research. In S. Hesse Biber & R. B. Johnson (Eds.), The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry (pp. 91–109). Oxford: Oxford University Press. DeJonckheere, R., Lindquist-Grantz, S. T., Haddad, K., & Vaughn, L. M. (2019). Intersection of mixed methods and community-based participatory research: A methodological review. Journal of Mixed Methods Research, 13(4), 481–502. Farias, L., Rudman, D. L., Magalhaes, L., & Gastaldo, D. (2017). Reclaiming the potential of transformative scholarship to enable social justice. International Journal of Qualitative Methods, 16, 1–10. Foucault, M. (1981). The order of discourse. In R. Young (Ed.), Untying the Text: A PostStructuralist Reader (pp. 48–79). London: Kegan Paul. Foucault, M. (2002). The Archaeology of Knowledge. London: Routledge. Foucault, M. (2003). The Order of Things. London: Routledge. Freire, P. (2006). Pedagogy of the Oppressed (M. B. Ramos, Trans.). London: Bloomsbury Academic Press (Original work published 1970). Garnett, B. R., Smith, L. C., Kervick, C. T., Ballysingh, T. A., Moore, M.,  & Gonell, E. (2019). The emancipatory potential of transformative mixed methods designs: Informing youth participatory action research and restorative practices within a district-wide school transformation project. International Journal of Research  & Method in Education. https://dx.doi.org/10.1080/1743727X.2019.1598355 Grayling, A. C. (2019). The History of Philosophy. New York, NY: Penguin Publishing Group.

The Philosophical Foundations of a Transformative Approach

51

Greene, J. C., & Caracelli, V. J. (1997). Defining and describing the paradigm issue in mixed-methods evaluation. New Directions for Evaluation, 74, 5–17. Doi:10.1002/ ev.1068. Guba, E. G., & Lincoln, Y. S. (1994). Competing paradigms in qualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.), Thousand Oaks, CA: Handbook of Qualitative Research (pp. 105–117). Sage. Guba, E. G., & Lincoln, Y. S. (2005). Paradigmatic controversies, contradictions, and emerging confluences. In N. K. Denzin & Y. S. Lincoln (Eds.), The Sage Handbook of Qualitative Research (3rd ed., pp. 191–216). Thousand Oaks, CA: Sage. Habashi, J., & Worley, J. (2009). Child geopolitical agency. Journal of Mixed Methods Research, 3(1), 42–64. Habermas, J. (1984). Theory of Communicative Action. Vol. 1: Reason and the Rationalization of Society (T. A. McCarthy, Trans.). Boston, MA: Beacon Press. Hall, M. E. (2020). Examining issues facing communities of color today: The role of evaluation to incite change. New Directions for Evaluation, 166, 13–22. Harris, R., Holmes, H.,  & Mertens, D. M. (2009). Research ethics in sign language communities. Sign Language Studies, 9(2), 104–131. Hood, S., Hopson, R., & Frierson, H. (Eds.). (2005). The Role of Culture and Cultural Context. Charlotte, NC: Information Age Publishing. Hood, S., Hopson, R., & Frierson, H. (2015). This is where we continue to stand. In S. Hood, R. Hopson  & H. Frierson (Eds.), Continuing the Journey to Reposition Culture and Cultural Context in Evaluation Theory and Practice (pp. ix–xciii). Charlotte, NC: Information Age Publishing. Howell, E. (2013). An Introduction to the Philosophy of Methodology. Thousand Oaks, CA: Sage. Johnson, R. B. (2012). Guest editor’s editorial: Dialectical pluralism and mixed research, American Behavioral Scientist, 56, 751–754. Johnson, R. B., & Schoonenboom, J. (2015). Adding qualitative and mixed methods research to health intervention studies: Interacting with differences. Qualitative Health Research, 26(5), 587–602. Doi:10.1177/1049732315617479. Kuhn, T. S. (1970). The Structure of Scientific Revolutions (2nd ed.). Chicago, IL: University of Chicago Press (Originally published 1962). Marra, M. (2015). Cooperating for a more egalitarian society: Complexity theory to evaluate gender equity. Evaluation, 21(1), 32–46. McIntyre-Mills, J., Karel, J. A., Arko-Achemfuor, A., Romm, N., & Serolong, L. (2019). Efforts to inspire transformative research with farmers in a small town in the North West Province of South Africa. International Journal for Transformative Research, 6(10), 10–19. Mertens, D. M. (1999). Inclusive evaluation: Implications for transformative theory for evaluation. American Journal of Evaluation, 20(1), 1–16. Mertens, D. M. (2010). Divergence and mixed methods. Journal of Mixed Methods Research, 4(3), 3–5. Mertens, D. M. (2017). Transformative research: Personal and societal. International Journal for Transformative Research, 4(1), 18–24. Mertens, D. M. (2018a). Mixed Methods Design in Evaluation. Thousand Oaks, CA: Sage. Mertens, D. M. (2018b). Transformative mixed methods and policy evaluation. Diritto & Questioni Pubbliche, 18(1), 247–264.

52

Donna Mertens

Mertens, D. M. (2020). Research and Evaluation in Education & Psychology: Integrating Diversity with Quantitative, Qualitative, & Mixed Methods (5th ed.). Thousand Oaks, CA: Sage. Mertens, D. M. (2023a). Mixed Methods Research. London: Bloomsbury Press. Mertens, D. M. (2023b). Transformative mixed methods research. In R.J. Tierney, F. Rizvi, & K. Erkican (Eds.), International Encyclopedia of Education (4th ed., pp. 402–410). Amsterdam: Elsevier. https://dx.doi.org/10.1016/B978-0-12-818630-5.11040-1. Mertens, D. M., Bazeley, P., Bowleg, L., Fielding, N., Maxwell, J., Molina-Azorin, J. F., & Niglas, K. (2016). Expanding thinking through a kaleidoscopic look into the future: Implications of the mixed methods international research association’s task force report on the future of mixed methods. Journal of Mixed Methods Research, 10(3), 221–227. Mertens, D. M., & Boland, A. (2018). Complex ecologies in international development evaluation: Focusing on women and people with disabilities. In R. Hopson  & F. Cram (Eds.), Tackling Wicked Problems in Complex Ecologies (pp. 103–128). Stanford, CA: Stanford Business Books. Mertens, D. M. & Wilson, A. T. (2019). Program evaluation theory and practice. New York, NY: Guilford. Miller, R. L. (2018). Evaluating HIV practices and evidence-supported programs in AIDS community-based organizations. In R. Hopson  & F. Cram (Eds.), Tackling Wicked Problems in Complex Ecologies (pp. 69–102). Stanford, CA: Stanford Business Books. Miller, R. L. (2020). Reducing Stigma and Discrimination in Access to HIV Health Care for Gay and Bisexual Men and Transgender Women Using Mystery Patients in Cameroon and Zimbabwe. East Lansing, MI: Michigan State University. Miller, R. L., Rutledge, J., & Ayala, G. (2021). Breaking down barriers to HIV care for gay and bisexual men and transgender women: The advocacy and other community tactics (ACT) Project. AIDS and Behavior. https://doi.org/10.1007/s10461-02103216-w (accessed 28 February 2021). Molina-Azorin, J. G., & Fetters, M. D. (2018). Future special issues at the Journal of Mixed Methods Research. Journal of Mixed Methods Research, 12(4), 369–370. Molina-Azorin, J. F., & Fetters, M. D. (2019). Building a better world through mixed methods research. Journal of Mixed Methods Research, 13(3), 275–281. Molina-Azorin, J. F., & Fetters, M. D. (2020). Virtual special issue on “paradigms in mixed methods research”. Journal of Mixed Methods Research, 14(1), 6–10. Neubauer, L. C., McBride, D., Guajardo, A. D., Casillas, W. D., & Hall, M. E. (Eds.). (2020). Examining issues facing communities of color today: The role of evaluation to incite change. New Directions in Evaluation, 166, Summer. Poradzisz, M.,  & Florczak, K. L. (2018). Transformative research: A  new frontier in nursing. Nursing Science Quarterly, 31(2), 117–120. Shannon-Baker, P. (2016). Making paradigms meaningful in mixed methods research. Journal of Mixed Methods Research, 10(4), 319–334. Stephens, A., Lewis, E. D.,  & Reddy, S. M. (2018). Inclusive Systemic Evaluation (ISE4GEMs): A New Approach for the SDG Era. New York: UN Women. Sullivan, M., Derrett, S., Paul, C., Beaver, C., & Stace, H. (2014). Using mixed methods to build research capacity within the spinal cord injured population of New Zealand. Journal of Mixed Methods Research, 8(3), 234–244. Sweetman, D., Badiee, M., & Creswell, J. W. (2010). Use of the transformative framework in mixed methods studies. Qualitative Inquiry, 16(6), 441–454.

The Philosophical Foundations of a Transformative Approach

53

Tarahita, D., & Rakhmat, M. Z. (2018). Indonesia’s Citarum: The world’s most polluted river. The Diplomat, 28 April 2018. https://thediplomat.com/2018/04/indonesiascitarum-the-worlds-most-polluted-river/ (accessed 21 August 2019). Tashakkori, A., Johnson, R. B., & Teddlie, C. (2021). Foundations of Mixed Methods Research. Thousand Oaks, CA: Sage. Widianingsih, I., & Mertens, D. M. (2019). Transformative research and the sustainable development goals: Challenges and a vision from Bandung, West Java. International Journal of Transformative Research, 6(1), 27–35. Wilkerson, I. (2020). Caste: The Origins of Our Discontents. New York, NY: Random House.

4 PHILOSOPHICAL UNDERPINNINGS OF MIXED METHODS Decolonizing Evaluation Practice Through Decolonizing Paradigms Bagele Chilisa

4.1

Transformative Mixed Methods With a Postcolonial Indigenous Paradigm

In this chapter, I discuss the philosophical foundation of a postcolonial indigenous research paradigm. A postcolonial indigenous paradigm is but just one of the cultural indigenous paradigms that fall under the umbrella of what I have named here an Indigenous Science paradigm. Elsewhere, I have referred to these cultural paradigms as postcolonial indigenous paradigms. The main argument is that postcolonial indigenous paradigms with assumptions about a reality that emphasizes connectedness and interdependence of people with nature, knowledge that is relational and ethics built on reciprocity, and respect add significant dimensions to the motivation and justification for mixed methods research. This chapter commences with an explanation of indigenousness and indigenous knowledge systems followed by assumptions about postcolonial indigenous paradigms and their implications for the motivation and justification for a postcolonial indigenous MMR. The argument is that relational ontology, epistemology, and axiology emphasize connectedness and relationality as indigenous systems’ thinking that promotes interaction of knowledge production structures and the importance of building relationships with and among participants and with the environment to improve the quality of data, and provide pathways towards equitable and sustainable futures. This chapter further discusses how to plan for a postcolonial indigenous MMR, its method theory, and the MMR designs. 4.1.1 Situating an Indigenous Science Paradigm

It is common practice to discuss and illustrate MMR within the context of research paradigms. In that context, motivation and justification for MMR are DOI: 10.4324/9781003273288-5

Philosophical Underpinnings of Mixed Methods

55

made according to philosophical assumptions of four common paradigms sometimes referred to as the big four namely the post-positivist, constructivist, transformative, and pragmatist (Mertens, 2010). Post-colonial Indigenous paradigms come under the umbrella of a fifth paradigm in this typology of knowledge systems (Wilson, 2008; Chilisa, 2019; Smith, 2012; Romm, 2015; Held, 2019). The fifth paradigm brings unique dimensions in the discourse on the philosophical foundations of MMR. Reluctance by researchers and practitioners to engage with the fifth paradigm, however, threaten its growth and application to addressing real-world problems. This reluctance to the application of the fifth paradigm to research and evaluation practice is due partly to misconceptions about the meaning and value of indigenous knowledge systems of the formerly colonized societies and the application of these knowledge systems to research. It is therefore necessary to begin by clarifying the terms indigenous and indigenous knowledge systems. Elsewhere (Chilisa, 2019), I  have defined indigenous in the context of research and evaluation to refer to a cultural group’s ways of perceiving reality ways of knowing and the value system that inform the research and evaluation process. Euro-Western paradigms are, for example, informed by the history, cultures, and experiences of Euro-Western societies and are therefore indigenous to them. What has remained peripheral is that which is indigenous to the majority of the formerly colonized peoples of Africa, the Indigenous peoples of Canada, Australia and the USA also referred to as First nations. Their local Indigenous ways of perceiving reality, ways of knowing and value systems evolving from their histories, cultures, experiences, and their resistance to domination and muting of their voices that come with the universalization of Euro-Western research paradigms is what comes under the umbrella term ‘indigenous’. Indigenous in the context of research therefore refer neither to people nor to the exotic but to a science of knowledge that is rational, deserving a space in the discourse on research paradigms and counting as a fifth paradigm worth teaching across global academic institutions. Mixed method research informed by an Indigenous science paradigm can therefore be learnt and adapted across contexts and carried out by all researchers irrespective of their geopolitical origin in the same way that MMR from other paradigmatic positions is adaptable to diverse contexts by researchers from diverse cultures and geo-political regions. The adaptation should nevertheless be carefully done to ensure that differences are not erased by over generalizing indigenous knowledge. Place is important within indigenous approaches, thus adaptation should be preceded by ‘the question do methods have a place or do places have methods’ (Levac et al., 2018). Indigenous knowledge can be specific to locations, regions, and groups of peoples – for instance, indigenous knowledge for women, the deaf community, and the poor. Euro-Western research methodologies come out of the histories, experiences, values, philosophies, and cultures of Euro Western

56

Bagele Chilisa

societies and are therefore indigenous to the Western academy, its institutions, and the dominant group. Postcolonial indigenous knowledge is connected with the formerly colonized and the historically oppressed. Indigenous knowledges, therefore, differ from conventional knowledges in their absence of colonial and imperial power (Chilisa, 2019). These indigenous knowledges are also postcolonial, since they arise from people who have lived colonial history and have learned to value justice through collectively fighting for their liberation. This does not however imply that colonization has ended. Colonialism is still alive. Grenier (1998) notes that indigenous knowledge can be used synonymously with traditional and local knowledge to differentiate the knowledge developed by a given social group or community from the knowledge generated by the Western academy and its institutions. Further clarifying indigenousness, George J. Sefa Dei (2002) notes that, with reference to Africa, indigenousness may be defined as knowledge consciousness arising locally and in association with long-term occupancy of a place. From an African perspective, indigenousness refers to the traditional norms, social values, and mental constructs that guide, organize, and regulate African ways of living in making sense of the world. 4.1.2 Indigenous Knowledge Science

Indigenous knowledge is today recognized, not only as content but also as a science that frame indigenous thought and philosophy (Briggs, 2013). Held (2023) argues that Western Science colonizes the world by referencing alternative science knowledge as indigenous knowledge systems. She argues that Indigenous scholars perpetuate the hierarchy by consistently referring to their indigenous science as Indigenous knowledge systems. She advances the argument that ‘science is a collection of principles, and practices varying between branches of science as well as individuals, social groups and cultures’ and that ‘Western science is just but one of the many localised and contextualised sciences.’ Indigenous knowledge can serve as a foundation for problem-solving strategies for local communities (World Bank, 1998). Despite this recognizable value, it still remains underutilized. The question of how to bring indigenous knowledge to the research process to offer sustainable solutions to development problems is therefore overdue. Of what use is MMR if it cannot bring on board local communities problem-solving strategies? Seeking understanding in research that privileges local community’s ways of knowing is a critical dimension to the motivation and justification of MMR from a postcolonial indigenous science paradigmatic lens. Indigenous is also linked to assertions of self-determination and selfgovernance. Thus, indigenous further invites researchers to reflect on the fundamental questions of whose research is it? What are the structures in the research environment that can possible transmit the ideologies of the powerful

Philosophical Underpinnings of Mixed Methods

57

and perpetuate dominance of some groups? What are the strategies for building relationship to promote interaction of all knowledge systems? Who has the right to know and what can be known? Who will own the data and the report? These fundamental questions point to the need to address structures that almost dictate what can be researched. While the main focus of MMR has been on the methodologies, the focus of MMR from a postcolonial indigenous science paradigm is on the knowledge systems, the structures that aid knowledge production, and the decolonization process that is required to create and enable pathways towards equitable and sustainable futures. The theory, methodology, and motivation for MMR should arise from a well-defined indigenous science paradigm. What follows is a discussion of an indigenous science paradigm and implication for a postcolonial indigenous MMR. 4.2

An Indigenous Science Paradigm

An Indigenous Science Paradigm builds on the ontological, epistemological, and axiological assumptions shared across the majority two-thirds formerly colonized and marginalized groups Although named differently by different scholars – for example, postcolonial indigenous paradigm informed by the African world views (Chilisa, 2017), Kaupapa M¢ori Evaluation paradigm informed by the M¢ori world view (Cram & Mertens, 2015), and Indigenous Hawaiian evaluation framework informed by Aloha (CREA-HI, 2019) – these cultural paradigms share the same social theory, history of colonization, resistance, and the struggle for the right to know. I highlight common ontological, epistemological, and axiological assumption from scholars in Canada, Australia, North America, New Zealand, and Africa that inform the philosophical foundations of an MMR. 4.2.1

Philosophical Justification of MMR: Relational Ontology: Spirituality, Love, and Harmony

Postcolonial indigenous paradigms have in common a relational ontology. A relational ontology addresses the nature of being and how worldviews on being are implicated in the social construction of realities. From an African perspective, an Ubuntu worldview expresses an ontology that addresses relations among people, relations with the living and the nonliving, and a spiritual existence that promotes love and harmony among peoples and communities. Ubuntu and the African adage, “I  am we; I  am because we are; we are because I  am” (Goduka, 2000), explains the web of connection of people with each other and with the living and the nonliving. In Australia and Canada, scholars articulate a relational ontology that emphasizes relations with people, with the environment/land, with the cosmos, and with idea (Wilson, 2008). Relationship building is an essential aspect of everyday life

58

Bagele Chilisa

experience for First Nations communities in Australia, Canada, and African people. The principle is in direct contrast to the Eurocentric view of humanity, ‘I think, therefore, I am’, which was expressed by René Descartes. The latter, Ivy Goduka (2000) observes, expresses a concept of self that is individually defined and ‘is in tune with a monolithic and one-dimensional construction of humanity’ (p. 29). In Africa, existence-in-relation and being-for-self-andothers sum up the African conception of life and reality (Onyewumi, 1998). To illustrate the importance of this relationship building, greetings among people in Africa and Indigenous people in Canada and Australia include asking an acquaintance not only about their well-being but also about the wellbeing of their relatives and the environment (Wilson, 2008; Chilisa, 2019). When a neighbour or a relative is not well, those connected to them will describe themselves as not well. When there is a tragedy in the community, wellness will be referenced to the community tragedy. In the context of research, the researcher becomes part of circles of relations that are connected to one another and to which the researcher is also accountable. The implication for research is that participants in a research study are able to make connections with each other when this greeting ritual is respected. Greeting becomes a way of building relationships and rapport among participants and researchers inevitably improving the validity of the research evidence. African people and First Nations Peoples of Canada, North America, and Australia also recognize a spiritual connection with the environment/land. Their relationship with the environment/land has implications for the way research is conducted. Spirituality can be viewed as a connection to the cosmos so that any exercise that increases connection or builds relationships is spiritual and ceremonial in nature. A  recognition of spirituality allows researchers to explore the interconnections between the researcher’s experience of the sacred and the practical aspects of research. Understanding comes through factual and oral history that connects the researched to the ancestral spirits. Knowledge is also regarded as a sacred object, and seeking knowledge is a spiritual quest that may begin with a prayer or a ceremony. Knowing can thus come through prayer as a way people connect themselves with those around them, the living, the nonliving, and the ancestral spirits. In this way, the mind, the body, and the spirit are regarded as legitimate ways of gathering information and coming to know (Pelletier, 2003). 4.2.1.1 Relational Ontologies: Implications for a Transformative Postcolonial Indigenous Mixed Methods Research

A relational existence opens questions about ownership of data and knowledge that is produced. In a study to understand and explore local community values, beliefs, and practices that relate to research ethics in Botswana and how communities would want their interests represented, one major

Philosophical Underpinnings of Mixed Methods

59

concern raised by the communities was the total dispossession of specimen collected from their bodies for research which could range from blood samples to human parts. Their ontological existence requires a practice where they could claim back what is part of them instead of creating data banks on what is part of them in well-resourced countries (Koloi-Keaikitse et al., 2021). Ontological existence challenges scientific colonization, a process that creates the belief that researchers have unlimited rights and aces to any data source and information including the right to export the data for purposes of creating new knowledge system ranging from vaccines and new patents to theories, formulae, and books (Chilisa & Phatshwane, 2022). An emphasis on relational existence address the social injustices perpetuated through knowledge structures such as ethics instruments and legislature that remain unchallenged in research. Another well documented example of injustices in research is the well documented story of how a UK company working with a South African Council of Scientific and Industrial research stole knowledge from the San, an Indigenous ethnic group in Southern Africa to manufacture a diet pill that fetches large amounts of money for pharmaceutical companies (Chilisa, 2019; Commey, 2003). Clearly then indigenous MMR addresses not only methodologies that produce inequalities in societies but also the inequalities produced and sustained through multiple knowledge structures that define the research contour long before the researcher enters the field. 4.2.2

Relational Epistemology

A relational epistemology draws our attentions to relational forms of knowing as opposed to individual descriptions of knowing. Whereas traditional epistemologies focus on the objects of knowledge, relational epistemologies focus on subjects or communities as knowers. Knowing is something that is socially constructed by people who have relationships and connections with each other, the living and the nonliving, and the environment. Knowers are seen as beings with connections to other beings, the spirits of the ancestors, and the world around them that inform what they know and how they can know it. African perspectives view relational epistemology as knowledge that has a connection with the knowers. Epistemology is the well-established general beliefs, concepts, and theories of any particular people, which are stored in their language, practices, rituals, proverbs, revered traditions, myths, and folktales. This knowledge is practiced in various fields such as medical science, religion, child bearing, agriculture, psychology, and education. This relational epistemology has favourite ways, usually institutionalized in the society, of acquiring new knowledge and evaluating accepted fact; it has accepted authorities (whether people, institutions, or texts) in matters of knowledge and beliefs (Kaphagawani  & Malherbe, 2000). Perspectives on relational epistemology from Australia and Canada focus on the researched

60

Bagele Chilisa

as knowers with a web of connections that inform what they know and how it can be known. Explaining knowing informed by the multiple connections of knowers with other beings and the environment, Deloria (1995) observes, for instance, that indigenous communities gain knowledge and understanding of the world by participating in events and observing nature such as the birds, animals, rivers, and mountains. Wilson (2008) and Getty (2010) add that knowledge comes from the people’s histories, stories, observation of the environment, visions, and spiritual insights. A common thread in relational epistemologies is that knowledge arises out of the people’s relationship and interaction with their particular environments. This view underscores the right of the formerly colonized and indigenous peoples to construct knowledge in accordance with the self-determined definitions of what they want to know and how they want to know it. 4.2.2.1

Relational Epistemologies: Implications for a Transformative Postcolonial Indigenous Mixed Methods Research

In transformative postcolonial indigenous mixed methods, the researcher challenges prejudices, stereotypes, and deficit theorizing, about the ‘other’. The researcher further resists reliance on dominant knowledge systems that marginalize the formerly colonized’s epistemologies and perpetuate inequalities of knowledge systems and impoverishment of communities through research practices that steal people’s knowledge and deny them their relational existence. The researcher invokes postcolonial theory as a tool of framing the ways of knowing, data collection tools, analysis, and interpretation. Four concepts can assist the researcher in outlining inequalities of knowledge systems and the resultant inequalities between the rich and the poor namely academic imperialism, the role of literature and language in research, resistance to colonizing ideologies, and decolonization. Academic imperialism: It refers to the tendency and practice to denigrate, dismiss, and attempt to quash alternative theories, perspectives, frameworks, methodologies and data collection tools, and interpretations, reporting, and dissemination of findings practices that are non-Western (Chilisa, 2019). A  transformative indigenous MMR pays attention to inequalities of knowledge systems and creates strategies for knowledge systems to interact. The researcher also navigates strategies to address inequalities between the researcher as knower and colonizer, and the researched as ‘the to be known’ and as ignorant and other power-based contextual variables that are the sources of injustices. Language: In a transformative indigenous MMR, language is the main vehicle through which indigenous knowledge is preserved, enabling concepts, names, and theories, artefacts, and objects to be known the way communities

Philosophical Underpinnings of Mixed Methods

61

would want them to be known. Academic researchers have, for example, a tendency to rename or translate into dominant languages’ indigenous concepts, artefacts, and objects, resulting in the epistemic violence and loss of ownership of communities’ heritages. A herb that is brought from the community into a medical science laboratory, for example, may lose its indigenous name and be given a name that communities would never guess it is the same herb whose knowledge they shared with the academics (Chilisa, 2017). Indigenous knowledge is predominantly oral, while Western knowledge system is dominated by the written text. A  combination of the two addresses knowledge and power asymmetries while at the same time bringing to the research diverse sources of theorizing, of data sources and evidence and new interpretations. Resistance and decolonization: The motivation and rationale of a transformative indigenous MMR are to contribute to the process of decolonization by interrogating ‘colonial nonsense about the other’ (Bhabha, 1994) and to resist unidirectional borrowing of knowledge from the West and articulate strategies for bringing together multiple knowledge systems to reverse deeprooted asymmetries of knowledge and power in research and to envision new data collection tools and theoretical frameworks that privilege indigenous knowledge systems. 4.2.3

Relational Axiology

The Bantu in southern Africa discuss a relational axiology that is embedded in the ubuntu relational ontology principles of (1) I am we, I am because we are; (2) relations of people with the living and the nonliving; and (3) spirituality, love, harmony, and community building. From these principles, an ethical framework emerges that emphasizes accountable responsibilities of researchers and respectful relationships between the researchers and the researched that takes into account the researched’s web of relationships with the living and the nonliving. Linda T. Smith (1999), writing on rights, regulations, and relations with the Maori people in New Zealand, proposes that the researcher using an indigenous framework needs to interrogate questions on ownership of research, the interests it serves, the benefits to the researched, and the role of the researched in framing the research, designing the research questions, carrying out the work, writing up the research findings, and disseminating the results. When pharmaceutical make money from stolen indigenous knowledge and academics rename indigenous artefacts and hide them from communities, what is in question is not only the research methodologies but also structures that produce the knowledge. Nine principles are discussed in the literature (Chilisa & Mertens, 2021) that are vital to the success of building relationships between the researchers

62

Bagele Chilisa

and the communities and connecting with the environment as follows. We summarize these principles as follows: 1. Relationality: The emphasis is on belongingness, togetherness, interdependence, relationships, collectiveness, love, and harmony. There is emphasis on valuing community strength and building community relationships to inform research or evaluation intent, motive, and methodology. 2. Responsibility: It is about the role of a researcher or evaluator in pursuing social, economic, and environmental justice, resisting dominant ideologies that silence local communities, and contributing to the worth, health, unity, and harmony within the community and of all stakeholders. 3. Reverence: Indigenous research recognizes the critical nature of spirituality and values as an important contribution to ways of knowing. Many indigenous people place value on sacred sites and spiritual practices. The researcher with a postcolonial lens needs to figure out what is revered and how they will participate in it. 4. Reciprocity: Whose research is it? Who will benefit from it and how will they benefit? These are fundamental questions that address the pitfall of colonial research that serves the interests of the researchers. 5. Respectful representation: Respect requires that the process, from the initiation of the research, the questions asked, the methodology, the datacollection procedures, and the reporting and dissemination of the report, is guided by the community and that the community has ownership and access to the data collected. 6. Reflexivity: The principle of reflexivity requires researchers to continuously reflect on their position within existing powers and ensure that the research addresses the priority needs of communities. 7. Responsivity: Responsiveness is the ability of researchers or evaluators to learn from the process, recognize the evolving changes, and adapt their approaches and methodologies to become the change, context, and be culturally sensitive and appropriate. 8. Rights and regulations: This calls for ethical protocols that accord communities the rights and opportunities to prioritize their needs and participate in the research design. 9. Decolonization: This calls for resistance to blindly borrowing Western theories, conceptual frameworks, and methodologies and to adapting these methodologies and theories where necessary to make them contextually and culturally relevant and, where possible, coming with new method theories from the data. (Chilisa & Phatshwane, 2022) 4.2.3.1

Implications for Ethical Standards

The emphasis in a transformative indigenous MMR is on questioning the binary knowledge system that privileges and promotes unidirectional borrowing of

Philosophical Underpinnings of Mixed Methods

63

knowledge from the West, interrogating the knowledge structures that perpetuate a biased system of knowledge production and creating space for diverse and inclusive strategies and methodologies for knowledge production. Ethical standards in an indigenous MMR therefore starts with bringing together Eurowestern standards informed by an individualistic view of existence and a collective view of existence common among Indigenous peoples, First Nations and the formerly colonized of Africa. The so called universal ethics standards applied by academic and research institutions need to be combined with community ethical standards and ethical protocols. The principle of social justice in an indigenous MMR requires researchers to recognize the historical injustices and work towards promoting the aspirations, needs, and priorities of the formerly colonized and historically oppressed and those consistently marginalized on the basis of gender, race, ethnicity, ableness, geopolitical location, and so on. Applying a postcolonial indigenous MMR therefore requires invoking postcolonial discourses, feminist theory, and critical race theory to disrupt power differentials. In addition, it requires that universal principles of social justice be combined with a community or social group informed justice. An indigenous MMR therefore interrogate systems of ethics and systems of social justice allowing them to interact. Upholding the nine ethics principles outlined earlier requires the use of available indigenous tools and methods and the development of new ones. Storytelling, yarning, sharing circles, art-based methods, Pagtanung-tanong, Talanoa and Faafeletui, songs, taboos, myths, and proverbs are some of the methods that can be used to explore all dimensions of the motivation and justification for MMR Levac et al. (2018). Table 4.1 is a summary of the social theory, assumptions about the nature of reality (ontology), epistemology (knowledge), axiology (values), and the method theory of a postcolonial indigenous paradigm from an African perspective compared to the big four paradigms. From the discussion here, relational ontology, epistemology, and axiology emphasize connectedness and relationality as an indigenous systems thinking that promotes interaction of knowledge production structures and the importance of building relationships with and among participants and with the environment to build pathways towards equitable societies and sustainable futures. Indigenous tools and methods create a conducive environment for the interaction of knowledge systems and the quality and validity of research evidence. Third- and fourth-world communities have resisted intrusion into their lives since the colonial period. Schumaker (2001) has documented how, in present-day Zimbabwe, the researched villagers who, to some degree, had an antagonistic relationship with the anthropologist would give them unreliable information. Today researcher questions can be met with silence where cultural norms are not honoured by the researcher. Ellis and Earley (2006) give an example from their fieldwork of how the researcher’s relation with the cosmos became part of the research process that informed the construction of knowledge. The Anishambe, one of the indigenous people of North

TABLE 4.1 Beliefs Associated With the Four Paradigms and an Indigenous Research Paradigm

Positivist Paradigm Reason for doing the research

Interpretive Paradigm

Transformative Paradigm Pragmatic Paradigm

To discover To understand and To destroy myths and laws that are describe human empower people generalizable nature to change society and govern the radically universe

Philosophical Informed mainly underpinnings by realism, idealism, and critical realism

To solve problems and develop interventions

Indigenous Research Paradigm To challenge deficit thinking and pathological descriptions of the formerly colonized and reconstruct a body of knowledge that carries hope and promotes transformation and social change among the historically oppressed Informed by indigenous knowledge systems, critical theory, postcolonial discourses, feminist theories, critical race-specific theories, and neoMarxist theories

Informed by Informed by critical Largely informed by the hermeneutics theory, postcolonial pragmatist philosophy and discourses, feminist of John Dewey phenomenology theories, race-specific theories, and neoMarxist theories Ontological One reality, Multiple socially Multiple realities shaped The practical effects of Socially constructed multiple realities assumptions knowable constructed by human rights ideas shaped by the set of multiple within realities values, democratic and connections that human beings have probability social justice values, with each other, the environment, and political, cultural, the cosmos, the living, and the economic, race, non-living ethnic, gender, and disability values Place of values Science is value Values are an Researchers prioritize Researchers’ values All research must be guided by in the research free, and integral part of the value of furthering matter and knowledge a relational accountability that process values have no social life; no social justice and is valuable only if it has promotes respectful representation, place except group’s values human rights. positive consequences reciprocity, and rights of the when choosing are wrong, only researched. The ethics theory is a topic different informed by appreciative inquiry and desire-based perspectives

Nature of knowledge

Objective

Subjective; idiographic

Dialectical Knowledge should be Knowledge is relational, as is all the understanding aimed viewed only in terms of indigenous knowledge systems built at critical praxis its practical use. on relations It is informed by a theory Any knowledge that leads It is informed by the set of multiple What counts as Based on precise Truth is context relations that one has with the dependent that unveils illusions to pragmatic solutions truth observation universe is useful. The mandate and of science is not to measurement find truth but facilitate that is human problem-solving verifiable Participatory, liberatory, and Mixed methods, the Methodology Research designs: Research designs: Research designs: transformative research approaches research questions or combination of qualitative, quantitative, and methodologies that draw from the research objectives quantitative and phenomenology, correlational, indigenous knowledge systems, should determine the qualitative action ethnographic, quasianchored in relationship building methodology research; participatory symbolic experimental, and peoples’ connectedness with research interaction, experimental, each other and the environment, naturalistic causal values the physical, the spiritual, the comparative, emotional, historical, social, and the survey ideological aspect of the research phenomena Qualitative and Techniques based on relationship Mainly interviews, Culturally responsive Techniques of Mainly techniques of data quantitative methods building and connectedness of participant gathering data questionnaires, collection people with each and with the observation, observations, environment, ethnophilosophy, pictures, tests, and language frameworks, indigenous photographs, experiments knowledge systems, talk stories, diaries, and and talk circles; songs, rituals, and documents adapted techniques from the other four paradigms Source: Adapted from Chilisa B (2019) Indigenous Research Methodologies, London, Sage.

66

Bagele Chilisa

America, use aseema (tobacco) as a cultural symbol for thanking people, asking for help, praying for information, and sharing stories. According to Edward Benton-Banai (1988), tobacco sharing comes from the legend about Waynaboozhoo. Waynaboozhoo told the people that to seal peace, brotherhood, and sisterhood among bands, tribes, and nations, they had to smoke tobacco in the Pipe. The explanation was that the smoke that came from the Pipe would carry their thoughts and prayers to the Creator just as their Tobacco offerings in the fire would (Ellis  & Earley, 2006). In a research on grieving for the death of a loved one among the Anishambe, Ellis and Earley (2006) had challenges gaining entry into the setting and conducting interviews. They got advice from one of the Anishambe people to offer tobacco to the participants. Once they brought tobacco, Ellis and Earley (2006) report, key informants responded positively to the request. The lesson learned is that building relationships, contributes to the quality and validity of the research evidence and point the way to indigenous methods that researchers need to adopt. Without these relationships, the research literature is likely to build stereotyped findings which in essence reflect resistance from the colonized because the researcher is unable to access the realities of the communities’ experiences. The resistance has been largely ignored, because it questions the validity of the colonial research-built theories and the universalization of Euro-Western research paradigms (Chilisa, 2019). It is clear that in most postcolonial indigenous societies, building relations requires a process that connects the researcher to the researched through sharing of values or through practices that recognize and honour connectedness of the people with one another and with their environment. As discussed earlier, the system of connectedness introduces into the research space indigenous systems thinking that broadens the research environment to bring multiple structures in the research process, which are plausible threats to research evidence and thus necessitate interrogation. Mixed methods research from a post-colonial indigenous science paradigmatic perspective begins with mapping the diverse structures implicated in knowledge production, mapping relationships that create space for power sharing and the diversity of the ways of knowing. Mixed methods research is therefore a systems thinking about how to combine knowledge systems, Indigenous and non-indigenous, to more effectively challenge the legacy of colonialism, address social injustices, enhance the validity of research evidence, and promote social transformation. The research environment is made up of the knowledge systems, from research funders, academic institutions and their academic structures, ethics review boards, the researched knowledge systems, and the researchers’ predominant academic knowledge. What follows is discussion of research as a system with structures and how these structures motivate and justify a distinct transformative postcolonial indigenous MMR.

Philosophical Underpinnings of Mixed Methods

4.3

67

Planning MMR With a Postcolonial Indigenous Lens

While MMR has emphasized the use of quantitative and qualitative data sources to improve the quality and validity of the findings, it has emerged that the findings may still be misleading or worse still irrelevant, because the whole research process is premised on dominant knowledge systems that may include knowledge systems of the funders and academic institutions. For this reason, knowledge systems should interact and converge at the planning stage of the real-world problem under inquiry. At the planning stage, dominant structures may include research funders, academic institutions, and researched communities and the researchers. In evaluation research, power held by donor or funding agencies is perhaps the greatest power that defines what can be known and how it can be known leading to funder colonialism (Billman & Chilisa, in press). It is common practice for funders, for example to link pre-determined research topics to research methodologies aligned to Western knowledge systems. This has contributed to the death of other knowledge systems. MMR from a postcolonial indigenous perspective invokes postcolonial and decolonial theories to address asymmetries of power in knowledge production and issues of culturally appropriate and relevant research from the planning stage. Funders are gradually embracing culturally appropriate and relevant research as a measure of rigor. What remains is researchers and their academic institutions’ appreciation of MMR not only as combination of qualitative and quantitative data to promote rigor and validity but also as a process to decolonize and interrogate knowledge systems envisioned as practices, routines, power structures, mindsets, values, cultures, philosophies, ideologies that affect how knowledge is produced and by whom. In this context, MMR is best described as an interaction of knowledge systems with the purpose to address asymmetries of power and create space for all knowledge systems. The aim is to generate a composite, multidimensional understanding of the problem and the relationship and the interconnections of its parts. This requires a dialogue among stakeholders that map out the knowledge structures, the different ways of knowing (epistemologies), the interests, cultures, values, and world views. The rationale and motivation for a postcolonial MMR at the planning stage is illustrated with a case study of the National Institute of Health grant (R24 HD05669). 4.3.1

Clarifying the Real-World Problem and Building Relationships: A Case Study

At the planning stage, the motivation for a postcolonial MMR is to address and reverse deep asymmetries of knowledge and power and colonial prejudices by mapping out stakeholders’ relationships with knowledge structures, each other, and the environment or ecosystem. At the planning stage, stake holders should inform themselves of the institutional ethics review board

68

Bagele Chilisa

guidelines, the Community ethics guidelines or community values likely to impact the research process, the funder knowledge systems, each researcher’s perspective on the multiple ways of knowing or paragmatic perspective and stakeholders’ interests and meaning making of the problem of study. A case study of the National Institute of Health (NIH) grant serves as an example. In 2007, the University of Botswana, in partnership with the University of Pennsylvania, responded to a USA NIH grant on HIV/STI prevention. The research started with a process of mapping the relationships that the researchers had with each other, with the knowledge systems and perspectives on existence. This was followed by building relationships and addressing hierarchical structures that privileged dominant ways of knowing, cultures, and literatures. The research personnel from the U.S. university came from minority and less privileged groups in the USA. Issues of culturally appropriate and relevant research were as important to them as to the researchers in Botswana. Our dialogue on building relationships started with agreeing on a collaboration model that transformed hierarchical relationships that exist between the universities in the North and those in the South; academic institutions and communities and a model that created spaces for the integration of cultural knowledge with global knowledge to promote cultural relevancy and usefulness of research outcomes to communities as well as ensuring that the researchers remained accountable to the communities. We agreed on a collaboration model that gave the role of principal investigator (PI) and other leadership roles to University of Botswana researchers. This was done to break the stereotype that expertise can only come from Northern universities (Chilisa, 2006; Pryor et al., 2009). 4.3.2

Understanding and Prioritizing the Diverse Aspects of the Problem

The USA NIH grant on HIV/STI prevention was broad, leaving the researchers to bring in their interests to shape its scope. The academic researchers were interested in adolescent HIV/AIDS prevention. In an effort to integrate interests and create a composite understanding of HIV prevention, organizations working on HIV prevention were brought to a dialogue to discuss the dimensions of the problem and how the country was responding to the HIV pandemic. From the discussion, it emerged that schools reached out to the majority of adolescents through the school curricula. The churches were also playing an important role in mitigating the impact of HIV among adolescents. It also emerged that there were also organization working with adolescents living with HIV to encourage adherence to medication and prevent re-infection. From these discussions, a grant proposal was written that would address HIV/AIDs prevention and design intervention for school-based adolescents, church-going adolescents, and adolescents living with HIV/AIDS.

Philosophical Underpinnings of Mixed Methods

69

This way of thinking about MMR ensures that research questions that are eventually addressed are a priority to the communities more so because while research questions may be relevant, they may not necessarily address realworld problem that are a priority to the communities. 4.3.3

Integrating Multiple Knowledge Systems

We formed two advisory boards: an internal and an external advisory board to bring together community and academic knowledge systems. We invited community members from the churches, schools, and organizations working on mitigating the impact of HIV among adolescents to serve as community theorists that could bring to the research the cultural knowledge, values, and processes of knowledge creation that could enhance our understanding of the problem and make the interventions we designed age and culturally relevant and useful to the adolescents. Their main role was to decolonize the research process by bringing to the production of knowledge Batswana worldviews and knowledge to ensure that the research remained relevant and useful to the Batswana. An external advisory board consisting of intellectuals with expertise on global knowledge in intervention research from the USA and Southern Africa was formed to serve the interest of the funders by bringing into the discussion the dimensions of rigor as understood by other interested parties. Members of the external advisory board served as peer reviewers who consistently asked us to identify and make explicit indigenous theoretical orientation, methods, cultural knowledge, and culturally specific findings throughout the study. 4.4

Postcolonial Indigenous MMR Method Theory

Various scholars define indigenous MMR, the motivation, purpose, and rationale for its application and practice at the level of method theory implementation, analysis, and reporting of the research findings. Scholars Botha (2011), Chatwood et  al. (2015), Chilisa (2019), and Chilisa and Tsheko (2014) emphasize the role of a paradigmatic lens and or philosophy in shaping the method theory. When qualitative methods are discussed within EuroWestern paradigms, the constructivist or interpretive paradigm inform how these methods can be applied. Similarly, a method theory in an indigenous MMR privileges indigenous knowledge systems and paradigms. It seeks to combine indigenous qualitative methods anchored in relationship building and connectedness of the people with each other and with the environment with conventional qualitative methods, indigenous qualitative methods with conventional quantitative methods or indigenous quantitative methods with conventional quantitative methods under an indigenous paradigmatic lens. Figure 4.1 illustrates the combination. Mixing can also take place in a ‘third

70

Bagele Chilisa

FIGURE 4.1

Mixing Methods

space’. Five designs are discussed: a parallel multi-epistemological approach, third-space MMR, concurrent designs mixing qualitative indigenous with conventional qualitative methods, sequential using qualitative conventional to inform quantitative research, and mixing conventional quantitative with indigenous qualitative. I start with a discussion of relationship building and connectedness in indigenous qualitative methods. 4.4.1

Indigenous Methods and Methodologies: Relationship Building and Connectedness

One unique characteristic of indigenous method theory is that the qualitative methods are anchored on relationship building and connectedness of people with each other and with the environment. Take for example the creative relationship framework (Parr, 2002), an indigenous methodology influenced by the Maori of New Zealand culture. In the creative relationship framework, building rapport through formation of relationships is

Philosophical Underpinnings of Mixed Methods

71

critical. Rapport conveys a state of harmony between people through relationship and connection (Parr, 2002). It requires mutual respect and trust, generosity, affection, and enjoyment of other’s company enhanced by faceto-face interaction, involvement of the community, and other interactions (Chilisa, 2019). Stories, songs, prayers, and rituals are meaningful sources of understanding the researched individually and collectively (Battiste, 2000) The American Indian Medicine wheel methodology uses the four seasons of the East symbolizing spring, the South symbolizing summer, the West, autumn and the north, winter to ground the methods in relationship building and connectedness with the environment. The East represents the spiritual aspects of life. In the East, researchers therefore acknowledge their interconnectedness with research participants and the wider community, integrating a wide range of senses in coming to know (Walker, 2001). The South represents the natural world and requires knowledge, understanding, appreciation, and utilization of indigenous peoples’ languages, metaphors, symbols, characters, stories, teachers, and teaching. The West represents the bodily aspects of knowing. Researchers go within themselves, discovering what is important between self, others, nature, and traditional teachings (Bopp et  al., 1989). The North represents the mental process of balancing intellect with wisdom. The Pagtatanung-Tanong interview method from indigenous Filipino culture, The Divine/client construction of a story from the Batswana of Southern Africa, the focused life-story relational interview from the Maori in New Zealand all demonstrate the centrality of relationship building and connectedness of people with each other and with the environment (Chilisa, 2019). In the Divine/client construction of a story interview method and the Pagtatanung–Tanong interview method, the time and place of the interview are essential for the quality of data. 4.4.2

A Multi-Epistemological Mixed Method Approach

A postcolonial indigenous MMR can take the form of parallel production of knowledge from an indigenous knowledge system approach and a Eurowestern knowledge system approach based on the assumption that both are rational and bring multiple perspectives and shed new light on the phenomenon of the study. In this approach, while the researchers may develop an integrated approach to study the same phenomenon, their interests, ways of perceiving reality, and ways of knowing consistently guide the goals of the study, data collection, analysis, and interpretation of research findings. In a research to study cancer healing systems, Berger-Gonzalez et al. (2016) describe a multi-epistemological research partnership, where indigenous Mayan medical specialists from Guatemala worked with Western physicians on equal basis. First, a Bidirectional Emic-Etic (BEE) was developed to promote researcher reflexivity, reduce power differential between knowledge

72

Bagele Chilisa

systems and promote knowledge integration. The BEE tool had five steps as follows: Step 1: It is the emic of self where each cultural group reflects on the variability of knowledge approaches to gain the essential features of the knowledge system that guide the study. Step 2: It is the ethics of the ‘other’ where two groups try to understand each other’s knowledge system. Step 3 Each group presents its understanding of each other’s knowledge system and highlights areas where integration of knowledge systems seems possible and where there is divergence. Step 4: It is etic of self where groups explore possible contradictions between mental constructions and actual practice. Step 5: This last step is a joint effort where both groups come up with an integrated research protocol to address the objective of the study. In this study, the Mayan researchers criticized the western researchers for their approach of treating the disease instead of the whole person with her or his social support system maintaining that it was also inhumane for hospitals to treat patients independent of their family support systems. Westerners in comparison questioned the Mayan diagnostic tools for diseases affecting internal organs (Berger-Gonzalez et al). The study demonstrates a mixed method approach based on an indigenous systems thinking of interconnectedness and relationality to inform stakeholders’ relationships with knowledge structures, each other, and the environment or ecosystem. The BEe instrument can serve as a powerful tool to address power asymmetries and build relationships among stakeholders and with the ecosystem. 4.4.3

Third-Space-Based MMR

Knowledge systems can also be combined in a ‘third space’. Discussing paradigms along binary opposites of Euro-Western and Postcolonial indigenous paradigms essentializes these paradigm, creating the impression that they are fixed and unchanging. Within Euro-Western and postcolonial indigenous paradigms practice, there is marginalization and muting of voices on the basis of gender, race, ethnicity, ableness, health, socio-economic status, sexual orientation, age, and so on. In the third space, the space in between (Bhabha, 1994), ‘all cultural statements and systems are constructed, therefore all hierarchical claims to inherent originality or ‘purity’ of cultures is unattainable’ (Bhabha, 1994, p.  54). The space in between is a tapestry, mosaic, of balanced borrowing of less hegemonic Euro-Western knowledge system and its democratic and social justice and combining with the best democratic, liberatory, and social justice essentialized indigenous knowledge

Philosophical Underpinnings of Mixed Methods

73

and subgroup knowledges (Chilisa, 2019). The third space is requires care in combining knowledge systems with the complexity of intersectionality in mind. It reminds researchers of the voices possibly silenced in each of the cultural paradigms that should be heard. 4.4.4

Concurrent Design: Indigenous Qualitative + Conventional Qualitative

Indigenous MMR at the method theory can entail a concurrent use of indigenous qualitative data collection tools and conventional qualitative data collection tools. The motivation and rationale for the mixing include initiation, expansion, development (Botha, 2011), nurturing relationships, broadening the literature base, and bringing into the research, the spiritual, the historical, the social, the physical, emotional, and ideological aspects (Chilisa  & Tsheko, 2014; Chatwood et al., 2015). In a multi-phase study to elicit adolescents’ and their parents’ beliefs, attitudes, and intentions towards sex, HIV prevention beliefs, HIV/AIDS intervention programmes, and preferred modes of educating adolescents on sexuality issues, proverbs, metaphors, stories, and myths were used as culturally appropriate methods of gathering data on sociocultural factors that influence adolescent local knowledge regarding HIV prevention strategies such as abstinence, condom use, limiting partners, and safe male circumcision (Chilisa & Tsheko, 2014) Conventional methods included structured interviews around the theory of planned behaviours. The indigenous methods brought indigenous literature through proverbs, new domains in understanding behaviours as well as cultural ideologies that discriminated against girls and women. 4.4.5

Sequential Design Qualitative + Quantitative

There is a concern that current statistical analysis is based on narrow aspects of indigenous peoples’ daily lives (Walter  & Andersen, 2013). Most global survey instruments and tests measuring attitudes, behaviours, and knowledge attributes are informed by dominant Euro-Western ways of knowing. There is thus a need to broaden concepts and constructs in survey instruments to include marginalized knowledge systems of the formerly colonized and oppressed. In the aforementioned study, in the second phase, the focus was on designing a culturally relevant survey instrument and to use it to quantitatively measure behaviours, beliefs, and attitudes of adolescents towards sex, abstinence, condom use, circumcision, and HIV. In this phase qualitative data collected in the first phase informed the design of a quantitative survey instrument to measure the prevalence of risky behaviours. The survey questionnaire items were built from qualitative data based on the Theory of Planned Behaviours collected using structured interviews and data derived from cultural

74

Bagele Chilisa

knowledge that came through stories, myths, proverbs songs, metaphors, and local language. The use of songs, taboos, and myths to source parents’ and their children’s views on sex and sexuality brought into the discussion concepts and constructs not common in the literature (Chilisa & Tsheko, 2014). 4.4.6

Conventional Quantitative + Indigenous Qualitative Methods

In this approach, conventional quantitative research findings are expressed within the framework of an indigenous paradigm. The mixing of conventional quantitative research happens at the presentation of quantitative research findings and Blackstock (2009) calls it enveloping quantitative research in an indigenous envelope. The approach constitutes a new way of mixing conventional quantitative research with indigenous research. The process of enveloping quantitative research in an indigenous framework starts with researchers’ understanding of the research goal, building respectful relations with the indigenous communities, developing the research question and the methodology, interpretation and dissemination strategies in partnership with the community, and observing ethical protocols that honour indigenous knowledge and are culturally responsive (Blackstock, 2009). Blackstock (2009) demonstrates this approach on The Canadian Incidence Study on Reported Child Abuse and Neglect. The goal of the mixing was to evidence community reality that does not deny the influence of a relational worldview, emotions, or spirituality. The Medicine Wheel was used to present a holistic worldview that seeks balance between the spiritual, the cognitive, the physical, and the emotional. The researchers invoked the spiritual in the presentation and dissemination of the findings by employing symbolic art, poetry, legends, and teachings to add meaning to the findings. In the study, the physical was honoured by printing the report on an ‘ecologically friendly ink and on paper that protected old growth forests’. Thus, according to Blackstock (2009), the report demonstrated value, respect, and relational existence of people with the environment. In the cognitive domain, ancestral knowledge is valued, and whose language is used to communicate the findings is essential in bringing the spiritual, the physical, the emotional, and cognitive together. The definitions of mixed methods research, rationale, and motivation are summarized in Table 4.2. 4.4.7

Conducting a Postcolonial Indigenous Mixed Method Design

The motivation and rationale for a post-colonial indigenous MMR should be made explicit at five phases in the research study: the planning stage, the literature review, method theory, ethics, and analysis, reporting, and dissemination of results. Table 4.3 presents some guiding questions.

TABLE 4.2 Definitions of Mixed Methods Research, Rationale, and Motivation

Author

Definition of Mixed Methods Research

Focus

Rationale

Chatwood et al. (2015)

Chatwood et al. (2015) propose a definition of MMR that considers methodologies of combining Western and indigenous knowledge as distinct paradigms in indigenous research. When the spiritual, the emotional, the physical, and the cognitive are brought together with a western quantitative approach, Cindy calls the approach Enveloping quantitative research in an indigenous envelope and does not use the term mixed methods Qualitative methods with qualitative data emanating from an indigenous paradigmatic lens in a single study or multiple phases. The indigenous mixed methods also takes the form of combining quantitative, and qualitative methods and indigenous research frameworks in a single or multiphase study,

Paradigm focuses and combines Western and indigenous knowledge as distinct paradigms Philosophical perspective that integrates quantitative research with an indigenous worldview

To create space for indigenous knowledge, epistemologies, and values in the research process to enhance respect and equality of all knowledge systems

Paradigm and methods focus

• To bring to the research process indigenous tools that build and nurture relationships • To invoke indigenous knowledge to inform ways in which concepts and new theoretical frameworks for research studies are defined • To broaden the literature base, we depend not only on written texts but also on the largely unwritten texts of the formerly colonized and historically disadvantaged people • To bring to the centre of the entire research process the spiritual, historical, social, and the ideological aspect of the research phenomena

Blackstock (2009)

75

(Continued)

Philosophical Underpinnings of Mixed Methods

Chilisa and Tsheko (2014)

To contextualize research and provide more knowledge pathways in the form of the physical, the spiritual, and the emotional.

Definition of Mixed Methods Research

Botha (2011)

Consider as mixed methods combining conventional qualitative research with indigenous research. The purpose of mixing is to draw on the interaction of these methods to clarify the relationship between Western and indigenous ways of knowing so that more appropriate theories, practice, and relations can be developed for their interrelation (p. 314) Mixed methods as mixing indigenous and non-indigenous paradigms or methods.[

Jon and Mozley, 2012

Focus

Rationale 1. Initiation (to initiate new ways of grounding methodologies at the local level) 2. Development (to develop new theories, values, and practices that inform indigenous research) 3. Expansion (to decolonize the areas of collaboration between indigenous and Western modes of qualitative research, reveal new perspectives, and expand the boundaries of qualitative ways of knowing)

Paradigm and methods

Bagele Chilisa

Author

76

TABLE 4.2 (Continued)

TABLE 4.3 Thinking With an Indigenous Systems Thinking Approach in MMR

Phase 1: Mapping research environment structures and building relationships

What are the structures in the research environment that can possible transmit the ideologies of the powerful and perpetuate dominance of some groups? What are the strategies for building relationship to promote interaction of all knowledge systems? How will the research build relationships to address academic imperialism and funder colonialism to allow knowledge systems to interact on equal basis? What are the knowledge systems to be combined? What constitutes literature and how will oral and written text literature be combined? What analytical tools will help to critique the literature and build bridges between the knowledge systems? Will the research take a stance against Western archival knowledge and its colonizing and Othering ideologies and highlight indigenous knowledge system that speak differently? Will the research challenge literature that perpetuates victim-blaming and deficit-based theories and analysis and bring alternatives from indigenous knowledge systems? How will the study use a balanced borrowing from a Euro-Western paradigm and indigenous paradigm? How will the method theory be conceptualized? Is it a postcolonial indigenous qualitative combined with mainstream qualitative research, or a postcolonial indigenous combined with a mainstream quantitative? How will the knowledge systems be combined to inform the choice of the research topic, the research questions asked, the data collections tools, and interpretation and dissemination of results? What indigenous data collection tools give voice to the researched? What methods balance power between the researcher and the researched and the knowledge systems? What methods will the study use to accurately generate and record marginalized voices and indigenous and local knowledge that has been predominantly excluded through Euro-Western conventional methodologies? How will the study invoke the histories, world views, and indigenous knowledge systems to imagine and suggest new methodologies? Will the study in any way restore and revitalize indigenous knowledge? How will the research process based on relational accountability, respectful representation, reciprocal appropriation, rights and regulation responsivity, relationality, reverence, responsibility, reflexivity, responsiveness, and decolonization ethical principles be combined with mainstream research ethics? In what language will the findings be communicated? How will indigenous reporting methods be combined with mainstream methods to communicate findings to diverse settings and communities?

Phase 2: Literature

Phase 3: Method Theory

Phase 4: What ethical principles guide the study? Phase 5: Dissemination of Results: Combining Mainstream and indigenous reporting strategies

77

Guiding Questions

Philosophical Underpinnings of Mixed Methods

Phase

78

4.5

Bagele Chilisa

Conclusion

Most of the discussions on MMR focus on methodologies or methods of gathering data for triangulation, complementarity, development, initiation, or expansion purposes. In the post-positivist paradigm, use of multiple or mixed methods is done to compare results against other observations and results, and the approach remains predominantly quantitative. In the constructivist paradigm, research is guided by the ontological assumption that perceives reality as multiple and an epistemological assumption where knowledge is perceived as subjective. The methods are predominantly qualitative. In the transformative paradigm, the methods are not as important as how they are used to advocate for change. In the pragmatist paradigm, qualitative and quantitative data is mainly used for the broad purposes of breadth and depth of understanding and corroboration (Chilisa, 2019). In comparison, a postcolonial indigenous MMR combines perspectives from an indigenous paradigm with non-indigenous perspectives. The MMR approach combines indigenous quantitative and qualitative methods and methodologies with non-indigenous methods and methodologies under an indigenous paradigm. It privileges indigenous knowledge as the dominant paradigmatic lens that informs the mixing. It provides knowledge pathways in the form of the physical, the emotional, the cognitive, and the spiritual that appeal to the ways of knowing of the researched indigenous communities when indigenous qualitative and quantitative methods are allowed to mix with mainstream quantitative and qualitative methodologies. The motivation and rationale for mixing are summarized as follows: 1. Creating space for indigenous knowledge, epistemologies, and values in the research process to enhance respect and equality of all knowledge systems and adopt a more holistic approach to rigor in research inquiry. 2. Contextualizing research and bringing to the centre of the entire research process the physical, the spiritual, the emotional, historical, social, and the ideological aspect of the research phenomena for more rigorous, credible, and inclusive research inquiry. 3. Bringing to the research process indigenous tools that build and nurture relationships and connectedness of the people with each other and with the environment to ground research at local level. 4. Invoke indigenous knowledge to inform ways in which concepts and new theoretical frameworks for research studies are defined to expand the boundaries of MMR ways of knowing and to ensure rigor and credibility in research inquiry. 5. To broaden the literature base, we depend not only on written texts but also on the largely unwritten texts of the formerly colonized and historically disadvantaged people so that no knowledge is left behind that can contribute to the completeness of research inquiry.

Philosophical Underpinnings of Mixed Methods

79

In this chapter, the conceptual framework that informs the planning, the MMR designs, and method theory are discussed. It is noted that, in the postcolonial indigenous MMR, researchers start by interrogating structures of knowledge production and the instruments that inform the knowledge production process guided by an indigenous systems thinking approach. Relationality and interconnectedness, a form of indigenous system’s thinking approach, guide the research process from its conceptualization, data collection, analysis, and interpretation of results to dissemination of research findings. A postcolonial lens and discourses on scientific colonialization, academic imperialism, resistance to epistemic violence and dominance of Euro-Western paradigms, and a decolonization intent equip the researcher with the tools to disrupt the knowledge and power inequalities and create pathways for planning and conducting research that create space for all knowledges, give voice to all stakeholders and provide opportunity for equitable and sustainable futures. It has been shown that research can exacerbate inequalities as when pharmaceutical companies steal knowledge from indigenous communities (Commey, 2003) or cause psychological harm as when researchers extract human body samples and store them in data banks in the developed world science laboratories, or epistemic violence as when indigenous knowledge systems are marginalized and dismissed as trivial. From a postcolonial indigenous paradigm MMR, such research may be reliable but not valid. A postcolonial indigenous MMR offers multiple designs. The third-space designs allow researchers to borrow the best theoretical lens and tools from the Euro-Western and postcolonial indigenous paradigms to create a rebalancing of power across all sections of the society and across all knowledge systems. A multi-epistemological MMR design allows knowledge systems and the resultant research findings to coexist while at the same time fostering multi-directional sharing of ideas. Other designs privilege indigenous philosophies, values, and methods that inform strategies to address knowledge and power asymmetries, build relationships contribute to new concepts, theoretical frameworks, data collection tools analysis and knowledge dissemination strategies resulting in a holistic approach to rigor in research inquiry. The methods and methodologies are built on principles of relationship and connectedness and invoke spirituality, the emotional, the physical, and the cognitive. References Battiste, M. (2000). Reclaiming Indigenous Voice and Vision. Vancouver, Canada: UBC Press. Benton-Banai, E. (1988). The Mishomis Book: The Voice of the Ojibway. Hayward, WI: Indian Country Communications. Berger-Gonzalez, M., Stauffacher, M., Zinsstag, J., Edwards, P.,  & Krutli, P. (2016). Trans-disciplinary research on cancer-healing systems between biomedicine and

80

Bagele Chilisa

the Maya of Guatemala: A tool for reciprocal reflexivity in a multi-epistemological setting. Qualitative Health Research, 26(1), 77–91. Bhabha, H. (1994). The Location of Culture. New York, NY: Routledge. Billman, J., & Chilisa, B. (in press). The power and politics of knowledge production in program evaluation: Funder, methodological, and pedagogical colonialism. In Lori Wingate (Ed.), Core Concepts in Evaluation: Contemporary Commentary on Classic Writings. Thousand Oaks, CA: Sage Publications. Blackstock, C. (2009). First nations children count: Enveloping quantitative research in indigenous envelope. First Peoples Child & Family Review, 4(2), 135–143. Bopp, J., Bopp, M., Brown, L., & Lane, P., Jr. (1989). The Sacred Tree. Twin Lakes, WI: Lotus Light. Botha, L. (2011). Mixed methods as a process towards indigenous methodologies. International Journal of Social Research Methodology, 14(4), 313–325. doi:10.108 0/13645579.2010.516644. Briggs, J. (2013). Indigenous knowledge: A  false dawn for development theory and practice? Progress in Development, 12(3), 231–243. Chatwood, S., Paulette, F., Baker, R., Eriksen, A., Hansen, K. L., Eriksen, H., Hiratsuka, V., Lavoie, J., Lou, W., Mauro, I., Orbinski, J., Pabrum, N., Retallack, H., & Brown, A. (2015). Approaching Etuaptmunk-Introducing a consensus based mixed method for health services research. International Journal of Circumpolar Health, 74, 1, DOI: 10.3402/ijch.v74.27438. Chilisa, B. (2006). Decolonising ethics in social science research: Towards a framework for research ethics. In A. Rwomire (Ed.), Challenges and Responsibilities of Doing Social Research in Botswana: Ethical Issues (pp.  199–207). Nairobi: OSSREA. Chilisa, B. (2017). Decolonising trans-disciplinary research: An African perspective for enhancing knowledge integration in sustainability science. Sustainability Science. 12(5), 813–827. Chilisa, B. (2019). Indigenous Research Methodologies. London: Sage. Chilisa, B.,  & Mertens, D. M. (2021). Indigenous made in Africa evaluation frameworks: Addressing epistemic violence and contributing to social transformation. American Journal of Evaluation, 42(2), 241–253. Chilisa, B., & Phatshwane, K. (2022). Qualitative research within a postcolonial indigenous paradigm. The SAGE Handbook of Qualitative Research Design, 225. Chilisa, B.,  & Tsheko, G. N. (2014). Mixed methods in indigenous research: Building relationships for sustainable intervention outcomes. Journal of Mixed Methods Research, 8(3), 222–233. Commey, P. (2003). New scramble for Africa. The New African, December. www. africasia.com/newafrican/na.php? Cram, F., & Mertens D. M. (2015). Transformative and indigenous framework for multimethod and mixed methods research. In Sharlene Nagy Hesse-Biber, and R. Burke Johnson (Eds.), The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry (pp. 91–109). Oxford: Oxford University Press. CREA-HI. (2019). Evaluation with Aloha: A framework for working in Native Hawaiian contexts. https://www.creahawaii.com/aloha Deloria, V. (1995). Red Earth, White Lies: Native Americans and the Myth of Scientific Fact. New York, NY: Scribner. Ellis, J. B.,  & Earley, M. A. (2006). Reciprocity and constructions of informed consent: Researching with indigenous populations. International Journal of Qualitative Methods, 5(4), 1–13.

Philosophical Underpinnings of Mixed Methods

81

Getty, G. A. (2010). The journey between western and indigenous research paradigms. Journal of Transcultural Nursing, 211, 5–14. Goduka, I. N. (2000). African or indigenous philosophies: Legitimizing spiritually centered wisdoms within the academy. In P. Higgs, N. C. G. Vakalisa, T. V. Mda, & N. T. Assie-Lumumba (Eds.), African Voices in Education (pp. 63–83). Lansdowne, South Africa: Juta. Grenier, L. (1998). Working with Indigenous Knowledge: A  Guide for Researchers. Ottawa, Canada: International Development Research Centre. Held, M. B. (2019). Decolonizing research paradigms in the context of settler colonialism: An unsettling, mutual, and collaborative effort. International Journal of Qualitative Methods, 18, 1–16. Held, M. B. (2023). Undoing the colonial and racist hegemony of Western science. Journal of Multidisciplinary Evaluation, 19(44), 88–101. Kaphagawani, D. N., & Malherbe, J. G. (2000). African epistemology. In P. H. Coetzee  & A. P. J. Roux (Eds.), Philosophy from Africa (pp.  205–216). Oxford, UK: Oxford University Press. Koloi-Keaikitse, S., Geller, G., & Ali, J. (2021). Cultural values and beliefs of selected local communities in Botswana: Implications for human subjects research ethics. Journal of Empirical Research on Human Research Ethics, 16(4). https://doi. org/10.1177/15562646211023332 Levac, L., Mcmurtry, L., Steinstra, D., Baikie, G., Hanson, C., & Mucina, D. (2018). Learning Across Indigenous and Western Knowledge Systems and Intersectionality: Reconciling Social Research Approaches (Technical Report). University of Guelph, Social Science and Humanities Research Council of Canada, Guelph, Canada. Mertens, D. M. (2010). Transformative mixed methods research. Qualitative Inquiry, 16(6), 469–474. Onyewumi, O. (1998). Deconfounding gender: Feminist theorizing and Western culture. Signs: Journal of Women in Culture and Society, 23(4), 1049–1062. Parr, R. (2002). Te Matahauariki Methodology: The Creative Relationship Framework (Te Matahauariki Institute Occasional Paper Series). Hamilton, New Zealand: University of Waikato. Pelletier, S. R. (2003). Indigenous research in social work: The challenge of operationalizing worldview. Native Social Work Journal, 5, 117–139. Pryor, J., Kuupole, A., Kutor, N., Dunne, M., & Adu-Yeboah, C. (2009). Exploring the fault lines of cross-cultural collaborative research. Compare, 39(6), 769–782. Romm, N. R. A. (2015). Reviewing the transformative paradigm: A critical systemic and relational (Indigenous) lens. Systematic Practice and Action Research, 28, 411–427. Schumaker, L. (2001). Fieldwork, Networks, and the Making of Cultural Knowledge in Central Africa. Durham and London: Duke University Press. Sefa Dei, G. J. S. (2002). African development: The relevance and implications of “indigenousness.” In G. J. Sefa Dei, B. L. Hall, & D. G. Rosenberg (Eds.), Indigenous Knowledges in Global Contexts: Multiple Readings of Our World (pp. 70–86). Toronto, Canada: University of Toronto Press. Smith, T. (1999/2012). Decolonizing Methodologies: Research and Indigenous People. London, UK: Zed Books. Walker, P. (2001). Journeys around the medicine wheel: A story of indigenous research in a Western University. The Australian Journal of Indigenous Education, 29(2), 18–21.

82

Bagele Chilisa

Walter, M., & Andersen, C. (2013). Indigenous Statistics: Quantitative Research Methodology. Walnut Creek, CA: Left Coast Press. Wilson, S. (2008). Research Is Ceremony: Indigenous Research Methods. Manitoba, Canada: Fernwood. World Bank. (1998). Indigenous Knowledge Systems in Sub-Saharan Africa: An Overview. Africa: IK Notes.

5 THE DIALECTIC STANCE Navigating Difference Jori N. Hall

5.1

Introduction

As has been noted in the mixed methods literature, ‘a major purpose of mixed methods theory is to help practitioners make the mixing of methods more intentional, multilevel, and thoughtful and thus yield more compelling results’ (Greene & Hall, 2010, p. 120). Yet, while mixed methods research has gained popularity, there remains a lack of understanding about the theories used to guide the integration of qualitative and quantitative methods. In addition to facilitating more intentional, comprehensive, and compelling results, mixed methods theory plays an important role in defending how and for what reason different methods in a single study are mixed. Given the importance of theory to guide mixed methods practice, this chapter contributes clarification on one of the mixed methods theories that – despite its extensive articulation in the mixed methods literature (Cronenberg & Headley, 2019; Greene et al., 2001; Greene, 2005, 2007; Greene & Caracelli, 1997; Greene & Hall, 2010; Johnson, 2008, 2017) – remains elusive in its conceptualisation and implementation: the dialectical stance. To do so, this chapter provides some background on the theoretical foundations of the dialectic stance, offers some conceptual clarification on the dialectical stance, and presents an example of how the dialectic stance has been implemented in mixed methods practice. To conclude, the practical implications or lessons learned from applying the dialectic stance are discussed. 5.2

The Philosophical Roots and Meanings of Dialectics

Despite agreement on the Greek roots of the term dialectic, its meaning is admittedly unstable. For instance, Duncombe and Novaes (2016) point out DOI: 10.4324/9781003273288-6

84

Jori N. Hall

that, whereas the English version of dialectic is expressed commonly as a noun, the Greek version, įȚĮȜİࣀIJȚࣀȒ, is more like an adjective. As such, it communicates a capacity or skill: ‘the dialectical art’ (Duncombe & Dutilh Novaes, 2016). This version of the term relates to the more common understanding of dialectic, which refers to a process or debate style in which a premise or a taken-for-granted idea is challenged or contested in search of truth. This understanding of dialectic is typically associated with the debate style of Socrates (407 BCE–399 BCE) and Plato (428 BCE–348 BCE). A more modern contribution to the meaning of the term dialectics comes from the German philosopher, Georg Wilhelm Friedrich Hegel (1770–1831). While scholars have debated the nature of Hegel’s dialectic (Mueller, 1958), it is generally considered similar to the debate style of Socrates and Plato used to arrive at truth whereby a thesis or an idea is presented and challenged by an opposing idea. However, a distinguishing feature of Hegel’s debate style is noticed in the desired outcome. That is, as a result of a Hegelian dialectic exchange (see Hegel’s Science of Logic), the tensions between the conflicting perspectives or ideas are reconciled, producing a new idea or sublation. Implicit in Hegel’s dialectic is the philosophic assumption that dialectic interplay leads to the development or evolution of ideas. It is important to point out a philosophical underpinning of Hegel’s notion of dialectics. For Hegel, thoughts or ideas and the world are interrelated; that is, they are the same. Informed by Hegel’s dialectic philosophy, Karl Marx (1818–1883) and Friedrich Engels (1820–1895) offer another view on dialectics that emphasises the material world. In contrast to Hegel’s philosophical perspective, the materialist view on dialectics does not suggest that thoughts and the real world are the same or part of one whole nor is the focus on the reconciliation of ideas or processes. Rather, materialist thinkers distinguish thoughts from material reality. A key assumption, then, is that the material world has an objective reality separate from the mind or one’s perception of the world. From this perspective, dialectical materialism primarily concerns processes in the world, the contradictions inherent in those processes, and how those processes change or evolve. A  key difference from Hegel’s dialectic is that the dialectical materialist thinkers are less optimistic in the sense that their stance suggests that conflicting ideas and world processes are not always reconciled or resolved. Whereas Hegel’s dialectic focused on contradictions in rational thought and how their resolutions guide the world’s evolution, dialectical materialist thinkers focus on the conflicts across different classes of people and resulting processes. This less optimistic notion of dialectics can be considered a reflection of what was happening in the world in the 1840s. At that time, many industrial changes were taking place in countries such as Germany. As changes in society were taking place, concerns were rising

The Dialectic Stance

85

about societal issues such as poverty. In response, Marx and Engels advanced a dialectic stance that emphasised changing the social systems that led to these issues – or a revolutionary movement. The meaning of materialist dialectics, like the other versions of dialects before it, remains contested. As a result, the conceptualisation of dialectic is a bit unstable. However, there are common themes that cut across the different conceptualisations of dialectics. First, the varying conceptualisations call into question the idea that the world (physical, social) will remain unchanged; the development of the world is assumed and accepted. Second, dialectics is a way to deal with change, which inevitably involves contradiction or conflict. Dialectics recognise that conflict is inherent in the world. In this way, dialectics is both a starting point and a means to address conflict. Reviewing these themes across definitions makes the contribution of the dialectic tradition to critical theory visible. 5.3

Dialectics’ Contribution to Critical Theory

Dialectics can be found in critical theorists’ perspectives on social relations. Critical theorists posit that society or the world is rife with conflicts between different social classes. As a result, in the 1930s, for instance, critical theorists attempted to create a stance that incorporated the social sciences and philosophy. This stance or ‘dialectical methodology’ aimed to communicate the relationships between the individual, the economy, and society (Kellner, 1993). Dialectics for the critical theorists at that time ‘was the art of making connections and discerning contradictions which opened the space for thought and action in the oppressively closed totalitarian universes of fascism, Stalinism and, as they saw it, the totally administered societies of corporate capitalism’ (Kellner, 1993, p.  47). By seeking to bring methodology and philosophy together, they aimed to ‘unite theory and practice (whether they actually succeeded in doing this is another story)’ (Kellner, 1993, p. 47). The influence of the dialectic tradition on critical theory can also be seen in the text, Dialectic of Enlightenment. Written by Frankfurt philosophers Max Horkheimer and Theodor W. Adorno in 1944 (translated by John Cumming in 1972), the text argues that Enlightenment1 – emphasising rationality and objective scientific thought – fails to consider the impact of its own development on society. The text is ultimately a critique of Enlightenment itself. Horkheimer and Adorno posit that positivistic thinking, focused on classification and calculation, dominates sectors such as scientific thinking, philosophy, commerce, and politics. This dominance has granted both positive and negative consequences for society. While it provides technological and economic advancements, it also subjects individuals to the technical mechanisms that led to these advancements; as a result, the Enlightenment movement supports

86

Jori N. Hall

social hierarchies or inequalities. Horkheimer and Adorno articulate the paradoxical nature of Enlightenment this way: On the one hand the growth of economic productivity furnishes the conditions for a world of greater justice; on the other hand, it allows the technical apparatus and the social groups which administer it a disproportionate superiority to the rest of the population. (p. XIV) Consequently, individuals are simultaneously served by these scientific mechanisms and ‘wholly devalued’ by them (p. XIV). According to Horkheimer and Adorno, the paradoxical nature of Enlightenment is supported by historical evidence, revealing how Enlightenment is a ‘myth’ in the sense that it does not live up to its own ideals (i.e. liberty, progress). In Dialectic Enlightenment, they suggest that the relationship between the Enlightenment movement and mythology is itself a dialectical one. That is, while the Enlightenment positions itself as superior to mythology, it functions as mythology – an instrumental tool to guide social relations and processes. Horkheimer and Adorno (1944) also make the point that the Enlightenment movement and mythology are similar in the sense that they both have origins as responses to fear of what is different or unknown: ‘The dualization of nature as appearance and sequence, effort and power, which first makes possible both myth and science, originates in human fear, the expression of which becomes an explanation’ (Horkheimer & Adorno, 1944, p. 15). Whereas mythology handles difference by mimicking it in an attempt to remove difference or identify with it, Enlightenment reacts to the fear of difference by categorising and monitoring it and trying to control it (Geuss, 2016). Horkheimer and Adorno’s view on Enlightenment is often considered pessimistic given its emphasis on how the Enlightenment movement can function to control social processes, thereby creating and maintaining social inequalities. These and other critiques of the Enlightenment movement outlined in Dialectic of Enlightenment make the text a philosophical foundation for critical theory. 5.4

Dialectics as a Paradigm for Mixed Methods Social Science Enquiry

By the late 20th century, social science researchers were increasingly critiquing the dominance of positivism. One critique was that positivism restricts social science research to quantitative methods and methodologies. The social and political movements (i.e. Civil Rights, Feminism) of the time also drove the desire for alternative methods that give more attention to personal narratives – especially from those who are historically minoritised. Because qualitative and quantitative methods are often characterised as promoting different

The Dialectic Stance

87

paradigms or assumptions about the world and how to know it, they were advanced as the main options for social science enquiry. This meant social science research was commonly framed as either qualitatively or quantitatively oriented. Because of this, debates known as the paradigm wars began to occur. One argument, known as the purist view, suggests that paradigmatic stances are incommensurable. Therefore, different methods (e.g. quantitative and qualitative) cannot be combined in a signal study and should be either qualitatively or quantitatively oriented. Another argument suggests that different paradigms and enquiry methods can be combined in a signal study: the dialectic stance. The dialectic stance has gained popularity within fields such as mixed methods and evaluation. Social science scholars in these fields tend to hold a critical view of science. They argue that ‘using multiple and diverse methods is a good idea, but is not automatically good science’ (Geuss, 2016). In their view, good science requires a philosophical foundation to guide methodological decisions. In line with the dialectic tradition, the generative tensions associated with using different methods and paradigms in a single study serve as a starting point for their critique of the purist paradigmatic perspective and guide their defence of the incorporation of different paradigms and methods in a single study. Mixed methods is defined as ‘research in which the investigator collects and analyses data, integrates the findings, and draws inferences using both qualitative and quantitative approaches or methods in a single study or program of inquiry’ (Tashakkori  & Creswell, 2007, p.  4). The dialectic stance, in the context of mixed methods, is defined as a philosophical perspective that ‘actively welcomes more than one paradigmatic tradition, along with more than one methodology and type of method, into the same inquiry space and engages them in respectful dialogue with one another’ (Greene & Hall, 2010, p. 124). The following assumptions guide this stance. First, ‘difference between philosophical paradigms or logics of justification for social scientific inquiry not only exist but are important’ (Greene  & Caracelli, 1997, p.  8). Second, all paradigms represent a legitimate but partial way to understand the world. Third, differences in philosophical traditions in a mixed methods study cannot be reconciled; rather, they must be respectfully honoured to preserve their integrity (Greene & Hall, 2010; Kidder & Fine, 1987). Fourth, dialectical engagement in a mixed methods study between different paradigms, methodologies, and methods can result in new knowledge or more comprehensive understandings. Greene and Hall (2010) note the dialectic stance ‘affords a meaningful engagement with difference, an engagement intended to be fundamentally generative of insight and understanding that are of conceptual and practical consequence’ (p.  124). The emphasis on engaging difference here is noteworthy as it relates to how the Enlightenment movement and mythology are

88

Jori N. Hall

responses to the fear of difference. Broadly speaking, like Horkheimer and Adorno (1944), Greene and Hall (2010) acknowledge that difference plays a major role in science. Yet Greene and Hall discuss this role differently. For Greene and Hall (2010), difference is discussed not in relation to fear, but, in terms of the value difference has to generate an enhanced understanding of the phenomena of interest. Greene’s (2007) articulation of a mixed methods way of thinking provides additional insights on the value of difference for the mixed methods dialectic stance. 5.5

The Difference ‘Difference’ Makes in the Mixed Methods Dialectic Stance

A mixed methods way of thinking is an orientation towards social science enquiry broadly speaking and the mixed method dialectic stance specifically (Greene, 2007). This orientation values different ways of viewing the world and recognises the inherent complexity of social phenomena (Hall & Copple, 2023). As such, multiple ways of knowing or different paradigms are invited to engage the complexities of social phenomena. Another important aspect of this orientation is that it is not solely focused on generating answers or convergence as a result of mixed methods enquiry. Equally and perhaps even more important are the divergent findings, questions, or incongruities produced as a result of the enquiry process (Creamer & Edwards, 2019). A dialectic mixed methods stance, then, should not be equated with convergence or consonance, as in triangulation. Certainly, triangulation is a worthy mixed methods (and multiple methods) purpose, as congruent results from more than one method afford great confidence in the inferences made. (Greene & Hall, 2010) The point here is that, while mixed methods enquiry results that converge have merit, the dialectic stance values results that diverge as they offer opportunities to generate new insights, interrogate taken-for-granted understandings, gain more clarity from further analysis, and remain open to surprises. Another aspect of a mixed methods way of thinking that is epitomised by the dialectic stance is its value commitment to difference. Greene (2007) outlines how the value commitment to difference makes a difference to the dialectic mixed methods. First, difference refers to the various values embraced by different paradigms (Greene, 2007). For example, the values of the dialectic stance include diversity, dialogue, and acceptance, whereas pragmatism includes a ‘commitment to values of democracy, freedom, equality, and progress’ (Greene & Hall, 2010).

The Dialectic Stance

89

Second, difference relates to different methodologies (i.e. case study) or methods (i.e. focus groups). Importantly, various methodologies and methods are selected in alignment with the paradigmatic stances represented in the mixed methods study and have consequences for the phenomena of interest. Third, difference concerns the human diversity represented in the mixed methods enquiry context. This can include traditional markers of diversity (i.e. race, ability, religion), non-traditional markers of diversity (i.e. hip-hop culture) (Greene, 2007), and the members of an interdisciplinary mixed methods team (Creamer & Edwards, 2019). Greene (2007) further contends that as a result of using more than one paradigm, the results of the mixed methods study can be made stronger, yielding a more complete understanding of the phenomena of interest. This view is akin to Hegel’s notion of dialectics in the sense that it suggests different ideas about reality or knowledge can generate robust findings. A mixed methods way of thinking also suggests that difference in paradigms, methodologies, methods, and human diversity leads to tensions in the context of a mixed methods study. Creamer and Edwards (2019) discuss how these tensions can be productive. Finally, a mixed methods way of thinking has important implications for the dialectal stance. These implications are made more visible when comparing the dialectic stance to another stance. Greene and Hall (2010) compared the dialectic stance to pragmatism – another paradigm that mixed methods researchers commonly use to guide their work. To guide the comparison between the dialectic stance and pragmatism, Greene and Hall (2010) used various aspects of enquiry. For example, one aspect concerned the frameworks used to guide enquiry. A  mixed methods way of thinking implies that the dialectic stance advances multiple theories or frameworks to guide enquiry projects. In contrast, the pragmatic stance is driven by concerns or problems important to the context – not necessarily paradigms or theories. Another aspect included inferences. A dialectic stance implies inferences (or findings) are based on both integrating data sets and seeking areas of divergence. In comparison, the pragmatic stance suggests that inferences need to produce actional information – or knowledge that is useful to address the issue or concern at the centre of the enquiry. When considering the contributions of the enquiry, a mixed methods way of thinking implies the dialectic stance is concerned with how difference (i.e. human, philosophical, and methodological) was engaged and the consequences associated with that engagement. Alternatively, the pragmatic stance is concerned with contributing solutions to the problems at hand and the consequences of the same. For more details on the implications of dialectic stance and pragmatism in mixed methods practice, see Greene and Hall (2010). My discussion now turns to applying the dialectic stance in mixed methods practice.

90

5.6

Jori N. Hall

Applying the Dialectic Stance in Mixed Methods Social Science

Mixed methods scholars remark on the dearth of empirical guidance on how to apply the dialectic stance (Betzner, 2008; Creamer & Edwards, 2019). Notwithstanding, this lack of guidance makes room for enquirers to be innovative and creative when employing the dialectic stance and interrogating dissonance. In this section, I offer some considerations to keep in mind when applying the dialectic stance in mixed methods practice based on the methodological literature. One key consideration is the type of phenomenon to be investigated. Betzner (2008) conducted an empirical study, comparing the dialectical stance to pragmatism. Based on the empirical investigation, Betzner concluded that ‘overall, the dialectic method appears more suitable to exploring more complex phenomenon as compared to the pragmatic approach’ (Betzner, 2008, p. iv). Another consideration is the purpose of the mixed methods study. Based on their empirical investigation of mixed methods evaluation studies, Greene et al. (1989) outline four purposes for conducting mixed methods: complementarity, development, triangulation, and initiation. The initiation purpose is most in alignment with the dialectic stance as it ‘seeks the discovery of paradox and contradiction, new perspectives of frameworks, the recasting of questions or results from one method with questions or results from the other method’ (Greene et al., p. 259). Design is yet another consideration. While there is no prescribed design for applying the dialectic stance in practice, some scholars consider designs that give equal weight to the qualitative and quantitative components of a mixed methods design most appropriate for the dialectic stance as one component does not dominate the other (Johnson, 2017). Additionally, researchers need to consider integration or where the ‘mixing’ occurs in the mixed methods design. According to Fetters and Molina-Azorin (2017), mixing can occur in at least 15 enquiry dimensions: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12.

Philosophical Theoretical Investigator Team Literature review Rationale Study purpose Research design Sampling Data collection Data analysis Interpretation

The Dialectic Stance

91

13. Rhetorical 14. Dissemination 15. Research integrity While a detailed description of how mixing can occur within each mixed methods dimension is beyond the scope of the current chapter (for more details, see Fetters & Molina-Azorin, 2017), it is worthwhile to point out that although the dialectic stance emphasises mixing at the philosophical level, dialectic mixed methods designs include mixing in other enquiry dimensions. Furthermore, applying the dialectic stance requires a learner orientation. Cronenberg (2020) put it this way: ‘the researcher must interpret the opposing viewpoints as an opportunity to learn, deepen understanding, and potentially challenge the discipline with new insights’ (p. 32). Building on Greene and Hall’s (2010) thinking about the dialectic stance, Cronenberg (2020) operationalises the learner orientation by specifying four characteristics that are foundational when applying the dialectic stance in mixed practice. In what follows, I outline these characteristics as I understand them. The first characteristic involves the researcher seeking and sustaining dialogue among the data collected guided by the paradigms utilised in the mixed methods study. Cronenberg (2020) notes that if the study is conducted by an individual researcher, then the dialogue across the data collected can be maintained via the researcher’s journal or memos. If the dialectic mixed methods study is conducted by a team, the dialogue can be maintained by team conversations and written documents. The second characteristic requires the researcher to give equal voice to the paradigmatic perspectives selected for the study. In this regard, equal voice not only refers to giving equal status to the qualitative and quantitative components used in the study but also refers to relying on qualitative and quantitative data sources equally during the interpretation of findings. The third characteristic focuses on preserving data integrity. Cronenberg (2020) argues that researchers should not employ data transformation techniques when employing the dialectic stance because these techniques do not maintain the integrity of each data source; rather, they inherently involve one data source subsuming another. The fourth characteristic stresses that researchers value divergence as much as they value convergence. In this way, the learning orientation of the dialectic stance is honoured, because the researcher is open to gaining knowledge from both types of findings – potentially generating underexplored, ignored, or new aspects of the phenomenon of interest.

5.7

Strategies to Navigate Difference

When employing a dialectic stance, mixed methods social science enquirers can expect both convergence and divergence as a result of their analysis

92

Jori N. Hall

of the different data sources. Of particular interest to the dialectic stance is navigating divergence or difference. Scholars provide some guidance on navigating difference, indicating that investigating the incongruities across different qualitative and quantitative methods can lead dialectical mixed methods researchers to (a) reexamine the data analysis procedures conducted for each data source, (b) collect and analyse additional data, (c) revisit the existing literature relevant to the phenomenon of interest, (d) consult literature from other domains, (e) construct theoretical frameworks, (f) revise existing theoretical frameworks, or (g) challenge traditional ways a construct has been defined or measured (Creamer & Edwards, 2019; Trend, 1978). All of these outcomes are potentially generative from a dialectic perspective. 5.7.1 Dialectic Framework

One creative strategy for navigating difference that has been proposed is the dialectic analysis framework offered by Cronenberg (2020). This framework is based on Cronenberg’s empirical work mixing the realist paradigm and the interpretivist paradigm and includes four key steps. The first step involves generating assertions. The second includes searching for both qualitative and quantitative data sources to support each assertation. The third step requires the enquirer to reject any assertation that cannot be supported by both qualitative and quantitative data sources. And the final step entails producing a written dialogic exchange whereby each assertation is described and supported by the qualitative and quantitative findings. This four-step process is done in conjunction with an open mind towards searching for convergence and divergence and a critical reflection process (i.e. researcher journaling, memoing) to assist with navigating any tensions or incongruencies that may arise in the data (Cronenberg, 2020). 5.7.2 Visualisations

Another creative technique when applying the dialectic stance that has become increasingly popular to notice, examine, and possibly reconcile incongruent findings is visualisations – namely joint displays (Creamer & Edwards, 2019). A joint display is an analytic technique that allows mixed methods researchers to visually ‘integrate the data by bringing the data together through a visual means to draw out new insights beyond the information gained from the separate quantitative and qualitative results’ (Fetters et al., 2013, p. 2143). While joint displays are not solely used to analyse divergent data, they have the unique capacity to visually display incongruencies and contribute to further investigation that potentially leads to the reconciliation of divergence (Creamer  & Edwards, 2019). Mixed methods researchers have creatively used joint displays to organise their data in several ways, namely through a

The Dialectic Stance

93

drawing, social network map, table, figure, and a matrix (Creamer & Edwards, 2019; Fetters et al., 2013). The process of navigating difference within the context of mixed methods practice has been described as a spiralling cycle wherein dynamic interactions occur within and across datasets, increasing understanding of the phenomenon with each cycle (Caracelli & Greene, 1993). Other dialectic mixed methods enquirers have described navigating difference as an ‘intense iterative approach’ (Cronenberg & Headley, 2019, p. 278). Pluye et al. (2009) provide additional analytical insights, alerting researchers to the issues associated with integration with an eye towards divergence. Specifically, they highlight how the tension generated from interrogating incongruities may lead some researchers to force a reconciliation of divergent findings. In the following section, I briefly note some empirical studies that employed the dialectic stance and review an example. 5.8

Dialectical Engagement: An Empirical Example

Although the empirical literature on applying the dialectic stance is scant, there are examples of employing this stance in fields such as evaluation (Betzner, 2008); education (Cronenberg  & Headley, 2019); health (Moffatt et al., 2006), and sport coaching (Campbell et al., 2022). In this section, I offer an example of dialectic mixed methods practice in the field of educational research. The example offers one way to apply the dialectic stance – not the only way. The example was selected given the high level of detail provided by the researcher, Marcia Gail Headley, about the relevancy of the dialectic stance for the research and how the stance was employed, especially the data analysis components. In the following sections, I provide some study highlights. For more details, readers are highly encouraged to carefully review the particulars of the study provided in Headley’s (2016) dissertation. 5.8.1

Context

This study investigates disciplinary literacy, in particular, symbolic mathematics language literacy (SMaLL), which Headley (2016) defines as the ability to read and write symbolic mathematics using the conventions of the writing system for the language of mathematics. In other words, the ability to use the writing system of the language of mathematics and academic conventions for writing symbolic mathematics to exchange ideas in print. (p. 15) Put another way, SMaLL is about ‘how, for example, students learn to look at x2 and say, “x, squared” or look at x and say, “the square root of x” ’

94

Jori N. Hall

(Cronenberg & Headley, 2019, p. 272). SMaLL is a developmental phenomenon that occurs at different levels: cultural, behavioural, and neurobiological. The main goal of this exploratory study, then, was to understand middle school students’ experiences with SMaLL, taking into consideration the language in the classroom, students’ metacognitive reflections, and students’ cognitive processes. Specifically, this study involved mathematics students in seventh and eighth grade at a middle school in a school district in Ohio. In terms of demographics, the majority of the students in the school district identified as White with less than 10% identifying as a student with disabilities, limited English proficiency, or economically disadvantaged. The study was conducted in the context of school district’s implementation of the Common Core State Standards (CCSS) – a set of educational standards adopted by some states/school districts in the USA for teaching and testing that indicate what students should know in mathematics and English language arts between kindergarten and 12th grade (www.corestandards.org/). Because SMaLL is a multilevel phenomenon, it was examined through the developmental bio-cultural constructivism (DBCCC) theory of ontogenetic, or lifespan, human development. When ‘applied to SMaLL, DBCCC holds that the ability to read and write the language of mathematics develops as a result of complex, ongoing reciprocal interactions between classroom context, students’ attempts to learn, and students’ brains’ (Headley, 2016, p. 26). Given this theory, the following key assumptions guided this study. First, SMaLL is an aspect of human development. Second, symbolic mathematics is culturally constructed and varies depending on context. And third, the cultural construction of the language of mathematics is related to the curriculum and educational policy – in this case, the CCSS. 5.8.2

Dialectic Mixed Methods Design

The mixed methods purpose of the study was initiation (Greene, 2007). This purpose was viewed in alignment with the dialectic stance as the goal of the study was to seek a better understanding of the different aspects of SMaLL and generate new understandings of SMaLL, particularly concerning the intersections of literacy and mathematics. With these goals in mind, the following questions guided the study:

• What is SMaLL for adolescent students in middle grades learning mathematics under the Common Core State Standards?

• How do students reading typical academic texts with symbolic mathematics experience SMaLL? To address these questions, a multilevel mixed-methods design was used. This design was employed and justified in alignment with the dialectic stance

The Dialectic Stance

95

given the need to examine the complexity of the phenomenon. Here, the complexity of the phenomenon refers to the multilevel nature of SMaLL, the different content areas (English literacy and mathematics literacy) that constitute SMaLL, and the different aspects of DBCCC – culture (e.g. classroom language and context), behaviour (e.g. student metacognition), and neurobiology (e.g. within-person cognition). Essentially, Headley (2016) contends that a multilevel mixed methods design aligns with the dialectic stance in that different methods were required to examine the different levels of SMaLL and initiate novel ways of thinking about SMaLL that support mathematics curriculum, instructional practice, and students’ mathematics achievement. In brief, a multilevel mixed methods design incorporates both qualitative and quantitative data and analytical tools to generate meta-inferences about more than one aspect of a multilevel phenomenon – system, levels, mechanisms – that transcend what could be inferred from a traditionally qualitative or traditionally quantitative approach alone. (Headley & Plano Clark, 2020, p. 152) A multilevel mixed methods design can be driven by a multilevel theory (i.e. Bronfenbrenner’s ecology of human development) (Headley  & Plano Clark, 2020) and employed sequentially or concurrently. In Headley’s (2016) research, the multilevel mixed methods design was guided by two paradigms, post-positivist and constructivist, and included concurrent qualitative and quantitative data collection and analysis with a final stage of data analysis wherein data were integrated to generate meta-inferences to address the research questions. Moreover, traditionally in educational research, different levels of analysis refer to students, classrooms, and the district. For this study, different levels referred to the aspects of SMaLL (culture, behaviour, and neurobiology) indicated in the DBCCC framework. The post-positivist paradigm informed the use of survey tools and mathrelated tasks to examine and measure students’ ability to recognise mathematic conventions, behaviours related to mathematics, and students’ reading habits associated with mathematical texts, mathematics anxiety, and mathematics achievement. These data allowed for examining information about mathematics reading at the neurobiological level and reading habits at the cultural and behavioural levels. A  total of 158 surveys were completed by seventh and eighth-grade mathematics students. The constructivist paradigm guided the individual interviews conducted with students and focused on their metacognitive reflections and oral reading of selected text. Student notes and researcher field notes were also sources of qualitative data. Students who participated in the interviews varied by mathematics achievement and reader type to understand variation in SMaLL and

96

Jori N. Hall

represented a subset of the quantitative participants, creating a dialogic link between the quantitative and qualitative data and facilitating integration. The qualitative strand attended to culture, although it was primarily designed to attend to the behavioural level of SMaLL. A total of 18 seventh- and eighthgrade students were interviewed.

5.9

Dialectic Data Analysis and Meta Inferences

In terms of data analysis, the quantitative and qualitative data were analysed concurrently and neither were prioritised. Qualitative and quantitative findings were then put into dialogue with each other for integration purposes and the possibility of discovering new insights. For instance, Headley (2016) notes that ‘it was possible for a theme emerging from the qualitative data set to suggest a new exploratory analysis of the quantitative sample’ (pp. 149– 150). Headley maintained a learner orientation by remaining open to how one data set could inform the other. To uphold a dialectic stance, integration techniques were employed to highlight convergence and divergence across datasets to glean new insights into SMaLL. To illustrate, Headley provided the following example of divergence: ‘the quantitative data suggested students do not ask others for help with reading; however, the qualitative reports suggested teachers, classmates, and family members are primary resources’ (p. 162). Joint displays were also used to identify areas of convergence and divergence across the datasets. For example, Headley (2016) created one joint display ‘showing the qualitative participants in relation to the range and mean of each quantitative measure and used a quantitative lens to review the qualitative data’ (p.  163). Given her dialectic stance, data transformation was determined to be inappropriate as the sampling sizes for qualitative and quantitative data were unequal. Final integration techniques were guided by the DBCC framework to identify meta-inferences across levels of SMaLL. Journaling and memoing to monitor the implementation of the dialectic stance were also conducted. Based on the iterative dialogic integration procedures mentioned earlier and others, Headley (2016) asserted the following meta-inferences by level:

• Cultural level: • Reading mathematics symbols is a cultural process in which students seek assistance, primarily from teachers, to develop SMaLL skills.

• Behavioural level: • Reading symbolic mathematics requires students to produce an English translation.

The Dialectic Stance

97

• Neurobiological level: • Reading symbolic mathematics is dependent on orthography, phenology, semantics, and affect. These meta-inferences are significant as each one was supported by both qualitative and quantitative data and helped to establish a model of SMaLL. Based on Headley’s (2016) experience using the dialectic stance, she concluded a few lessons learned. First, there is no one correct way to employ the stance. How the dialectic stance is employed depends on the goals of the mixed methods project, research questions, phenomena of interest, context, and so on. Second, there is no prescription for employing the dialectic stance. ‘The dialectic thrives on plurality. Articulation of the procedures and forms of the dialectic are potentially as numerous as the potential research questions’ (Cronenberg & Headley, 2019, p. 276). Third, the dialectic stance requires a ‘radical commitment to integration’. Based on Headley’s (2016) dissertation research, this radical commitment involves returning the data. A few possible reasons are given for continuously returning to the data: to look for patterns in the data using different theoretical or disciplinary lenses; to compare the findings from one data source based on findings from another data source; and to assess (refine, reject, or confirm) assertions drawn from the dialectic analysis (Cronenberg & Headley, 2019). 5.10

Conclusion

In this chapter, I provided a brief review of the historical roots of the dialectic stance, outlined how it is understood within the context of mixed methods work, and offered an empirical example illustrating how the dialectic stance was employed in mixed methods practice. In a way, this chapter can be viewed as a journey that leads to a better understanding of the foundational philosophies associated with the dialectical stance, contemporary contributions to the dialectic stance, and the mixed methods practices associated with the dialectic stance. Throughout this journey, I  assert how to navigate difference remains a central question in the fields of philosophy and mixed methods. I hope this chapter contributes to the growing dialectic-related literature (Johnson, 2017), advances critical thinking and practical strategies for navigating differences across scientific communities (Kuhn, 1970) within the context of mixed methods practice, and encourages mixed methods enquirers who are open to critical and intentional engagement with difference. Note 1 Enlightenment typically refers to an intellectual movement, largely in Europe, in the 17th and 18th centuries that privileged a family of ideas focused on challenging traditional religious dogma and the power of reason to improve various aspects of human life. For more information, see Peters (2019).

98

Jori N. Hall

References Betzner, A. (2008). Pragmatic and Dialectic Mixed Method Approaches: An Empirical Comparison. Minneapolis, MN: University of Minnesota. Campbell, S., Mills, J., Atkinson, O., Gearity, B., Kuklick, C., & McCullick, B. (2022). Engaging in paradigmatic dialogue: A  bibliometric analysis of coaching scholarship from 1970 to 2020. International Sport Coaching Journal, 1–13. Caracelli, V. J., & Greene, J. C. (1993). Data analysis strategies for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 15(2), 195–207. Creamer, E. G.,  & Edwards, C. (2019). Embedding the dialogic in mixed method approaches to theory development. International Journal of Research & Method in Education, 42(3), 239–251. Cronenberg, S. (2020). Paradigm parley: A framework for the dialectic stance. Journal of Mixed Methods Research, 14(1), 26–46. Cronenberg, S., & Headley, M. G. (2019). Dialectic dialogue: Reflections on adopting a dialectic stance. International Journal of Research & Method in Education, 42(3), 267–287. Duncombe, M., & Dutilh Novaes, C. (2016). Dialectic and logic in Aristotle and his tradition. History and Philosophy of Logic, 37(1), 1–8. Fetters, M. D., Curry, L. A., & Creswell, J. W. (2013). Achieving integration in mixed methods designs-principles and practices. Health Services Research, 48(2), 2134–2156. Fetters, M. D., & Molina-Azorin, J. F. (2017). The Journal of Mixed Methods Research starts a new decade: The mixed methods research integration trilogy and its dimensions. Journal of Mixed Methods Research, 11(3), 291–307. Geuss, R. (2016). Critical theory. In Routledge Encyclopedia of Philosophy (1st ed.). Routledge. https://doi.org/10.4324/9780415249126-S015-1. Greene, J. C. (2005). The generative potential of mixed methods inquiry. International Journal of Research & Method in Education, 28(2), 207–211. Greene, J. C. (2007). Mixed Methods in Social Inquiry (1st ed.). San Francisco, CA: Jossey-Bass. Greene, J. C., Benjamin, L., & Goodyear, L. (2001). The merits of mixing methods in evaluation. Evaluation, 7(1), 25–44. Greene, J. C., & Caracelli, V. J. (1997). Defining and describing the paradigm issue in mixed-method evaluation. New Directions for Evaluation, 1997(74), 5–17. Greene, J. C., Caracelli, V. J., & Graham, W. F. (1989). Toward a conceptual framework for mixed-method evaluation designs. Educational Evaluation and Policy Analysis, 11(3), 255–274. Greene, J. C., & Hall, J. N. (2010). Dialectics and pragmatism: Being of consequence. In A. Tashakkori & C. Teddlie (Eds.), Sage Handbook of Mixed Methods in Social and Behavioral Research (2nd ed., pp. 119–144). Thousand Oaks, CA: SAGE. Hall, J. N., & Copple, J. (2023). The value commitments of Jennifer C. Greene. In J. N. Hall, A. Boyce, & R. Hopson (Eds.), Disrupting Program Evaluation and Mixed Methods Research for a More Just Society: The Contributions of Jennifer C. Greene. Charlotte: Information Age Publishing. Headley, M. G. (2016). What Is Symbolic Mathematics Language Literacy? A Multilevel Mixed Methods Study of Adolescents in a Middle School (Order No. 10308492) (Doctoral dissertation). University of Cincinnati. ProQuest Dissertations and These Global.

The Dialectic Stance

99

Headley, M. G.,  & Plano Clark, V. L. (2020). Multilevel mixed methods research designs: Advancing a refined definition. Journal of Mixed Methods Research, 14(2), 145–163. Horkheimer, M., & Adorno, T. (1944). Dialectic of Enlightenment (J. Cumming, Trans.). New York, NY: Herder and Herder, Inc. Johnson, B. (2008). Editorial: Living with tensions: The dialectic approach. Journal of Mixed Methods Research, 2(3), 203–207. Johnson, R. B. (2017). Dialectical pluralism: A metaparadigm whose time has come. Journal of Mixed Methods Research, 11(2), 156–173. Kellner, D. (1993). Critical theory today: Revisiting the classics. Theory, Culture  & Society, 10(2), 43–60. Kidder, L., & Fine, M. (1987). Qualitative and quantitative methods: When stories converge. New Directions for Evaluation. https://doi.org/10.1002/EV.1459. Kuhn, T. (1970). The Structure of Scientific Revolutions (2nd ed.). Chicago: University of Chicago Press. Moffatt, S., White, M., Mackintosh, J.,  & Howel, D. (2006). Using quantitative and qualitative data in health services research – what happens when mixed method findings conflict? [ISRCTN61522618]. BMC Health Services Research, 6(1), 28. Mueller, G. E. (1958). The Hegel legend of “thesis-antithesis-synthesis.” Journal of the History of Ideas, 19(3), 411. Peters, M. (2019). The enlightenment and its critics. Educational Philosophy and Theory, 51(9), 886–894. Pluye, P., Grad, R., Levine, A.,  & Nicolau, B. (2009). Understanding divergence of quantitative and qualitative data (or results) in mixed methods studies. International Journal of Multiple Research Approaches, 3(1), 58–72. Tashakkori, A.,  & Creswell, J. W. (2007). Editorial: The new era of mixed methods. Journal of Mixed Methods Research, 1(1), 3–7. Trend, M. G. (1978). On the reconciliation of qualitative and quantitative analyses: A case study. Human Organization, 37(4), 345–354.

6 DIALECTICAL PLURALISM AND INTEGRATION IN MIXED METHODS RESEARCH R. Burke Johnson

6.1

What Is Dialectical Pluralism (DP)?

When I started using the term ‘dialectical pluralism’ (DP) in mixed methods research (MMR) around 2010 (Johnson, 2008c, 2011a, 2011b, 2012), my goal was to build upon and complement the work of Jennifer Greene and her ‘dialectical stance’ in evaluation and mixed methods research (Greene, 2005, 2007; Greene & Hall, 2010). Greene, and I to a lesser extent, came out of the evaluation literatures of the 1970s and 1980s. There, for example, the idea of integrating qualitative and quantitative research was present, for example, in the concept of multiplism (Cook & Reichardt, 1979; Cook, 1985; Shadish, 1986). This predates the formal founding of MMR. The importance of pluralism was also present in the evaluation literature (e.g., Greene, 1990; Sechrest, 1993). Greene was the first to discuss a ‘dialectical stance’ in contrast to other stances in MMR (Greene, 2005, 2007). House and Howe (1999) addressed the issue of deliberative democracy in evaluation. John Dewey provided the foundational work on the importance of pluralism over monism and social psychological strategies such as equal and open communication, active participation, democratic deliberation, and continual growth in groups; for example, see The Ethics of Democracy (Dewey, 1888), Democracy and Education (Dewey, 1916), Reconstruction in Philosophy (Dewey, 1920), The Public and Its Problems (Dewey, 1927), and his essay The Development of American Pragmatism (Dewey, 1931).1 I thought the new label ‘dialectical pluralism’ (DP) worked well for an integration of all of these important literatures. The label dialectical pluralism directly identifies the importance of pluralism in most dialectical approaches and pluralism is at the heart of mixed methods research. The particular name is not important – the process is important for MMR. DOI: 10.4324/9781003273288-7

Dialectical Pluralism & Integration in Mixed Methods Research

101

Building on Greene’s and many others’ prior work, my goals have been to (1) add more explicit and detailed discussion about the philosophical and methodological assumptions or ‘commitments’ of a dialectical approach in MMR (Johnson, 2017); (2) position DP as a metaparadigm that moves beyond single/purist approaches by communicating, combining, and integrating ideas from different paradigms, theories, methodologies, and relevant guiding values on a project-by-project basis; (3) emphasize the importance of pluralism; and (4) provide practical guidance for (mixed methods) researchers about how to use a dialectical approach successfully – I did this by adding some well-supported psychological and social psychological principles and strategies to facilitate dialectical thinking and its group processes (Johnson, 2017; Johnson et al., 2014; Tucker et al., 2020). The normative goal in DP is to work together constructively and thrive on the many important differences and natural tensions researchers face. DP is especially important for MM researchers that want to engage in what is variously known as equal-status, equal-emphasis, fully interactive, or fully integrated MMR. This kind of MM research or MM research design is perhaps the most challenging sort of MMR, because, for example, different paradigm ideas and divergent approaches are integrated and enacted in empirical research. I conceptualize dialectical pluralism as a conceptual ‘ideal type’. It is an ideal type in the Weberian (2021/1920) sense, meaning that DP is a theoretical model to which empirical cases can be planned, compared, analysed, and evaluated. A complete or 100% instantiation of DP will not be found in any single empirical case. DP is also an ‘ideal type’ in the everyday meaning of the word ‘ideal’; DP provides a useful and positive approach for producing collaborative visions, collaborative knowledge, and facilitating change. DP includes ideas and individual and group strategies found in multiple disciplines (e.g. social psychology, management, organizational behaviour, organization development, law, negotiation, sociology, and philosophy). Different users in MMR will use specific parts of DP, and this should be driven by our research questions. DP applications also vary by researchers’ expertise and training, nuanced contexts and needs, and different constraints and opportunities. My hope is that readers will find something helpful in DP for their practices. Stefurak et al. (2018) describe three real-world examples of DP applications. The first case study focused on a juvenile mental-health court programme. It required dialogue with three broad sets of stakeholders: (a) court personnel entrenched in legal and law enforcement systems, (b) mental health personnel more committed to humanistic and medical perspectives, and (c) adolescents themselves, and their parents, teachers, and more. Multiple collaborative group meetings were held to discuss the collaborative project goals and what kinds of data the different stakeholders can contribute and monitor. Over time, discussions included examining what goals were being met

102

R. Burke Johnson

and how they can work together for improvements for obtaining the desired project outcomes. The group process also had to work with some entrenched positions. Discussions of the reasons for the positions helped improve value and operational integration throughout the project. The collaborative dialogues helped the programme obtain multiple win-win solutions, but this did not always occur. The chapter also includes two more cases. One was a participatory evaluation of a residential mental health care centre for child welfare youth. The other DP application was for the development and formative evaluation of a group intervention for male juvenile offenders. DP has been applied in many different settings, such as emergency department experiences (Binne et al., 2021); attitudes towards mangroves in New Zealand (Dencer-Brown et al., 2019); academic support in child welfare services (Engell et  al., 2021); betterment of others by people with mental illnesses (Jordan et  al., 2022); counseling and psychotherapy (Stefurak et  al., 2023); perspectives and levels in learning and instruction research (Mejeh et al., 2023); physical activity (Sivaramakrishnan et al., 2023); social isolation and loneliness of older adults (Neves & Baecker, 2020); suicide prevention (Thorne & O’Reilly, 2022); and teacher equity-based dispositions (Tucker & Williams, 2019). In varying degrees, there are likely hundreds of examples in the literature and especially in practice where some of the DP-related concepts and principles were used for facilitating discussion, team process, working with stakeholders, and attempting to work towards win-win solutions (Deutsch, 2006). DP starts with several practical assumptions. Here are five: (a) researchers should consider multiple perspectives and sources of evidence that might appear to be conflicting and divergent; (b) no single perspective is exhaustive or perfect; (c) each perspective provides something good for consideration; (d) different perspectives drive creativity, innovation, change, more useful wholes; and (e) researchers should engage with multiple perspectives dialectically, dialogically, and empirically (Tucker et al., 2020). Dialectical pluralism (DP) is a ‘process and applied philosophy’ for producing integration or syntheses from differences with the goal of mutually beneficial (win-win) solutions (Rescher, 2000). It focuses on change, creativity, and deep listening and learning from others to continually produce new and practical solutions and integrations. Peter Abelard (1079–1142) expressed this idea quite a long time ago when he said ‘I preferred the weapons of dialectic to all the other teachings of philosophy, and armed with these I  chose the conflicts of disputation instead of the trophies of war’ (Letter I. p. 58; Abelard, 1981). Because of different project goals and missions, research questions, researchers, stakeholders, and contexts, instantiations of DP will vary on a project-by-project or study-by-study basis. DP is used to produce thoughtful, collaborative, integrated, workable wholes. The wholes, or integrations of ideas and practices, will reflect

Dialectical Pluralism & Integration in Mixed Methods Research

103

individual or intrapersonal mental models and group/collective interpersonal mental models. Both of these are dynamic as individuals and groups continue to update and reconstruct their thinking and mental models. A mental model is a schema or framework that includes our thinking about what we value, how things work, how to act, and how to meet our goals. Mental models include our philosophical and social embedded assumptions. Individuals and groups continually update and improve their mental models as they gain new information and experiences. Mental models are instantiated in particular domains at particular times and places. DP provides an interactive process for continually engaging and dialoguing with important tensions and differences that we and our teams face. Individual researchers and heterogeneous research teams routinely confront differences that need to be thoughtfully combined and integrated into mutually agreeable wholes. To facilitate group decisions, a fully inclusive deliberative democracy is helpful, where everyone has equal power and minority opinions are fully considered and integrated into the collaborative decisions (House & Howe, 1999; Johnson, 2015; Rawls, 1999). A deliberative democratic decision-making process can help motivate group members, and justify the group results because the process relied on deliberative democracy which includes procedural or process justice. Additional potentially important kinds of justice might include working towards social justice, distributive justice, retributive justice, global justice, and more (Johnson & Stefurak, 2014; Mertens, 2007; Tucker et al., 2020). In practice, the use of DP in group process can be difficult and imperfect, especially when a large number of stakeholders are included in the process (e.g. Greene, 2000). Many of the notable differences mixed methods researchers face are political, methodological, and philosophical. Politically, in a broad sense, we need to dialogue with what might be relevant regarding different values (e.g. social, political, epistemological values), and research/project objectives, specific valued/hoped-for outcomes, and intentional applications for intended groups. Methodologically, we need to dialogue with different (a) research methods (e.g. different methods of sampling, data collection, data analysis), (b) research methodologies (e.g. experimental methodology including weak, randomized, and quasi experimental research, nonexperimental quantitative research methodologies), and (c) qualitative methodologies (e.g. grounded theory, ethnography, phenomenology, narrative, and case study). Philosophically, we need to determine the kinds of knowledge we want and how to obtain sufficient justification for these claims of knowledge. We also need to dialogue with project-related philosophical and paradigmatic positions regarding ontology, epistemology, axiology/values, and methodological positions. In the remainder of this chapter, I address paradigm characteristics and integrations in MMR and philosophical and methodological positions for practice with DP.

104

R. Burke Johnson

6.2

Paradigms and Integration in Mixed Methods Research

MMR was significantly born from the ‘paradigm dialog’ or ‘paradigm wars’ (Gage, 1989; Goldman, 2022; Greene, 1990, 2015; Guba, 1990; Hammersley, 1992; Kuhn, 1962; Lincoln  & Guba, 1985; Tashakkori  & Teddlie, 1998). There are multiple paradigms in MMR that show different and important ways to understand and study the world (Johnson, 2011b). For example, see transformativism (Mertens, 2007, 2012), pragmatism (Biesta, 2003; Johnson  & Onwuegbuzie, 2004; Johnson, de Waal et  al., 2017; Johnson, Onwuegbuzie et al., 2017), realism (Maxwell, 2012; Maxwell & Mittapalli, 2010), and dialecticalism (Greene, 2007; Johnson, 2008b, 2008c, 2011a). Upon closer examination, the ‘dialectical paradigm’ is actually a pluralistic ‘metaparadigm’ for thoughtfully and systematically dialoguing with the many differences and tensions we face and integrating these into new and better practices and knowledge. Paradigms are not monoliths composed of true–false propositions, all of which might be True and logically consistent and therefore the paradigm as a whole also be said to be True. Paradigms are historical and social constructions. They typically contain complex sets of concerns, concepts, beliefs, positions, values, standpoints, in a loose set of ‘interrelated’ principles. In practice, paradigms are fluid. Regarding paradigms and their many particular assumptions and complexities, DP suggests how aspects from different paradigms can and should be fruitfully combined, despite some claims about what we cannot do. DP uses a creative and positive ‘can do’ motivation and disposition. DP rejects formal deductive or a priori arguments stipulating that paradigms are necessarily incommensurable and related claims that it is ‘logically incorrect’ for more than one paradigm to be used in empirical research. This notion is routinely proven false in myriad empirical applications. DP focuses on what can be done through empirical research, and once successfully conducted one will have empirical proof that it is possible. DP relies on many logics, including abductive, deductive, dialectical, dialogical, hermeneutical, inductive, fuzzy, practical, or everyday logic, and any additional logic that might facilitate production of useful and justified knowledge and practices. In empirical research, along with the pragmatist legacy, we are able to produce not universal Capital-T truths/knowledge but lowercase-t truths/ knowledge, provisional knowledge, probabilistic knowledge, or working knowledge (Johnson, 2008a). The knowledge we produce also is oftentimes ‘thick knowledge’ (i.e. knowledge that includes values) (Putnam, 2002). DP encourages integrations at the individual level, and aids in producing new, evolving, nuanced, and complex individual mental-models. Here are some individual-level DP strategies:

• Interact with and give attention to multiple authors, experiences, and perspectives;

Dialectical Pluralism & Integration in Mixed Methods Research

• • • • • • • •

105

Attempt to understand multiple sides of conflicting issues; Dialogue with multiple perspectives and mental models; Respect and utilize pluralism in one’s creative thought; Use synechism (i.e. rejecting dualisms and instead using continuums; see Peirce, 1893/1998); Work towards syncretism (i.e. reconciling and producing new unions of opposing principles and practices); Connect theory and practice in your thinking; Construct creative ‘golden means’ or balances; and Strive for dialectical integration in one’s thinking (Johnson, 2017).

A goal of DP is to continually improve our mental models and our resulting practices. DP recommends using well-supported strategies for facilitating development of group and collective mental-models. DP is needed when constructing representative heterogeneous groups that include multiple perspectives and stakeholders (Bennett et al., 2010; Johnson, 2017; Tucker et al., 2020). It is helpful to have one or more DP or group-process facilitator. One overall project strategy is to continually bring the project/team back to its motivating and agreed upon superordinate goal of the project/team to fulfil its mission. The mission typically is to complete or improve something, and to do it well (high quality). This may require group work to reach general/shared agreement about the mission. Let members create and select their complementary roles and assignments to meet the overall mission. Here are a few more of the many, research-based, group-level or group-process strategies:

• Produce collaborative logic models; • Identify people that are open to productive dialogue and change; • Facilitate development of a collaborative ‘us-we’ perspective rather than a ‘me-you’ individualism;

• Articulate common interests and goals (e.g. improve schools, facilitate • • • • • • • •

reductions in depression, help students learn and grow); Be sensitive and interested in differences and work towards producing win-win solutions; Fully hear, understand, and learn from important differences; Empathetically understand how and why others have different perspectives; Treat others as someone you can learn from and learn with; Use the ‘equal weight view’ (give equal weight to the opinion of the other and your opinion (Kelly, 2013); Practice reflexivity and reciprocity; Search for middle positions and balances; Combine relevant ideas and practices using a both-and logic into mutuallyworkable wholes;

106

R. Burke Johnson

• Understand that virtually all issues and decisions are value complex and therefore try to produce ‘value combinations’ that will help facilitate desired outcomes; and • Continue the process because initial differences will change through additional group process (Johnson, 2017; Tucker et al., 2020). When agreements are difficult to obtain, there are many additional conflict management strategies (see Coleman et al., 2014). ‘Functional conflict’ can serve as a catalyst for change (Robbins, 1978). One useful practice is to use fractionalization in which the facilitator enables the disputants to agree on smaller issues or parts of the larger disagreement and then attempt, again, to move to a larger agreement (Dues, 2010). Also helpful is Fisher and Ury’s (2011) principled negotiation that uses these important strategies (from the Harvard Negotiation Project):

• • • •

Separate the people from the problem; Focus on interests, not problems; Invent options for mutual gain; and Insist on using objective criteria.Please take a moment now to examine Table 6.1. It shows many of the key characteristics of dialectical pluralism. Next, I discuss philosophical assumptions of DP.

TABLE 6.1 Characteristics of Dialectical Pluralism (DP)

Dialectical pluralism . . . • Is a process philosophy for dialoguing with differences. • Is positive, facilitative, and forward directed. • Is dynamic, and open. • Is collaborative and holistic. • Requires creativity, thoughtful change, and a desire for integration. • Requires users to fully hear and ‘thrive’ on differences and produce new wholes and integrations. • Uses pluralism in many forms. • Emphasizes a logic of combination (‘both-and’) and a logic of integration (thesis, counterthesis, synthesis/integration). • Uses empirically based (i.e. a posteriori) arguments rather than deductively based (i.e. a priori) arguments about what is necessarily and universally true. • Can be used to dialogue with multiple paradigms including their many complex similarities, differences, and commonalities. • Can also dialogue with many nonparadigmatic differences and trade-offs (e.g. Table 5.2 in Johnson, 2017). • Is needed when working with heterogeneous groups and teams (e.g. researchers, participants, and stakeholders with different mental models). • Uses deliberative democracy to motivate and help justify collective/group decisions.

Dialectical Pluralism & Integration in Mixed Methods Research

107

• Produces new, workable, integrated wholes. • Facilitates development of individual-level integrative mental-models. • Facilitates development of group- or collective-level integrative mental-models. • Is used on a project-by-project basis because each study has different research questions. • Is especially important for mixed methods research, but is a general process theory that can be used far beyond MMR.

6.3

Philosophical ‘Commitments’ in Dialectical Pluralism

Dialectical pluralism is a metaparadigm for integration. The first word, ‘dialectical’, is meant to convey a systematic and continual process of dialectical and dialogical communication (where everyone has equal power) for producing integration. The word, dialectical, is an age-old concept of dialoguing with differences and complex ideas. Each new dialectical integration or synthesis is the starting point for the next dialogue. Most dialogues will involve trade-offs that need to be resolved. DP is useful at all levels of a paradigm, including their philosophical and methodological assumptions. DP is useful across and at each stage of a research study, such as the initiation, implementation, analysis, and inference stages (Tashakkori et  al., 2021). A  key goal is to produce new combinations and integrations of philosophies, methods, methodologies, and so forth. For much more on integration in MMR, see Bazeley (2018), Creamer (2022), Creswell and Plano Clark (2018), Fetters et al. (2013), Fetters and Molina-Azorin (2017), Greene and Caracelli (1997), Hesse-Biber and Johnson (2015), Hitchcock and Onwuegbuzie (2022), Tashakkori et al. (2021), and Tashakkori and Teddlie (2003, 2010). The second word in DP is pluralism. The second word, ‘pluralism’, is meant to convey a deep respect for diversity of views and approaches rather than a single viewpoint or approach. MMR in theory and practice has always been a strong supporter of pluralism; this support is usually implicit. Importantly, MMR has extensively contributed to the literature by explaining and showing how pluralism can be operationalized and conducted in practice through combining and integrating qualitative and quantitative approaches. Constructing a pluralistic intellectual environment and nurturing a culture of critical dialogue inquiry help draw out a more robust exchange of ideas. MMR is all about appreciating and combining or integrating multiple ideas and approaches. Philosopher Waller Watson (1990) had this to say about pluralism: The transition to pluralism does not require an abandonment of the unwarranted privileging of one’s own philosophy, which is the result of an illusion of its superiority, which is the result of the fact that we understand

108

R. Burke Johnson

our own philosophy better than we understand the philosophies of others. A key factor in the development of pluralism is therefore a more accurate and sympathetic reading of texts. (p. 357) Watson’s position fits perfectly with the DP principle of fully listening, understanding, and dialoguing with ‘the other’. In DP, we recognize that pluralism is real and we continually need to examine and appreciate our differences so that we can build on our differences to advance research in theory and practice. When we learn from and with others, we can increase creativity and help produce innovative and better MM research. Whether we admit it or not, some of academics’ debates and positions are also influenced by our individual egocentrisms (i.e. implicitly: ‘I know that I’m right’) and our ethnocentrisms (implicitly: ‘Those others are odd and they don’t get it’). We all need to continually develop and improve our mental models; that’s the nature of mental models. Philosopher Eugene Garver (1990) made this bold claim, ‘We are all pluralists today. . . . Some people are reluctant pluralists, disappointed with the inescapable fact of plurality, while others embrace it with delight and hope’ (p. 388). Pluralism is popular in many fields today. The reasons for using pluralism are varied, but the reasons often surround issues such as ontology, epistemology, explanation, values, research methodology, and substantive theories. Here are just a few recent examples of pluralism in diverse fields: cognitive science (Dale et al., 2009), political science (Crasnow, 2019; Stuhr, 2020), psychology (Novis-Deutsch, 2020), economics (Moneta & Russo, 2014), philosophy (see special issue on pluralism in The Monist, 1990, Vol. 73, No. 3), metaphilosophy (Zangwill, 2020), medicine (Cournoyea & Kennedy, 2014), business (Br’es et al., 2018; Schormair & Gilbert, 2021), bioethics (Edwards et  al., 2020), suicide research (Maung, 2020), psychiatry (Jerotic  & Aftab, 2021), medical science (Edwards et  al., 2020), and public administration (Overeem & Verhoef, 2015). There are many additional pluralisms across disciplines, contexts, and practices. Pluralism is especially important in MMR. One could make a case that pluralism in some form(s) is MMR’s raison d’etre or its reason for existence. We need to find mixed positions for ontological arguments, epistemological arguments, methodological arguments, and so forth. When applied, pluralism is a philosophy about hearing multiple important perspectives, and searching for integration using, for example, complementarity and integration, perhaps using Aristotle’s principles of ‘moderation’ and searching for ‘golden means’. DP shows us how to work with pluralism to produce new integrations. Specific mixes or ‘packages’ of ideas and their integrations are often needed for each new research study because of its particular research questions. Let’s further examine some of these ideas now.

Dialectical Pluralism & Integration in Mixed Methods Research

6.3.1

109

Ontology in Dialectical Pluralism

At its base, DP relies on ontological pluralism, which is defined here as an understanding and often appreciating multiple kinds of reality and multiple ontologies. Ontology refers to what is to be considered to be real. Philosopher W. V. O. Quine (1980) perspicuously stated: ‘A curious thing about the ontological problem is its simplicity. It can be put in three Anglo-Saxon syllables: “What is there?” It can, moreover, answered in a word “everything” ’ (p. 1). DP views many ontological realities as potentially important for a research study. Here are some different ways I find helpful to ‘carve nature at its joints’ (Plato is believed to have said this). One important way to ‘carve nature’ includes subjective reality, intersubjective reality, and objective reality. They all are ‘ontologically real’. I like to say, ‘there is enough reality for all of us’. These kinds of reality are all important in MM research. Subjective reality is personal and refers to an individual’s inner world, including qualia, inner experiences, nuanced thoughts and feelings, and specific perspectives. Intersubjective/social/conceptual reality refers to social, conceptual, and shared realities such as language, concepts, norms, customs, culture, and so forth. It also includes ‘variables’ which are of primary interest in quantitative research. Objective reality refers to material, physical, and process reality, and causation between any of the major kinds and levels of reality. Examples of material reality are houses, mountains, and airplanes. Examples of physical-relationship-process reality are chemical interactions, gravitation, heat, and statistical relationships among variables in quantitative research. Importantly, if there were no people in the world, there would be neither human subjective nor intersubjective reality. Another important way to see important kinds of reality is disciplinary reality. This refers to the realities studied by different academic disciplines. All research disciplines (that I know of) examine something that is real in some way. Economists study reality such as public goods, consumer surplus, and the ‘invisible-hand’. Sociologists study reality such as social structure, roles, and institutionalized racism and sexism. Psychologists study reality such as self-esteem, cognitive dissonance, and groupthink. Political scientists study reality such as political parties, revolutions, and terrorism. Chemists study reality such as kinetic energy and chemical reactions. Physicists study reality such as energy, wave theory of light, and thermodynamics. The list of disciplinary realities goes on. There are many competing ontological theories for dialogue. Here are a few named ontologies: physicalism/materialism, idealism, social ontology, ontological realism, ontological relativism, ontological holism, monism, and process ontology. Here are three examples: (a) ontological realism is the view that at least some entities are real, and independent of our experience or knowledge of them, and of our concepts about them; (b) ontological antirealism is the view that some entities that are currently claimed to exist are

110

R. Burke Johnson

mind-dependent; reality is at least partially constructed by human minds; (c) ontological relativism suggests that whatever exists can only be said to exist for a particular person or culture and truths do not generalize across cultures or from one person to another (definitions from Johnson et al., 2004, pp. 9–10). DP is useful in examining multiple realties and considering multiple ontologies, and sometimes combining different ontologies into new and creative mixes or syntheses, and an ‘ontological package’ including a research team’s specific ideas about what is important in each new research study. For example, our team has decided to study some subjective, intersubjective, and objective realities using MMR; our team will incorporate some useful ideas we see in ontological realism and ontological relativism. “Holism” provides one of many justifications for carefully combining diverse ideas, including those that might seem quite different, into new, dynamic wholes. DP rejects arguments about claims of what cannot be done with different ontologies. If you can do something in the world, that is proof. Even at the general level of paradigms, new wholes can be produced. This is an important goal of DP. Pragmatism, for example, can be combined with virtually any paradigm. Consider the combination of transformativism (Mertens, 2007, 2012) and some variety of pragmatism (Johnson, de Waal et  al., 2017; Johnson, Onwuegbuzie et  al., 2017). This combination packs transformative values into our pragmatic answering of research questions and our deliberative actions. Another useful paradigmatic combination is critical realism (Maxwell & Mittapalli, 2010; Maxwell, 2012; Pawson, 2016) and pragmatism, where critical realism might add some of the useful theoretical frames seen in Archer et al.’s (1998) and Maxwell’s (2012) critical realisms (e.g. the morphogenetic approach to social theory, emergence). Transformativism and critical realism can also be combined, where transformativism adds important sets of values to critical realism in MM research. There are many additional paradigms and combinations. DP encourages users to combine and integrate ideas found in different paradigms – this should be done carefully, systematically, and with impunity!

• The DP ontological principle: Recognize multiple kinds of reality and the presence of different ontologies and the tensions they produce as a strength to be embraced rather than a weakness that stunts growth. Researchers can produce from this a new, practical ontological mix or package of relevant ideas and objects for each research study. 6.3.2

Epistemology in Dialectical Pluralism

Epistemology is ‘the branch of philosophy concerned with knowledge and its justification. It addresses questions about what knowledge is, how knowledge

Dialectical Pluralism & Integration in Mixed Methods Research

111

is obtained, and what standards must be met to make a justified claim of knowledge’ (Tashakkori et  al., 2021, p.  398; also see Feldman, 2003). In research, this means, ‘What kinds of knowledge do we hope to produce?’ (Johnson, 2008a), ‘What kinds of data do we need?’, and ‘What standards will we use to show that our knowledge is justified?’ (cf. ‘validity’ in quantitative research, ‘trustworthiness’ in qualitative research, and ‘legitimation’ in MMR, Johnson  & Christensen, 2020; Johnson & Stefurak, 2013; Onwuegbuzie  & Johnson, 2006), and ‘Should we feel comfortable that we have made a justified claim of knowledge?’ (Copi et al., 2019; Feldman, 2003). DP relies on epistemological pluralism, which refers to understanding and often appreciating ideas found in different epistemologies. For a research study, the collaborating team can produce an epistemological combination, mix, or agreed-upon and practical ‘package’ of ideas. The package might include, for example, ‘What combination of “validity,” “trustworthiness,” and “legitimation” standards will our team use for our research study?’ When one’s mental model includes epistemological-pluralism thinking, it is usually perspectival (Giere, 2006) and interpretivist to a degree. Individuals and groups often have different perspectives about what might appear to be the same ‘ontological object’ (e.g. a wristwatch, a motor in an automobile, a government, a paradigm). An ontological object, however, is a ‘complex whole’, and different perceivers will ‘see’ it in different ways and identify different aspects of it because they bring their own mental models to the observation. People influenced by different paradigms will see objects differently because of the language, concepts, and emphases found in their paradigms (Kuhn, 2012). This idea is also seen in Hilary Putnam’s (1987) concept of internal realism where knowledge is shared and produced within language communities or paradigms and is understood and considered justified using their internal standards (cf. Scheffler, 2009). Another key relevant idea is the-later-Wittgenstein concepts of forms of life and language games where sense-making, meaning, and rules differ across language groups, communities, contexts, and places (Wittgenstein, 2009/1953) Generally speaking, DP and MMR rely on a ‘logic of combination’ called the both-and logic, that rejects binaries and combines different ideas and realities into new wholes (Johnson et  al., 2019, p.  143; Tashakkori et  al., 2021, pp. 18, 65, 358). The both-and logic rejects binary and false choices and, instead, searches for what is relevant, good, and true in different positions to produce collaborative/integrative syntheses. To move even closer to full integration, DP and MMR emphasize a logic of integration/synthesis (e.g. thesis, antithesis, integration/synthesis). Integration is the desired outcome of this logic. In practice, both of these logics produce integrations and are used together. Let’s take a moment to describe one particular epistemological theory, found in John Dewey’s works. It aligns pretty well with the empiricism used

112 R. Burke Johnson

in most research in the social sciences. Most of the ‘truths’ or ‘knowledge claims’ found in the empirical sciences can be called ‘provisional truths/ knowledge’, ‘working truths/knowledge’, or ‘the best truths we have so far’. Empiricism mostly rejects ‘final, timeless, universal, and certain truths’. This is fundamentally because the empirical sciences are always be open to the possibility of new data and new and changing truths. Dewey (1939) put it like this: The “truth” of any present proposition is, by the definition, subject to the outcome of continued inquiries; its “truth,” if the word must be used, is provisional; as near the truth as inquiry has as yet come, a matter determined not by a guess at some future belief but by the care and pains with which inquiry has been conducted up to the present time. (pp. 56–57, emphasis added) This quote is one of the few places where Dewey defined truth, because he thought our time was better spent avoiding unsolvable metaphysical quagmires and instead work to produce the kind of knowledge we can obtain. For Dewey, truth and knowledge are better viewed as continual, ongoing processes. A strong truth-or-knowledge claim must provide a ‘warranted assertion’ and meet the epistemological criterion of warranted assertibility. Dewey stated that ‘knowledge in its strictest and most honorific sense is identical with warranted assertion’ (Later Works, V. 12, p. 146). According to this pragmatic epistemology or theory of truth, warranted assertibility is what researchers can obtain. Epistemological pluralism recognizes the potential importance of more than one epistemology and asks us to put these into dialogue. Dewey’s epistemology of empiricism or pragmatism is only one theory. Here are a few additional epistemological theories to consider and dialogue with scepticism, coherentism, contextualism, reliabilism, foundationalism, feminism and other standpoint epistemologies, postpositivism, naturalism, relativism, communitarianism, interpretivism, critical theory, evolutionary epistemology, scientific realism, and constructivism. There are many additional epistemologies, many of which have yet to be articulated! As mentioned earlier, epistemology also asks the question of ‘What are the required standards for justifying knowledge’. This is where researchers need to produce defensible, high-quality research. In quantitative research this is called ‘validity’ and includes statistical conclusion validity, construct validity, internal validity, and external validity. In qualitative research, this is sometimes called ‘trustworthiness’ and includes many nuanced typologies. One especially useful typology for qualitative research is Joseph A. Maxwell’s descriptive-, interpretive/emic-, and theoretical validity/trustworthiness.

Dialectical Pluralism & Integration in Mixed Methods Research

113

In MMR, the analogue of validity (in quantitative research) and trustworthiness (in qualitative research) is legitimation (Onwuegbuzie & Johnson, 2006; Johnson & Christensen, 2020). It is important that MM researchers and methodologists articulate the kinds of legitimation they see as important. The growing Onwuegbuzie–Johnson typology constructed for MMR currently includes 13 kinds of legitimation, and researchers and methodologists in MMR should continually add more types/kinds. The current list includes emic-etic legitimation, paradigmatic/philosophical legitimation, commensurability approximation legitimation, weakness minimization legitimation, sequential legitimation, conversion legitimation, sample integration legitimation, causation legitimation, pragmatic legitimation, integration legitimation, and multiple stakeholder legitimation, and multiple validities legitimation (recent definitions are found in Chapter 11, Johnson & Christensen, 2020, 2024). The last and most important type of MMR legitimation is multiple validities legitimation. It is comprehensive and dynamic – it refers to degree to which the relevant or pertinent ‘validities’ (quantitative, qualitative, and mixed methods) for a particular research study are addressed and resolved successfully. Multiple validities legitimation is important for justification in all empirical research! This is a pluralist concept and is facilitated with DP, because the researcher or team needs to examine and dialogue with the relevant standards and construct a nuanced mix or practical ‘package’ of ideas specifying what is relevant for each research study. For example, ‘Our study is focused on explanation and our mixed research team emphasizes the relevance and importance of internal validity, interpretive trustworthiness, and emic-etic, sample integration, and sequential legitimation’. Last, Michael Patton (2015) thoughtfully articulated five sets of standards that MMR should continually dialogue with and determine what is relevant for an MM research study. These sets are grouped for different research paradigms and communities. Patton listed and explained the criteria applicable for several research paradigms or communities. His set includes the following: (a) traditional scientific research criteria, (b) social construction and constructivist criteria, (c) artistic and evocative criteria, (d) participatory and collaborative criteria, (e) critical change criteria, (f) systems thinking and complexity criteria, and (g) pragmatic, utilitarian-focused criteria (Patton, 2015, pp. 680–681).

• The DP epistemological principle: Carefully and continually listen to and dialogue with different epistemologies (especially from others on the heterogeneous research team), embrace the differences and tensions, and determine and use what is epistemically relevant and important for your particular research questions. This usually means constructing an epistemological mix or practical ‘package of ideas’ for each research study.

114

6.3.3

R. Burke Johnson

Axiology/Values in Dialectical Pluralism

DP includes axiological or value pluralism, which refers to understanding and often appreciating ideas found in multiple ethical theories and values. Complex decision-making always involves/includes values and ethics. Mixed methods researchers should consider their particular values, including their implicit values, and dialogue with any additional values and value theories that might be relevant to their research and dialogue with others. When operating in a heterogeneous group, the principle is to respectfully and deeply listen and learn from and with others. When searching for collaborative agreement, groups should also focus on their instrumental values, such as fulfilling the purpose(s) of the study in a way that works well and meets ‘multiple stakeholder legitimation’.2 Multiple stakeholder legitimation is ‘the extent to which a mixed researcher appropriately addresses the interests, values, and standpoints of multiple stakeholders’ (Johnson  & Christensen, 2020, p. 292). Using this principle, the collaborative team can attempt to put together its useful and agreed-upon or collective ‘package of values’ to serve the group and the key stakeholders. For example, a particular research team might decide to focus on valuing explanation, understanding, and social justice. Instrumental epistemological values are especially important in empirical research and in applied settings. Qualitative research especially values subjective and intersubjective understanding (Searle, 2008). Quantitative research especially values variables-based description, explanation, and prediction. Qualitative and quantitative research vary somewhat with the qualitative embracing the emic viewpoint and the quantitative embracing the etic viewpoint. MMR additionally embraces emic-etic legitimation, defined as ‘The extent to which the researcher accurately understands, uses, and presents the participants’ subjective insider or ‘native’ views (called the emic viewpoint) and the researcher’s ‘objective’ outsider view (called the etic viewpoint)’ (Johnson & Christensen, 2020, p. 290).3 Qualitative and quantitative research, along with MMR, also importantly values social betterment. Despite general statements of the differences between quantitative, qualitative, and MMR research, there can be some commonality across paradigms and their ideas and approaches (Johnson & Onwuegbuzie, 2004). There are many values different researchers use in various contexts. First are social/ practical values, such as beneficence, honesty, respect for participants, justice, honour, empowerment, freedom, equality, integrity, and compassion. Second are epistemic values and ends, such as truth/correctness, objectivity, prediction, explanation, generalization, discovery, verstehen, contextual and subjective understanding, parsimony, justification/warrant, and practical knowledge. The positivist claim of value-free objectivity is clearly not supported; this is seen in the positivist’s use of epistemological values as well as other values impacting the design, conduct, and interpretation of the research

Dialectical Pluralism & Integration in Mixed Methods Research

115

data/outcomes/results. Third are ‘epistemic virtues’, such as trustworthy/honesty, open-mindedness, curiosity/inquisitive, respect for multiple perspectives, good observer, patience, self-reflective/self-honesty, perspicaciousness, and patience. There are many theories of ethics that pluralists might examine. Here are some of them: ethical realism, utilitarianism, virtue ethics, social justice, social contract theory, ethical relativism, standpoint ethics, deontological, and duty ethics (e.g. Immanuel Kant’s categorical imperative), rights-based ethics, golden mean ethics, natural law, pragmatism and ethics of deliberative democracy (Dewey), value neutrality in research, Rawls’ theory of justice, consequentialism/outcome ethics, Christian ethics, normative and descriptive ethics, applied ethics, and metaethics. I will define four ethical types for comparison (Johnson et al., 2004; also Sheehan, 2011; Sheehan & Johnson, 2012). Ethical realism .  .  . is an axiological view positing that some ethical or moral statements are true or false independent of the dispositions, beliefs, attitudes, or practices of a given society or individual . . . ethical relativism holds that truths of ethical and moral judgments are entirely dependent upon individual or cultural context. .  .  . Value-neutrality is an axiological standpoint that promotes the value-neutrality of scientific inquiry as an ideal. Value-promotion suggests that science neither can nor should attempt to be value-neutral. (pp. 9–10) In DP, researchers are empowered to dialogue with and consider ideas and principles in multiple ethical theories. They also are enabled to dialogue with multiple values that team members bring to a collaborative research project. Although values usually vary by person and groups, there seem to be a few universal values as seen, for example, in the United Nations 1948 Declaration of Universal Human Rights or UDHR (available via Google or any search engine). I  suspect many (most?) humans and researchers, including ethical relativists, will agree with some (many?) of the 21 rights or articles. Here are the first four: Article 1. All human beings are born free and equal in dignity and rights . . . and should act towards one another in a spirit of brotherhood. . . . Article 2. Everyone is entitled to all the rights and freedoms set forth in this Declaration, without distinction of any kind, such as race, colour, sex, language, religion, political or other opinion, national or social origin, property, birth or other status. .  .  . Article 3. Everyone has the right to life, liberty, and security of person. . . . Article 4. No one shall be held in slavery or servitude; slavery and the slave trade shall be prohibited in all their forms.

116

R. Burke Johnson

However, universal value agreement is rare, and different (often conflicting) values will operate for most decisions MM researchers and their stakeholders must make. Almost all specific issues in research and policy are value complex, because multiple, often conflicting, values will be relevant to individual and group decisions. This is where DP enters the picture. Researchers and stakeholders will bring different values and value theories that will need to be examined and submitted to empathetic dialogue. The group-process strategies previously discussed can be used to produce what can be called a ‘heterogeneous integration’, such as focusing on a collaborative agreement and fractionalization. In short, for each research study or each research team, a unique package of the relevant values (some of which will be in conflict because of trade-offs) will be important. The DP group-process strategies can help make this possible.

• The DP axiological principle: The presence of multiple ethical theories and particular values and the tensions they produce are treated as strengths for dialogue. They should be viewed and embraced as an opportunity for growth and collaborative agreement, rather than a weakness that shuts down conversation. From this dialogue, a new axiological mix or practical ‘package’ of the relevant values can be produced for each research study. 6.3.4

Methods in Dialectical Pluralism

DP includes the principle of methods pluralism which refers to appreciating and dialoguing with multiple methods and the ideas found in them to produce useful combinations for answering our research questions. By ‘methods’, I am referring to methods of data collection, methods of sampling, methods of data analysis, and any other methods used in MMR. For example, a single method of data collection can provide both qualitative and quantitative data – this is called within-method mixing. This is because each major method of data collection can have a qualitative version, a quantitative version, and a mixed version (Johnson & Turner, 2003; Tashakkori et al., 2021). For example, a questionnaire can be (a) fully open-ended (producing qualitative data); (b) fully structured to measure variables (producing quantitative data), or (c) a combination of open-ended and open-ended items (producing qualitative and quantitative data). Two or more different methods of data collection can also provide qualitative and quantitative data – this is called between-methods mixing. For example, an MM researcher might use a structured-quantitative questionnaire and conduct exploratory-qualitative interviews. Taking into account within-method mixing and between-methods mixing, there are many additional possibilities for data collection to be discovered and articulated.

Dialectical Pluralism & Integration in Mixed Methods Research

117

Many pluralist analytic methods are found in a new Routledge book on mixed methods data analysis edited by Onwuegbuzie and Johnson (2021). This book was needed in MMR, because most previous writing had emphasized only results mixing but not analysis mixing. It was traditionally assumed in MMR that the qualitative data were analysed using only qualitative analysis methods, and the quantitative data were analysed only using quantitative methods. The both and logic is appropriate here. It is very important to first analyze qualitative data using qualitative analysis, and first analyze quantitative data using quantitative analysis. However, adding a ‘stage-two’ of mixed methods analysis or crossover analysis can often add additional information and insights. Crossover analysis: At its most integrated form, the mixed analysis might involve some form of cross-over analysis, wherein one or more analysis types associated with one tradition (e.g. qualitative analysis) are used to analyse data associated with a different tradition (e.g. quantitative data). (Onwuegbuzie & Combs, 2010, pp. 425–426) The book on mixed methods data analysis includes 40 chapters written by international scholars interested in showing how to conduct MM crossover analysis. The first part explains approaches for quantitative analysis of qualitative or text data (e.g. factor analysis and correspondence analysis). The second part explains approaches for qualitative analysis of quantitative data (e.g. coding techniques and mixed discourse analysis). The third part explains a number of ‘inherently mixed’ analytic methods (e.g. MM social network analysis, MM-GIS, QCA, and use of joint displays during analysis). The fourth section explains how to use computer packages for the kinds of data analyses covered in the book (e.g. QDA Miner with SimStat and WordStat, MAXQDA, SPSS, NVivo, Dedoose, and ATLAS). Researchers and their interdisciplinary teams will likely, over time, use more and more of these mixed analysis methods. Researches will also continually articulate new methods of MM analysis!

• The DP methods principle: Researchers and stakeholders should dialectically listen, consider, and combine/integrate multiple methods in a research study based on what is needed to answer the particular research questions. 6.3.5

Methodology and Dialectical Pluralism

DP includes methodological pluralism, which refers to appreciating and dialoguing with multiple methodologies and their associated ideas, concepts, and logics. A ‘methodology’ is larger than a method. For example, experimental research and ethnography are research methodologies, and so forth. One can collect qualitative and quantitative data using any methodology. It

118 R. Burke Johnson

becomes more integrated when aspects of different methodologies are integrated into a new mixed methods-methodology. For a first example of methodological pluralism, Shim et al. (2017, 2020) demonstrated how to produce a mixed methods-grounded theory (MM-GT). In the exploratory phase, Shim treated the published literature as ‘data’ and used a grounded theory logic to generate a formative theory. A  formative literature-based grounded theory produces an integrated literature-based substantive theory. At roughly the same time (concurrently), Shim used the traditional inductive, interview-based, grounded theory methodology and, following the rules of traditional GT, developed a standard grounded theory from the interview data (Corbin & Strauss, 2015; Glaser & Strass, 1967). Using metamodeling logic, those two theoretical models were integrated into a metamodel to complete the exploratory phase of the study. Metamodeling is an analytic method for ‘producing models from models’ (Johnson, 1998). In the second phase, the confirmatory phase, Shim designed and conducted a mixed methods-experiment (MM-EXP). Many of the concepts in the phase-I metamodel were operationalized and empirically tested in phase-II, and, concurrently, Shim also constructed yet another GT model based on the new qualitative interview data. In phase III integration, Shim merged the phase-I and phase-II models into the final phase-III model which was the ‘final’ integrated grounded theory. Methods and methodologies, and their logics, were used multiple ways in this MM-GT study, making it a strong empirical research study for making contributions to substantive theory in research literatures. For a second example of methodological pluralism, Schoonenboom and Johnson (2017; http://rdcu.be/tXiS) dialogued with each other, with colleagues, and with the different methodological issues that might be relevant to designing MM research studies. On the one hand is a style that could be called a ‘structured’ or ‘design-centric’ style (e.g. Creswell  & Plano Clark, 2018). This style, to some degree, teaches readers to select a design from a menu of ‘MM designs’, which oftentimes will work in basic/short studies. On the other hand, there is a style that could be called a constructive or innovative style for expanding thinking about mixed methods research designs and their purposive construction (e.g. Schoonenboom  & Johnson, 2017). New combinations of methodologies and methods are included in planning and design construction. The structured and constructive styles are commonly used in MMR. Both approaches are important. Schoonenboom and Johnson advocated for learning how to construct MMR designs. This places the process of ‘designing’ not just in the hands of MMR methodologists, but importantly in the hands of practicing researchers who need to construct nuanced designs to answer their specific research questions. Schoonenboom and Johnson also showed that, when designing research, multiple design dimensions should be considered to produce the

Dialectical Pluralism & Integration in Mixed Methods Research

119

best package of design features for the particular study. They discussed seven primary dimensions and eight secondary dimensions. From a team perspective, the different members of the team should dialogue with each other regarding the relevant design dimensions and jointly construct an agreedupon research design. For a third example of methodological pluralism, Johnson and Schoonenboom teamed with Federica Russo, who is a philosopher of science and causal pluralism. One of Russo’s goals has been to facilitate dialogue about causation between philosophers of science and practitioners of science. We published an article titled ‘Causation in Mixed Methods Research: The Meeting of Philosophy, Science, and Practice’ (Johnson et al., 2019). We reviewed causation concepts, causation theories, methodological principles, and causal criteria used in quantitative and qualitative research. Researchers are to put together a causal mosaic, puzzle, or ‘causal package’ of ideas relevant for each research study interested in studying causation. They answer this question: ‘What type(s) of causation and what causal standards do we think are relevant for our research study?’ The pluralist causal theory was summarized in 11 propositions. The pluralist causal theory is especially important for MMR, because, in practice, researchers often need to make justified claims about causation using qualitative, quantitative, and mixed data.

• The DP methodological principle: Researchers and stakeholders should dialectically listen to and consider multiple methodological concepts, issues, and inquiry logics, and construct the appropriate mix for each research study and its research questions. Please take a moment now to examine Table 6.2. It summarizes the philosophical and methodological ‘commitments’ or positions of dialectical pluralism. TABLE 6.2 Philosophical/Methodological ‘Commitments’ of Dialectical Pluralism With regard to philosophy and methodology, dialectical pluralism . . . • Is a process philosophy for examining differences we need to interact with and learn from. • Relies on pluralism, dialogue, and integration. • Purposively and systematically creates combinations and integrations of philosophies, values, methods, and methodologies on a project-by-project basis. • Relies on paradigmatic pluralism. • Consider, carefully listen to, and when needed, combine, and integrate ideas and approaches found in different paradigms to meet the needs of a particular research study. • Relies on ontological pluralism. • Appreciates and dialogues with multiple ontological issues and theories. • Examines and empirically studies multiple kinds or domains of reality.

(Continued)

120

R. Burke Johnson

TABLE 6.2 (Continued) • Relies on epistemological pluralism. • Appreciates and dialogues with multiple epistemological issues and theories. • Emphasizes two key logics: logic of combination (‘both-and’) and logic of integration (e.g. thesis, antithesis, integration/synthesis). • Relies on axiological/values pluralism. • Appreciates and dialogues with multiple values and ethical theories and concepts. • Considers and integrates parts from multiple ethical theories, social values, practical values, epistemic ends, and epistemic virtues to help produce an appropriate mix or ‘values package’ of what is relevant for each research study or research project. • Relies on methods pluralism. • Dialogues with multiple methods (e.g. methods of data collection, sampling, analysis) and to produce an appropriate mix or ‘methods package’ on a study-by-study basis. • Relies on methodological pluralism. • Considers, dialogues with, and combines or integrates parts of two of more research methodologies. • Separate methodologies can be used in separate strands and/or methodologies can be combined into a mixed methods methodology, such as a mixed methods-experimental methodology (MM-EXP), a mixed methods-grounded theory methodology (MM-GT), or any other MM combination of methodologies.

6.4 Conclusion

Dialectical pluralism is a metaparadigm for interacting with differences and tensions and producing integrated knowledge and practices. Pluralism and dialogue are its fundamental principles. DP provides a process for producing integrations or syntheses from differences. It helps produce collaborative visions and decisions, win-win solutions, and superior empirical research outcomes and acceptance. Strategies are provided for individuals to use as they continually develop and update their individual mental models. Strategies are also provided for groups, teams, and communities-of-practice to continually develop team or collective mental models. I discuss, in detail, the philosophical ‘commitments’ of DP. Notes 1 Some classical social psychologists impacting my work are Kurt Lewin, John Dewey, Morton Deutsch, George Herbert Mead, Rensis Likert, and Marvin E. Shaw. 2 Tony Onwuegbuzie and I  originally labelled this socio-political legitimation (Onwuegbuzie & Johnson, 2006) 3 Onwuegbuzie and I originally labelled this inside–outside legitimation (Onwuegbuzie & Johnson, 2006).

References Abelard, P. (1981). The Letters of Abelard and Heloise (B. Radice, Trans.). London: Penguin.

Dialectical Pluralism & Integration in Mixed Methods Research

121

Archer, M., Bhaskar, R., Collier, A., Lawson, T.,  & Norrie, A. (Eds.). (1998). Critical Realism: Essential Readings. Abingdon: Routledge. Bazeley, P. (2018). Integrating Analyses in Mixed Methods Research. London: Los Angeles: Sage. Bennett, L. M., Gadlin, H., & Levine-Finley, S. (2010). Collaboration & Team Science: A Field Guide. Washington, DC: National Institutes of Health. Biesta, J. J., & Burbules, N. C. (2003). Pragmatism and Educational Research. Lanham, MD: Rowman & Littlefield. Binne, V., Le Brocque, R., Jessup, M., & Johnston, A. N. B. (2021). Illustrating a novel methodology and paradigm applied to emergency department research. Journal of Advanced Nursing, 77, 4045–4054. Br’es, L., Raufflet, E.,  & Boghossain, J. (2018). Pluralism in organizations: Learning from unconventional forms of organizations. International Journal of Management Reviews, 20, 364–386. Coleman, P. T., Deutsch, M., & Marcus, E. C. (Eds.). (2014). The Handbook of Conflict Resolution: Theory and Practice (3rd ed.). San Francisco, CA: Jossey-Bass. Collay, M. P., Tucker, S. A., Korb, M. A., Distefano, R., & Johnson, R. B. (2012). Conceptualizing the “Conceptualization Stage” of an Ecology Based Transformative, Multipartner, Mixed Methods Research Project. Paper presented at the American Evaluation Association, San Francisco, USA, April 16. Cook, T. D. (1985). Post-positivist critical multiplism. In L. Shotland  & M. M. Mark (Eds.), Social Science and Social Policy (pp. 21–62). Newbury Park, CA: Sage. Cook, T. D., & Reichardt, C. S. (Eds.). (1979). Qualitative and Quantitative Methods in Evaluation Research. Beverly Hills, CA: Jossey-Bass. Copi, I. M., Cohen, R., & Rodych, V. (2019). Logic (15th ed.). New York, NY: Routledge. Corbin, J., & Strauss, A. (2015). Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. Thousand Oaks, CA: Sage. Cournoyea, M.,  & Kennedy, A. G. (2014). Causal explanatory pluralism and medically unexplained physical symptoms. Journal of Evaluation in Clinical Practice, 20, 928–933. Crasnow, S. (2019). Political science methodology: A  plea for pluralism. Studies in History and Philosophy of Science, 78, 40–47. Creamer, E. C. (2022). Advancing Grounded Theory with Mixed Methods. New York, NY: Routledge. Creswell, J. W., & Plano Clark, V. (2018). Designing and Conducting Mixed Methods Research (3rd ed.). Los Angeles, CA: Sage. Dale, R., Dietrich, E., & Chemero, A. (2009). Explanatory pluralism in cognitive science. Cognitive Science, 33, 739–742. Dencer-Brown, A. M., Alfaro, A. C., & Milne, S. (2019). Muddied waters: Perceptions and attitudes towards mangroves and their removal in New Zealand. Sustainability, 11, 1–33. Deutsch, M. (2006). Cooperation and competition. In M. Deutsch, P. T. Coleman, & E. C. Marcus (Eds.), The Handbook of Conflict Resolution: Theory and Practice (2nd ed.). (pp. 23–42). San Francisco, CA: Jossey-Bass. Dewey, J. (1888). The ethics of democracy. In Jo Ann Boydston (Ed.), The Early Works of John Dewey, V.1 1882–1898 (vol. 1, pp.  227–250). Carbondale, IL: Southern Illinois Press. Dewey, J. (1916). Democracy and Education. New York, NY: Macmillan Co. Dewey, J. (1920). Reconstruction in Philosophy. New York, NY: Henry Holt and Company. Dewey, J. (1927). The Public and Its Problems. New York, NY: Henry Holt and Company.

122

R. Burke Johnson

Dewey, J. (1931). The development of American Pragmatism. In Jo Ann Boydston (Ed.), The Later Works of John Dewey, Volume 2 1925–1927 (pp. 3–21). Carbondale, IL: Southern Illinois Press. Dewey, J. (1939). Experience, knowledge and value: A Rejoinder. In J. A. Boydston (Ed.), The Later Works of John Dewey, Volume 14 1939–1941 (pp. 3–90). Carbondale, IL: Southern Illinois Press. Dues, M. (2010). The Art of Conflict Management: Achieving Solutions for Life, Work, and Beyond. Chantilly, VA: The Teaching Company. Edwards, S. J. L., Boch, T., Palm, U., Wang, S., Cheng, G., Wang, L., & Pitts, P. (2020). The case for methodological pluralism in medical science. The American Journal of Bioethics, 20(9), 39–41. Engell, T., Løvstad, A. M., Kirkøen, B., Ogden, T., & Hagen, K. A. (2021). Exploring how intervention characteristics affect implementability: A  mixed methods case study of common elements-based academic support in child welfare services. Children and Youth Services Review, 129, 1–12. Feldman, R. (2003). Epistemology. Upper Saddle River, NJ: Prentice Hall. Fetters, M. D., Curry, L. A., & Creswell, J. W. (2013). Achieving integration in mixed methods designs: Principles and practices. Health Services Research, 48(6pt2), 2134–2156. Fetters, M. D., & Molina-Azorin, J. F. (2017). The journal of mixed methods research starts a new decade: The mixed methods research integration trilogy and its dimensions. Journal of Mixed Methods Research, 11(3), 291–307. Fisher, R., & Ury, W., & Patton, B. (2011). Getting to Yes: Negotiating Agreement Without Giving in. New York, NY: Penguin Books. Gage, N. L. (1989). The paradigm wars and their aftermath: A  historical sketch of research on teaching since 1989. Educational Researcher, 18(7), 4–10. Garver, E. (1990). Systematic pluralism. The Monist, 73(3), 388–410. Giere, R. N. (2006). Scientific Perspectivism. Chicago, IL: University of Chicago Press. Glaser, B., & Strass, A. (1967). The Discovery of Grounded Theory. Hawthorne, NY: Aldine. Goldman, S. L. (2022). Science Wars: The Battle Over Knowledge and Reality. Oxford: Oxford University Press. Greene, J. C. (1990). Three views on the nature and role of knowledge in social science. In E. G. Guba (Ed.), The Paradigm Dialog (pp. 227–245). Newbury Park, CA: Sage. Greene, J. C. (2000). Challenges in practicing deliberative democratic evaluation. In K. A. Ryan & L. DeStefano (Eds.), Evaluation as a Democratic Process: Promoting Inclusion, Dialogue, and Deliberation (pp. 13–26). San Francisco, CA: Jossey-Bass. Greene, J. C. (2005). The generative of mixed methods inquiry. International Journal of Research & Method in Education, 28(2), 207–211. Greene, J. C. (2007). Mixed Methods in Social Inquiry. San Francisco, CA: Jossey-Bass. Greene, J. C. (2015). Preserving distinctions within multimethod and mixed methods research. In S. Hesse-Biber & R. B. Johnson (Eds.), The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry (pp.  604–615). Oxford: Oxford University Press. Greene, J. C., & Caracelli, V. J. (Eds.). (1997). Advances in Mixed-Method Evaluation: The Challenges and Benefits of Integrating Diverse Paradigms. San Francisco, CA: Jossey-Bass.

Dialectical Pluralism & Integration in Mixed Methods Research

123

Greene, J. C., & Hall, J. N. (2010). Dialectics and pragmatism: Being of consequence. In A. Tashakkori & C. Teddlie (Eds.), Sage Handbook of Mixed Methods in Social & Behavioral Science (pp. 119–143). Los Angeles, CA: Sage. Guba, E. G. (1990). The alternative paradigm dialogue. In E. G. Guba (Ed.), The paradigm dialog (pp. 17–27). Newbury Park, CA: Sage. Hammersley, M. (1992). The paradigm wars: Reports from the front. British Journal of Sociology of Education, 13(1), 131–143. Hesse-Biber, S. N., & Johnson, R. B. (Eds.). (2015). Oxford Handbook of Multimethod and Mixed Methods Research Inquiry. Oxford: Oxford University Press. Hitchcock, J. H., & Onwuegbuzie, A. J. (2022). The Routledge Handbook for Advancing Integration in Mixed Methods Research. New York, NY: Routledge. House, R., & Howe, K. R. (1999). Values in Evaluation and Social Research. Thousand Oaks, CA: Sage. Jerotic, S., & Aftab, A. (2021). Scientific pluralism is the only way forward for psychiatry. Acta Psychiatrica Scandinavica, 143, 537–538. Johnson, R. B. (1998). Toward a theoretical model of evaluation utilization. Evaluation and Program Planning: An International Journal, 21, 93–110. Johnson, R. B. (2008a). Knowledge. In L. M. Given (Ed.), The Sage Encyclopedia of Qualitative Research Methods (pp. 478–482). Los Angeles, CA: Sage. Johnson, R. B. (2008b). Associate editor’s editorial: Living with tensions. Journal of Mixed Methods Research, 2, 203–207. Johnson, R. B. (2008c). Living with tensions: The dialectic approach. Journal of Mixed Methods Research, 2(3), 203–207. Johnson, R. B. (2011a). Dialectical Pluralism: A Metaparadigm to Help Us Hear and ‘Combine’ Our Valued Differences. Paper presented in plenary session at the Seventh International Congress of Qualitative Inquiry, Urbana-Champaign, IL, May. Johnson, R. B. (2011b). Do we need paradigms? A mixed methods perspective. MidWestern Educational Researcher, 24(2), 31–40. Johnson, R. B. (2012). Guest editor’s editorial: Dialectical pluralism and mixed research. American Behavioral Scientist, 56, 751–754. Johnson, R. B. (2015). Toward an inclusive and defensible multi and mixed science. In S. J. N. Hesse-Biber & R. B. Johnson (Eds.), The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry (pp.  688–706). Oxford: Oxford University Press. Johnson, R. B. (2017). Dialectical pluralism: A metaparadigm whose time has come. Journal of Mixed Methods Research, 11(2), 156–173. Johnson, R. B., & Christensen, L. B. (2020). Educational Research Methods: Quantitative, Qualitative, and Mixed Approaches (7th ed.). Los Angeles, CA: Sage. Johnson, R. B., & Christensen, L. B. (2024). Educational Research Methods: Quantitative, Qualitative, and Mixed Approaches (8th ed.). Los Angeles, CA: Sage. Johnson, R. B.,  & Onwuegbuzie, A. J. (2004). Mixed methods research: A  research paradigm whose time has come. Educational Researcher, 33(7), 14–26. Johnson, R. B., de Waal, C., Stefurak, T., & Hildebrand, D. (2017). Understanding the philosophical positions of classical and neopragmatists for mixed methods research. Kölner Zeitschrift für Soziologie und Sozialpsychologie (Cologne Journal for Sociology and Social Psychology), 69(2), 63–86. Open source. http://rdcu.be/tXiS. Johnson, R. B., Meeker, K., Loomis, E.,  & Onwuegbuzie, T. J. (2004). Development of the Philosophical and Methodological Beliefs Inventory. Paper presented at the 2004 American Educational Research Association in Chicago.

124

R. Burke Johnson

Johnson, R. B., Onwuegbuzie, A. J., de Waal, C., Stefurak, T., & Hildebrand, D. (2017). Unpacking pragmatism for mixed methods research: The philosophies of Peirce, James, Dewey, and Rorty. In The BERA/SAGE Handbook of Educational Research (pp. 259–279). London: Sage. Johnson, R. B., Onwuegbuzie, A. J., Tucker, S., & Icenogle, M. L. (2014). Conducting mixed methods research using dialectical pluralism and social psychological strategies. In P. Leavy (Ed.), The Oxford Handbook of Qualitative Research (pp. 557– 580). Oxford: Oxford University Press. Johnson, R. B., Onwuegbuzie, A. J., & Turner, L. A. (2007). Toward a definition mixed methods research. Journal of Mixed Methods Research, 1(2), 112–133. Johnson, R. B., Russo, F., & Schoonenboom, J. (2019). Causation in mixed methods research: The meeting of philosophy, science, and practice. Journal of Mixed Methods Research, 13, 143–162. Johnson, R. B.,  & Stefurak, T. (2013). Considering the evidence-and-credibility discussion in evaluation through the lens of dialectical pluralism. In D. Mertens  & S. Hesse-Biber (Eds.), Mixed Methods and Credibility of Evidence in Evaluation (pp. 37–48). San Francisco, CA: Wiley. Johnson, R. B., & Stefurak, J. (2014). Dialectical Pluralism: A metaparadigm and process philosophy for “dynamically combining” important differences. Qualitative Methods in Psychology (QMiP) Bulletin, 17, Spring, 63–69. Johnson, R. B.,  & Turner, L. A. (2003). Data collection strategies in mixed methods research. In A. Tashakkori  & C. Teddlie (Eds.), Handbook of Mixed Methods in Social and Behavioral Research (pp. 297–319). Thousand Oaks, CA: Sage. Jordan, G., Davidson, L., & Bellamy, C. (2022). Generativity among people with lived experience of mental illness and distress. American Journal of Orthopsychiatry, 92(3), 280–290. Kelly, T. (2013). Peer disagreement and higher-order evidence. In R. Feldman & T. A. Warfield (Eds.), Disagreement (pp. 111–174). Oxford: Oxford University Press. Kuhn, T. S. (1962). The Structure of Scientific Revolutions. Chicago, IL: University of Chicago Press. Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic Inquiry. Thousand Oaks, CA: Sage. Maung, H. H. (2020). Pluralism and incommensurability in suicide research. Studies in History and Philosophy of Biol & Biomed Science, 80, 1–10. Maxwell, J. A. (2012). A Realistic Approach for Qualitative Research. Los Angeles, CA: Sage. Maxwell, J. A.,  & Mittapalli, K. (2010). Realism as a stance for mixed methods research. In A. Tashakkori & C. Teddlie (Eds.), Mixed Methods in Social & Behavioral Research (2nd ed., pp. 145–167). Los Angeles, CA: Sage. Mejeh, M., Hagenauer, G., & Gläser-Zikuda, M. (2023). Mixed methods research on learning and instruction—Meeting the challenges of multiple perspectives and levels within a complex field. Forum: Qualitative Social Research/Sozialforschung, 24(1), Article 14. Mertens, D. M. (2007). Transformative paradigm: Mixed methods and social justice. Journal of Mixed Methods Research, 1(3), 212–225. Mertens, D. M. (2012). Transformative mixed methods: Addressing inequities. American Behavioral Scientist, 56(6), 803–813. Moneta, A., & Russo, F. (2014). Causal models and evidential pluralism in economics. Journal of Economic Methodology, 21(1), 54–76.

Dialectical Pluralism & Integration in Mixed Methods Research

125

Neves, B. B.,  & Baecker, R. (2020). Mixing methods and sciences: A  longitudinal cross-disciplinary mixed methods study on technology to address social isolation and loneliness in later life. Journal of Mixed Methods Research, 16(1), 88–113. Novis-Deutsch, N. (2020). Pluralism as an antidote to epistemic violence in psychological research. Theory & Psychology, 30(3), 409–413. Onwuegbuzie, A. J., & Combs, J. P. (2010). Data analysis in mixed research: A primer. In A. Tashakkori & C. Teddlie (Eds.), SAGE Handbook of Mixed Methods in Social & Behavioral Research (2nd ed., pp. 397–430). Los Angeles, CA: Sage. Onwuegbuzie, A. J., & Johnson, R. B. (2006). The “validity” issue in mixed methods research. Research in the Schools, 13(1), 48–63. Onwuegbuzie, A. J., & Johnson, R. B. (Eds.). (2021). The Routledge Reviewer’s Guide for Mixed Methods Analysis. New York: NY: Routledge. Overeem, P., & Verhoef, J. (2015). Value pluralism and the usefulness of philosophical theory for public administration. Administration & Society, 47(9), 1103–1109. Patton, M. Q. (2015). Qualitative Research & Evaluation Methods. Los Angeles, CA: Sage. Pawson, R. (2016). Evidence-Based Policy: A Realist Perspective. Los Angeles, CA: Sage. Peirce, C. S. (1998/1893). Immortality in the light of synechism. In The essential Peirce (pp. 1–10). Bloomington, IN: Indiana University Press. Putnam, H. (1987). The Many Faces of Realism. LaSalle, IL: Open Court. Putnam, H. (2002). The Collapse of the Fact/Value Dichotomy and Other Essays. Cambridge, MA: Harvard University Press. Quine, W. V. O. (1980). From a Logical Point of View. Cambridge, MA: Harvard University Press. Rawls, J. (1999). The Law of Peoples. Cambridge, MA: Harvard University Press. Rescher, N. (2000). Process Philosophy: A Survey of Basic Issues. Pittsburgh, PA: University of Pittsburgh Press. Robbins, S. P. (1978). “Conflict management” and “conflict resolution” are not synonymous terms. California Management Review, 21(2), 67–75. Scheffler, I. (2009). Worlds of Truth: A  Philosophy of Knowledge. West Sussex: Wiley-Blackwell. Schoonenboom, J.,  & Johnson, R. B. (2017). How to construct a mixed methods research design. Kölner Zeitschrift für Soziologie und Sozialpsychologie (Cologne Journal for Sociology and Social Psychology), 69(2), 107–131. Open-source. http:// rdcu.be/tXiS. Schormair, M. J. L., & Gilbert, D. U. (2021). Creating value by sharing values: Managing stakeholder values conflict in the fact of pluralism through discursive justification. Business Ethics Quarterly, 31(1), 1–36. Searle, J. R. (2008). Language and social ontology. Theory and Society, 37, 444–459. Sechrest, L. (Ed.). (1993). Program Evaluation: A Pluralistic Enterprise. San Francisco, CA: Jossey-Bass. Shadish, W. R. (1986). Planned critical multiplism: Some elaborations. Behavioral Assessments, 8, 75–103. Sheehan, M. (2011). Probing the philosophical beliefs of instructional design faculty and professionals (Unpublished doctoral dissertation). Sheehan, M., & Johnson, R. B. (2012). Philosophical and methodological beliefs of instructional design faculty and professionals. Educational Technology Research and Development, 60, 131–153.

126

R. Burke Johnson

Shim, M., Johnson, R. B., Gasson, S., Goodill, S., Jermyn, R.,  & Bract, J. (2017). A model of dance/movement therapy for resilience-building in people living with chronic pain. European Journal of Integrative Medicine, 9, 27–40. Shim, M., Johnson, R. B., Gasson, S., Goodill, S., Jermyn, R.,  & Bract, J. (2020). A mixed methods-grounded theory design for producing more refined theoretical models. Journal of Mixed Methods Research, 15(1), 61–86. Sivaramakrishnan, H., Quested, E., Cheval, B., Thøgersen-Ntoumani, C., Gucciardi, D. F., & Ntoumanis, N. (2023). Predictors of intentions of adults over 35 years to participate in walking sport programs: A social-ecological mixed-methods approach. Scandinavian Journal of Medicine & Science in Sports, 33(8), 1–19. Stefurak, T., Dixon, V. S., & Johnson, R. B. (2023). Dialectical pluralism in counseling and psychotherapy research. In S. Bager-Charleson & A. McBeath (Eds.), Supporting Research in Counselling and Psychotherapy (pp.  207–227). Cham: Palgrave Macmillan. Stefurak, T., Johnson, R. B., Shatto, E. H., & Jones, K. (2018). Developing and evaluating social programs using dialectical pluralism: Three case studies of youth placed at-risk. International Journal of Multiple Research Approaches, 10, 1–16. Stuhr, J. J. (2020). Truth, truths, and pluralism. Journal of Speculative Philosophy, 34(4), 526–544. Tashakkori, A., Johnson, R. B., Teddlie, C. B. (2021). Foundations of Mixed Method Research: Integrating Quantitative and Qualitative Approaches in the Social and Behavioral Sciences (2nd ed.). Los Angeles, CA: Sage. Tashakkori, A., & Teddlie, C. (Eds.). (1998). Mixed Methodology: Combining Qualitative and Quantitative Approaches. Thousand Oaks, CA: Sage. Tashakkori, A., & Teddlie, C. (Eds.). (2003). Handbook of Mixed Methods in Social and Behavioral Research. Thousand Oaks, CA: Sage. Tashakkori, A., & Teddlie, C. (Eds.). (2010). Handbook of Mixed Methods in Social and Behavioral Research (2nd ed.). Los Angeles, CA: Sage. Thorne, B.,  & O’Reilly, M. (2022). Operationalizing strategic objectives of suicide prevention policy: Police-let LOSST LIFFE model. Death Studies, 46, 2077–2084. Tucker, S. A., Johnson, R. B., Onwuegbuzie, A. J., & Icenogle, M. L. (2020). Conducting mixed methods research using dialectical pluralism and social psychological strategies. In P. Leavy (Ed.), The Oxford Handbook of Qualitative Research (2nd ed., pp. 836–874). Oxford: Oxford University Press. Tucker, S. A., & Williams, A. D. (2019). Developing Preservice Teacher Equity-Based Dispositions: Narratives of Connecting with Evidence by Practitioners, Researchers, and Evaluators. Paper presented at the American Evaluation Association, Toronto, Canada, April 7. Watson, W. (1990). Types of pluralism. Monist, 73, 350–366. Weber, M. (2021/1920). Basic Concepts in Sociology. New York, NY: Hassell Street Press. Wittgenstein, L. (2009/1953). Philosophical Investigations. Hoboken, NJ: Wiley-Blackwell. Zangwill, N. (2020). Epistemic pluralism: The missing link and the ambitions of epistemology. Metaphilosophy, 51(4), 485–498.

7 A PERFORMATIVE APPROACH TO MIXED METHODS RESEARCH Judith Schoonenboom

7.1 A Performative Approach to Mixed Methods Research

For a long time, mixed methods scholars have discussed how mixed methods research relates to “reality.” Does “the world” exist independent of our observations? It does in the critical realist perspective (Maxwell, 2012; Maxwell & Mittapalli, 2010), which assumes that researchers study various perspectives on this independent world. Or do different worlds exist, as assumed in dialectical perspectives on mixed methods research (Greene, 2007, 2015; Greene & Hall, 2010; Johnson, 2015, 2023)? In this discussion, a performative approach brings a new perspective. It extends the dialectical perspective by stating that worlds do not exist independent of our research but come into being through our research methods and concepts (Barad, 2007; Law, 2004; Pickering, 1995; Putnam, 1987). Through their methods and concepts, researchers bring a phenomenon and the world in which it exists into being. Consequently, different methods create different worlds or, to use a technical term, “research assemblages” (Coleman & Ringrose, 2022; Deleuze & Guattari, 1988; Fox & Alldred, 2015, 2018; Law, 2004). Consider the following utterance by a first-year university student, which has been taken from a series of interviews: “I  understand 80% of the lectures. Careful reading complements 20% of lack of understanding” (Lee & Greene, 2007). This utterance can be analysed in various ways, and each method of data analysis constitutes its own research world in which the utterance is embedded in a particular way. The research world of discourse analysis considers interviews as sequences of questions and answers. In this research world, the utterance is viewed as an answer to an interview question, and we can analyse how the utterance relates to the question it aims to answer and how it, in turn, is DOI: 10.4324/9781003273288-8 This chapter has been made available under a CC-BY-NC-ND 4.0 license.

128

Judith Schoonenboom

followed by a follow-up question. The research world of thematic analysis considers interviews and other texts as containers of content, and we can analyse the themes that are present in the utterance. In a thematic analysis, we could code the utterance as describing a “compensation strategy.” We can compare it with other utterances that describe “compensation strategies” to see where they agree and differ. The research world of narrative analysis consists of stories. In a narrative analysis, we could view the utterance as the beginning of a story in which the student explains how careful reading worked for them as a compensation strategy. These three research assemblages – discourse analysis, thematic analysis, and narrative analysis – are different worlds, because they contain different inhabitants and interactions. In each of these worlds, the utterance is embedded differently. Notably, the utterance in these three research worlds is not the same thing; we have three closely related yet different objects (Mol, 2002). In other words, these different methods bring different research worlds into being with different inhabitants and interactions. This chapter explores the foundational idea of a performative approach that different research worlds come into being through our methods and concepts. It has two different objectives. The first is to show that a performative approach can form the foundation for all research. Throughout this chapter, readers will recognize forms of data collection and data analysis that have been classified as postpositivist (questionnaires) or constructivist (interviews). A performative approach recognizes these different types of research yet views them all as acts that bring research worlds into being. The second objective is to show how a performative approach can form a foundation for mixed methods research. Here, we argue that a performative approach recognizes the differences between different research worlds and builds on them in a “performative” way. Section  7.2 describes an ontology of research worlds: their inhabitants, interactions, events, boundaries, fluidity, and how they come into being. One element is essential to research worlds: their end products, most notably verbal statements (theory). In research, worlds and their end products are coordinated (Mol, 2002), which is discussed in Section  7.3. Section  7.4 turns to mixed methods research, describing the consequences of a performative approach for mixed methods research and its coordinating research processes. Finally, Section 7.5 reflects on the contribution of a performative approach to mixed methods research. 7.2 An Ontology of Research Worlds 7.2.1 Introduction

This section describes an ontology of research worlds in a performative approach. What is a world? The simple answer is that a world has living and

A Performative Approach to Mixed Methods Research

129

nonliving inhabitants, and we can ask the following: What is happening in this world? What are its events? How do the inhabitants interact? This starting point does not differ from what we in everyday life would call a world. Let’s look at some research worlds in the following example from Visser et al. (2018): Example 1

A study conducted by Visser et al. (2018) examined academic procrastination (delaying study tasks) among first-year students in an elementary teacher education program in the Netherlands. A total of 186 students out of 215 completed the Academic Procrastination State Inventory (APSI), which assessed procrastination levels. In the following interview process, 22 students were interviewed: 8 with low procrastination, 8 with average procrastination, and 6 with high procrastination. The interviews revealed that students with low and average procrastination levels were highly motivated to become teachers. This strong motivation enabled students with an average procrastination level to continue studying, even when they disliked a task. Facing task aversion, students with high procrastination levels discontinued studying because they lacked this strong motivation. Example 1 contains several worlds. One of them is the world of students with an average procrastination level. We do not know much about this world. Still, we do know that it is inhabited by students with an average procrastination level as living inhabitants and study tasks as nonliving inhabitants. The events in this world comprise the activity of studying, in which a student interacts with a study task. More specifically, we learn how students with an average procrastination level interact with study tasks they dislike: they continue studying, because their motivation to become a teacher is high. The world of students with an average procrastination level differs distinctly from the world of students with high procrastination levels. A  decisive distinction is in their interaction with tasks they dislike. Students with high procrastination levels discontinue studying and do not complete disliked tasks because they lack a strong motivation to become a teacher. In addition to the different worlds of the students, Example 1 also contains different research practices (Mol, 2002) related to the object of “procrastination.” Procrastination can be studied in many ways, and each way constitutes a different research practice with its own procedures and rules. The first practice in Example 1 is “diagnosing procrastination using the APSI questionnaire.” This research world is inhabited by students and forms that interact in an event we could call “filling out the APSI questionnaire,” which is followed by a diagnostic event. “Filling out the APSI questionnaire” differs from the subsequent research world, “interviewing,” in which students tell the researcher about their procrastination experiences. Other possible practices

130

Judith Schoonenboom

of studying procrastination are absent, such as observing procrastination when students attempt to study. Finally, Example 1 contains a textual world, a summary of Visser et  al. (2018). A textual example has its own rhetorical and stylistic rules, which may differ from those of other textual worlds. These worlds are only a tiny fraction of the endless possibilities, and we will encounter more worlds later. Worlds are characterized by boundaries. We can distinguish the worlds of students with high procrastination levels from those of students with middle procrastination levels, and we can distinguish the world of “diagnosing procrastination” from, for example, “experiencing procrastination while studying,” “diagnosing dyslexia,” or “exercising at the gym.” These boundaries are fuzzy and may change, as we will see later.

7.2.2 Worlds Are Fluent and May Change

In addition to being populated by inhabitants and events and having boundaries, worlds have another characteristic in a performative approach: they are fluid. Exploring the interviews in Example 1, we notice that these students’ worlds are not static. We learn that a “strong motivation to become a teacher” is fed by events, is strengthened by positive experiences in class, and becomes more elaborate as the student continues – or declines because of negative experiences. These worlds of motivational experiences are fluid and will most likely have changed the next time a student is interviewed. It would be a mistake to think that worlds are organisms that would remain the same without forces from the outside causing them to change. In contrast, change is the natural state of worlds because of the way they exist. Worlds are maintained through a continuous chain of events consisting of their inhabitants’ internal and external interactions. In technical terms, worlds and their inhabitants are continuously “enacted” and “re-enacted” (Law, 2004). In Example 1, each student’s motivation to become a teacher is shaped and modified by ongoing experiences. Consequently, the ontology of worlds is a process ontology, in which everything is in constant flux (Rescher, 1996; Seibt, 2022). Therefore, at any point in time, the world we observe is nothing but a temporary snapshot. The worlds of research practices are also fluid. Over time, items of the APSI questionnaire may be changed, replaced, or added as new triggers for procrastination enter student life (social media), while others disappear (broken feather pen). Furthermore, as the student population changes, a specific score on the APSI may mean something different in 2028 than it did in 2018. The APSI may even be replaced by a different instrument altogether, and the practice “diagnosing procrastination using the APSI” may cease to exist.

A Performative Approach to Mixed Methods Research

131

7.2.3 How Worlds Come Into Being

This description of worlds as fluid constellations that are enacted and reenacted through the interactions between their living and nonliving inhabitants disregards one crucial moment in the life of a world: how it comes into being. Worlds can come into being in many ways. Most relevant to research worlds is when worlds are created, something we could call a “performative act.” Various types of performative acts of creating worlds have been described in the philosophical and scientific literature. One is to create a world through a performative speech act (Austin, 1962). For example, the world of a married couple used to be created by the words “I pronounce you man and wife,” uttered by an authorized person. As another example, the pandemic came into being when the WHO’s Director-General, Dr. Tedros Adhanom Ghebreyesus, declared it on March 11, 2020. In bringing research worlds into being, our concepts and methods, especially our methods of data collection, play an important role. Thus, the world of students with high procrastination levels in Example 1 would not exist without the APSI, the interviewer, or the interview schedule. These human and nonhuman actors work together to create a world. Together, they constitute the world; none of these actors can be removed. In technical terms, their contributions are entangled (Barad, 2007), and the world cannot be “disentangled” into separate parts. For example, we can reflect on the role of the interviewer and interview guidelines in constituting high-level procrastination worlds, but we cannot remove them. Our methods of data analysis create further research worlds. As we saw in the Introduction, different methods of data analysis – discourse analysis, thematic analysis, and narrative analysis – create three different research worlds in which one specific utterance has a different embedding and, accordingly, constitutes three different yet related objects. More generally, we can say that specific worlds, whether research or nonresearch worlds, cannot exist without the appropriate means to create them. Thus, in pre-Cambrian times, living creatures lacked the ability to perform study tasks written on paper because they would not be able to see them. Only during the Cambrian period did several species acquire the ability to use light as a source of information by developing eyesight (Halliday, 2022). Without eyesight, study tasks written on paper could not exist. 7.2.4 End Products of Research Worlds: An Ontology of Statements

An essential characteristic of research worlds is that they are not created for their own sake but to produce some end result; they are “machines” (Deleuze & Guattari, 1988; Fox & Alldred, 2015). In Example 1’s practice of “diagnosing procrastination,” the APSI questionnaire is not filled out for its

132

Judith Schoonenboom

own sake but rather to obtain a procrastination score. This distinguishes “diagnosing procrastination” from, for instance, filling out a sudoku or crossword puzzle. In technical terms, in the practice of “diagnosing procrastination,” filling out the APSI questionnaire is “territorialized” (Deleuze & Guattari, 1988), because it is used by someone to obtain a goal that lies outside filling out the questionnaire itself. In research, one end product of research worlds occupies a central position: research statements. In Example 1, various interwoven statements play a role, including “students with average and high procrastination levels experienced task aversion,” “students with low and average procrastination levels had a strong intrinsic motivation to become a teacher,” and “students with high procrastination levels procrastinated on averse tasks because they lacked a strong intrinsic motivation to become a teacher.” Research statements have different statuses. In Example 1, “high intrinsic motivation reduces academic procrastination” and “task aversion leads to procrastination” are accepted statements, which are mentioned as existing and apparently uncontroversial statements in the Introduction (Visser et al., 2018, p. 3). Accepted statements are the opposite of controversial statements, which are accepted by some researchers but not by others; we will discuss these later. Tentative statements are neither accepted nor contested by others. The outcomes of a study are often tentative statements, such as Example 1’s “Students with high procrastination levels procrastinated on averse tasks because they lacked a strong intrinsic motivation to become a teacher.” In addition, this statement is an inference of the study, because it was developed by Visser et  al. (2018). This distinguishes this tentative statement from the accepted statements mentioned earlier, which have been borrowed from other studies. In summary, research statements can be classified on at least two dimensions: status and origin, with their status being accepted, controversial, or tentative, and their origin being an inference developed in the study or a statement borrowed from elsewhere. Furthermore, research statements have different forms. “High intrinsic motivation reduces academic procrastination” is a simple statement, because it contains one subject, “high intrinsic motivation,” and one predicate, “reduces academic procrastination.” It is also an unconditional statement because the reduction of academic procrastination is presented as generally occurring when high intrinsic motivation is present. Similarly, “Task aversion leads to procrastination” is a simple, unconditional statement because it contains one subject (task aversion) and one predicate (leads to procrastination), and academic procrastination is presented as generally occurring when task aversion is present. Statements can also be conditional. An example is “Students with high procrastination levels procrastinated on averse tasks because they lacked a strong intrinsic motivation to become a teacher,” which was developed by Visser

A Performative Approach to Mixed Methods Research

133

et al. (2018). Conditional statements are complex, because they contain more than one subject–predicate structure. They are conditional because they consist of a statement, in this case, “task aversion leads to procrastination,” and a condition under which this statement does (or does not) apply, namely, “when the student lacks a strong intrinsic motivation to become a teacher.” In summary, Section  7.2 presents a performative ontology for research worlds. Research worlds have boundaries and inhabitants, some of whom are present, whereas others are absent. Worlds are fluid, because they are enacted and re-enacted through a continuous chain of events and interactions. A research world aims to produce an end product. One essential end product of research worlds is statements, which can have various statuses and forms. 7.3 Coordinating Research Worlds 7.3.1 Coordinating Worlds

Doing research is more than creating different worlds. These worlds must be related to each other; they must be “coordinated” (Mol, 2002). In this chapter, I  define coordinating research worlds as using the end product of one or more research worlds to create a new research world. In this section, I  discuss two types of research coordination: coordination of the research process (research design) and coordination of research outcomes (connecting inferences). Research design and connecting inferences are not automatic processes, because previous design decisions do not force later ones, and findings do not force conclusions. Each research process consists of a chain of decisions that are coordinated to work together. A  research study is an “arrangement of machines” (Fox & Alldred, 2015). 7.3.2 Coordinating the Research Process

Research processes are coordinated through the end products that each research world produces. In Example 1, “filling out the APSI questionnaire” is coordinated with “sampling.” The former’s end product – the filled-out APSI questionnaires – is used to create new worlds of groups of students with low, intermediate, and high procrastination levels. These groups are subsequently used for sampling. This coordination process territorializes the APSI questionnaire in two ways: by using it to diagnose procrastination levels and by using it for sampling. A similar pattern is visible in the other research worlds of Example 1. “Developing an interview guide” results in an interview guide that is subsequently used in “interviewing.” “Interviewing” results in audio files that are used in “Transcription,” and “Transcription” results in a transcription that is used in data analysis.

134

Judith Schoonenboom

This research coordination process has an important consequence: Research worlds tend to disappear once they have delivered their end product (Latour & Woolgar, 1986). Example 1 shows this in all its phases. Once the questionnaire has been selected, the considerations for selecting this questionnaire rather than another disappear. Once students have filled out the APSI questionnaire, how they interpreted the items and translated their experiences into one of the answer options is lost and inaccessible to the researcher. Once the students have been selected as representatives of one group, everything that contributed to bringing them to that position disappears. Once the audio files have been transcribed, any information not in the transcription has disappeared and will not play a role in the analysis. Once the three procrastination worlds and their differences have been described and explained, the interview transcripts disappear, except for a few illustrating quotations in Visser et al. (2018). Thus, each research practice delivers a product to a subsequent practice, leaving out much of its context and much information about how it came into being, focusing on one aspect of the experience to the exclusion of others. 7.3.3 Coordinating Research Outcomes

The second major coordinating research task is coordinating research outcomes. In this subsection, I  will focus on coordinating statements. Latour and Woolgar (1986) describe how research statements developed during the discovery of the thyrotropin-releasing factor (TRF) and its structure in the 1960s (see their Table 3.1 on p. 147). Before 1962, two mutually exclusive research worlds existed, each summarized in one statement: “There is a TRF” and “There is no TRF.” Each individual research paper defended one of these statements. Thus, before 1962, “There is a TRF” was a controversial statement. In 1962, the two statements were coordinated when the conclusion “There is a TRF” was reached. As a result, “There is a TRF” acquired the status of an accepted statement. The statement “There is no TRF” disappeared, and with it, the world in which TRF does not exist also vanished. Similar coordination processes took place in the following years. Between 1966 and 1969, a controversial statement existed, “TRF is a peptide,” with some scholars arguing that TRF is a peptide and others arguing that it is not. This controversy ended in January 1969, when the conclusion “TRF is a peptide” was reached, and the world in which TRF is not a peptide disappeared. Similarly, the structure of this peptide was controversial between April and November  1969, when the worlds with different structures of the peptide TRF disappeared, except for the world with the structure Pyro-Glu-HisPro-NH2. Thus, in each phase, coordinating statements make the worlds of the now “false” statement disappear. One world remains in which the now

A Performative Approach to Mixed Methods Research

135

uncontroversial accepted statement is true: after 1962, there is one world in which TRF exists; after January 1969, there is one world in which TRF is a peptide; and after November 1969, there is one world in which TRF has the structure Pyro-Glu-His-Pro-NH2. These coordination processes show that the worlds of research statements are also fluid. Worlds with old statements disappear, while worlds with new statements come into being, either temporarily or permanently. Furthermore, as a typical process in research, successive worlds become ever more detailed. First, TRF exists (1962); next, it exists and is a peptide (January 1969); and finally, it exists, it is a peptide, and its structure is Pyro-Glu-His-Pro-NH2 (November  1969). This development is intentional: Each TRF study is performed to confront a statement with a constructed research world, a process called the “mangle of practice” by Pickering (1995). Through repeated confrontations between a statement and research worlds, the statement is “mangled” and changes, becoming more sophisticated. To summarize, research worlds are coordinated in that the end product of one or more search worlds is used to create a new research world. The coordination of research process elements, also known as research design, uses the end product of one research practice to create and explore another research world. The coordination of statements leads to their further development, with new worlds coming into being and old worlds disappearing. Because this coordination involves end products, the world that produces this end product tends to become lost, a process we will discuss later. 7.4 Consequences for Mixed Methods Research

This section discusses a performative approach to mixed methods research. In the previous sections, we have seen that a performative approach can form the basis of all research. It recognizes different types of research yet views them all as acts that bring research worlds into being. A  performative approach, though, has a special meaning for mixed methods research because it also recognizes the differences between different research worlds and builds on them in a “performative” way. There is unanimous agreement among mixed methods scholars that the aim of mixed methods research is to bring together multiple perspectives to obtain a deeper and more inclusive understanding of a phenomenon than would be possible using one method alone. The question is how this definition translates into a performative approach for mixed methods research and how such research differs from other types of research that, as we have seen, can also be described in a performative approach. Whatever the approach, this aim of mixed methods research translates into at least three different research actions: to “take” the perspective (e.g. through the definition of concepts and

136

Judith Schoonenboom

data collection); to explore the perspective (e.g. through data analysis), which generates findings for each perspective separately (e.g. statements); and to bring the results of exploring the perspectives together (e.g. by connecting the statements). What are “multiple perspectives,” and how are they brought together? In a critical realist approach (Maxwell, 2012), multiple perspectives refers to multiple perspectives that exist on one world, which is assumed to exist independently from those perspectives. These perspectives can be explored using different methods, which results in findings for each perspective. These findings can be contradictory because different perspectives on this one world are possible. From a dialectical perspective, several worlds exist, one for each perspective. Here, the research metaphor is to develop and bring perspectives together in a “dialogue” between these worlds. In a performative approach, the word “perspective” is unfortunate, because it suggests a perspective on something already existing. Conversely, in a performative approach, the aim of mixed methods research involves bringing multiple research worlds into being through different methods, more specifically through the use of both quantitative and qualitative methods. Next, these different worlds are explored by asking: What is happening here? What are its inhabitants and its events? The final step of bringing together involves the coordination of the products that these worlds have delivered. Thus, from a performative approach, the aim of mixed methods research is to bring multiple research worlds into being through different methods, more specifically through using both quantitative and qualitative methods, exploring these worlds, and coordinating the products of these worlds to obtain a deeper and more inclusive understanding of a phenomenon than would be possible using one method alone. In Section 7.4.1, I describe how, in a performative approach, mixed methods research creates and explores different worlds. Section  7.4.2 describes how worlds and statements are coordinated in mixed methods research. This coordination is compared with the coordination in the TRF studies in Section 7.4.3. 7.4.1 Mixed Methods Research Creates and Explores Different Worlds

Given that different methods create different worlds, mixed methods research aims to explore and coordinate different worlds. A  performative approach recognizes many types of worlds and, thus, many ways in which different worlds can be included in a study. One type is the different worlds of people with different roles in a practice, which are explored in Example 2 (Schoonenboom, 2022, p. 59):

A Performative Approach to Mixed Methods Research

137

Example 2

A study by Clark and Moss (Clark, 2005; Clark & Moss, 2005) accompanied the redevelopment of the outdoor environment of a preschool in the United Kingdom. Their study involved 28 three- to four-year-olds, their parents, and preschool practitioners and managers. It answered two interrelated research questions: Which places do children see as important in this outdoor space? How do the children use these places? The children were involved in data collection and took photographs of important objects. One of the objects was the playhouse. According to Clark (2005): Observing the children revealed the house to be a key resource for them. The children confirmed this through their photographs, the tour and their interviews. Parents also mentioned the house as an important space in the preschool. However, the interviews with practitioners showed that the house was a source of tension. They felt it was too small. The review with children, practitioners and Learning through Landscapes recognised these opposing views and raised some possible solutions. The preschool has now turfed a new area for children to use to build their own temporary structures. (p. 34) Clark’s (2005) study included groups of people with different roles in the practice “redeveloping the outdoor environment of a UK preschool”: children, parents, practitioners, and staff from the organization Learning through Landscapes. This inclusion enabled a more comprehensive view of the outdoor environment, which we will consider later in more detail. In addition to including worlds that are different from the outset, researchers can create different worlds by splitting a presumedly whole world. Splitting worlds allows researchers to take a closer look at the phenomenon in different contexts. An especially fruitful technique in mixed methods research is splitting what was assumed to be one population. In Example 1, for instance, the research world started with one population: the population of first-year students in a teacher education programme. Based on their APSI scores, this population was subsequently split into three different populations of students with low, middle, and high procrastination levels. Whereas previous qualitative studies investigated convenience samples of students with high procrastination levels, Visser et al. (2018) were the first to study the overall population of first-year students. By splitting this overall population into three different populations with different procrastination levels, Visser et al. (2018) demonstrated the different roles of task aversiveness and motivation to become a teacher in each population.

138

Judith Schoonenboom

Splitting populations can occur at various stages of the mixed methods research process. In Example 1, the population was split at the beginning of the study. The quantitative APSI questionnaire was used to create three populations whose worlds were investigated through the use of interviews. Splitting populations can also be done at a later stage, as demonstrated by Example 3 (Schoonenboom, 2022, p. 61). Example 3

Glewwe et al. (2009) studied the effects of providing textbooks to schoolchildren in rural Kenya, in schools where textbooks had not been used before. They compared test scores of children in the 50 intervention schools with those of a control group, which showed no effect. In addition, the researchers went to each school and asked a child with a median score from each class to read their textbook aloud and answer a few questions. Further subgroup analysis of the test scores was used to determine the differential effect for children with different pretest scores. According to their abstract: A randomized evaluation in rural Kenya finds, contrary to the previous literature, that providing textbooks did not raise average test scores. Textbooks did increase the scores of the best students (those with high pretest scores) but had little effect on other students. Textbooks are written in English, most students’ third language, and many students could not use them effectively. (p. 112) In Example 3, the researchers first assumed one world of children in the intervention schools, for which they drew the simple inference that “providing textbooks does not have an effect.” After that, they split the population into two subpopulations: students with high pretest scores and students with middle and low pretest scores. Next, they developed two new simple inferences for each world: “providing textbooks does not have an effect” for the world of students with middle and low pretest scores, and “providing textbooks has an effect” for the world of students with high pretest scores. Finally, splitting populations can also be done in a reanalysis of an existing study, as shown in Example 4 (Schoonenboom, 2023a, p. 368): Example 4

Schoonenboom and Johnson (2021) used quantitative and qualitative data published in Lee and Greene (2007), a study on the relationships between the language problems of international students and their grade point average (GPA) in their first semester at one university in the US. Schoonenboom

A Performative Approach to Mixed Methods Research

139

and Johnson (2021) created a simple table containing one record for each student, along with their language score, GPA, and a quote from their interview, which could then be sorted and resorted to uncover patterns. One specific group of four students emerged. Despite their language problems, members of this group still obtained the highest GPA. The quotes show that three of the four students in this group referred to compensation strategies, while none of the students with language problems and less than the highest possible GPA did so. In this way, Schoonenboom and Johnson (2021) were able to formulate a hypothesis for further research: Language problems affect international students’ GPA unless they deliberately use compensation strategies. In their reanalysis, Schoonenboom and Johnson (2021) split the population of students in Lee and Greene (2007) into four subpopulations: (1) students with maximum academic achievement despite language problems; (2) students with less than a maximum academic achievement because of language problems; (3) students without language problems with a maximum academic achievement; and (4) students without language problems with a less than maximum academic achievement. The outcomes of their reanalysis are discussed in the following. In summary, mixed methods research creates and explores different worlds. These worlds are either different from the outset or created by splitting a presumedly whole world into several worlds. 7.4.2 Coordinating Research Statements in Mixed Methods Research 7.4.2.1 Establishing Differences

Unlike the TRF studies, the outcomes of mixed methods research must be coordinated not only between studies, but also within studies because each mixed methods study explores different research worlds. A first type of coordination in mixed methods research is to determine whether differences exist between statements in the explored worlds. The following are seven coordinating statements from Examples 1–4: 1. Observing the children revealed the house to be a key resource for them. The children confirmed [emphasis added] this through their photographs, the tour and their interviews. Parents also mentioned the house as an important space in the preschool (Clark, 2005, p. 34). 2. A randomized evaluation in rural Kenya finds, contrary to [emphasis added] the previous literature, that providing textbooks did not raise average test scores (Glewwe et al., 2009, p. 112).

140

Judith Schoonenboom

3. Although [emphasis added] nonsignificant correlations were found between test scores and GPA, qualitative findings indicated that English skills are an important factor affecting students’ course performance (Lee & Greene, 2007, p. 366) 4. Parents also mentioned the house as an important space in the preschool. However [emphasis added], the interviews with practitioners showed that the house was a source of tension. They felt it was too small (Clark, 2005, p. 34). 5. The results showed that students with average and high procrastination levels experienced task aversion (as opposed to students with low procrastination). 6. Textbooks did increase the scores of the best students (those with high pretest scores) but had little effect on other students (Glewwe et al., 2009, p. 112). 7. The quotes show that three of the four students [with the highest GPA despite language problems] referred to compensation strategies, while none of the students with language problems and less than the highest possible GPA did so (Schoonenboom, 2023a, p. 368). A first possible outcome of coordinating different worlds is that their resulting statements are identical. Thus, Statement (1) states that the playhouse is a key resource in the different worlds of different methods (observations, photographs, tours, and interviews) and different stakeholder groups (children and parents), as indicated by confirmed and also. In contrast, Statements (2)– (7) express differences between worlds, as indicated by contrary, although, however, opposed, but, and while. Differences are observed between studies (“contrary to the previous literature” in Statement 2); qualitative and quantitative methods (Statement 3); different stakeholder groups (parents and practitioners in Statement 4); different populations – students with low versus average and high procrastination levels in (Statement 5); students with high versus intermediate and low pretest scores in (Statement 6); and students with versus without compensation strategies in (Statement 7). 7.4.2.2 Resolving Differences

Another form of coordinating statement that plays a vital role in mixed methods research is resolving differences between worlds that had been established in an earlier step. Differences between worlds can be resolved in different ways. The following are five resolving statements from Examples 1–4: 8. Unlike students with high and intermediate procrastination levels, those with high procrastination levels were unable to overcome their task

A Performative Approach to Mixed Methods Research

9.

10.

11.

12.

141

aversion because they lacked a strong motivation to become a teacher, which the other groups had. Unlike other students with language problems, some students were able to overcome their language problems because they deliberately used compensation strategies. Textbooks did not have an effect in primary schools in rural Kenya because most children could not read their textbooks because the textbooks were written in English, which is not their native language. Unlike students with high pretest scores, the textbooks did not have an effect on students with low and intermediate pretest scores because these children could not read their textbooks, which the students with high pretest scores could. The review with children, practitioners and Learning through Landscapes recognised these opposing views and raised some possible solutions. The preschool has now turfed a new area for children to use to build their own temporary structures (Clark, 2005, p. 34).

In mixed methods research, differences between worlds are often resolved by explaining why these differences exist, which is indicated by because in the example statements. In Statement (8), lacking a strong motivation to become a teacher is presented as an explanation for the difference between students with intermediate and high procrastination levels. In Statement (9), the compensation strategies explain why one group of students with language problems still obtained the highest GPA possible. Explanations can build on each other within one study. In Example 3, the nonoccurrence of an effect was first explained by referring to most children’s inability to read their textbooks (Statement 10). Later, the effect on the subpopulation of students with high pretest scores was explained by their ability to read their textbooks (Statement 11). A different resolution can be found in Statement (12). Statement (12) expresses a difference between the practitioners, who considered the playhouse to be a source of tension, and the parents and children, who only had positive remarks about the playhouse. This difference is not resolved in an explanation. Instead, a solution in practice is developed that accommodates both views: The preschool has turfed a new area for children to build their own structures. Resolving a difference is never final. Each explanation raises new, unresolved questions, such as: Why did the students with low procrastination not experience task aversion? Why were some children able to read their textbooks, whereas others were not? Answering such questions may generate new differences. This idea fits well with the idea expressed by Uprichard and Dawney (2019) that the value of a mixed methods study may not

142

Judith Schoonenboom

be resolving a difference but instead revealing a difference. Depending on the context, establishing differences as in Statements (2)–(7) can be as valuable as, or sometimes more valuable than, resolving differences as in Statements (8)–(12). 7.4.3 Differences Between Mixed Methods Research and the TRF Studies

Mixed methods studies share with the TRF studies the fluidity of their research worlds: Statements are developed further and change correspondingly. However, there are also fundamental differences between the TRF studies and mixed methods research, to which we now turn. As a first difference, each TRF study explored only one world, and the statements and their worlds developed throughout many studies. Conversely, each mixed methods study explores various worlds. Consequently, statements also develop within one mixed methods study. This development is visible in the examples. Example 1 starts with the existing accepted statements, “high intrinsic motivation reduces academic procrastination” and “task aversion leads to procrastination.” These statements are combined and developed into “students with high procrastination levels procrastinated on averse tasks because they lacked a strong intrinsic motivation to become a teacher.” In Example 2, the inference “the playhouse is an important object” is developed into “the playhouse is an important object, but it is also a source of tension because it is too small.” In Example 3, the inference “providing textbooks does not have an effect in primary schools in rural Kenya” is specified. It is developed into “providing textbooks has an effect on students with high pretest scores but not on others because the former but not the latter were able to read their textbooks.” In Example 4, the inferences “language problems do not affect GPA” (quantitative) and “language problems do affect GPA” (qualitative) are combined and developed into “language problems affect international students’ GPA unless they deliberately use compensation strategies.” Second, and more important, the TRF studies and mixed methods research have different statement development processes. In the TRF studies, the development results from competition between mutually exclusive – hence controversial – statements: Either TRF exists, or it does not exist; it is or is not a peptide; it has this or that structure. In each case, one of the alternative statements ultimately wins, whereas the other statement disappears. Conversely, many simple statements with which a mixed methods study starts remain. They are not falsified but connected. The result of a mixed methods study is not one simple statement but a complex statement that commonly includes the initial statements and clarifies their relationship. For example, Statement (8) “Students with high procrastination levels procrastinated

A Performative Approach to Mixed Methods Research

143

on averse tasks because they lacked a strong intrinsic motivation to become a teacher” includes the initial simple statements “high intrinsic motivation reduces academic procrastination” and “task aversion leads to procrastination,” and it states that one is a condition for the other. Statement (3) is a special case – it suggests that the qualitative findings win and that language problems affect course performance. However, unlike the TRF studies, in which the winning statement concludes the discussion, this is not the case in (3), a statement in a mixed methods study. Thus, following Statement (3), Lee and Greene (2007) showed how individual differences in how language problems affected GPA could explain the nonoccurrence of an effect in the quantitative findings. In their reanalysis, Schoonenboom and Johnson (2021) were able to identify different groups. Thus, in all the examples, the end result of a mixed methods study is not one winning simple statement but a complex statement that incorporates previous simple statements. One final difference is that, in the TRF studies, solving the competition between statements also implies that the world of the “losing” statements disappears and that only the world containing the “winning” statement remains: After 1962, there is one world in which TRF exists, after January 1969, there is one world in which TRF is a peptide, and after November 1969, there is one world in which TRF has the structure Pyro-Glu-His-Pro-NH2. Conversely, in mixed methods research, statements are integrated while the different worlds of different groups remain. Thus, in Example 1, the different experiences of students with low, intermediate, and high procrastination levels remain after the statements they produced have been integrated into an explanation for their differences. Similarly, in Example 2, the different worlds of children, parents, practitioners, and Learning through Landscapes remain after one solution has been developed based on the statements they produced. To summarize, in contrast to the TRF studies, mixed methods research is not about competing statements, one of which ultimately wins, and its aim is not to arrive at one coherent world; instead, the goal is to explore different worlds that remain. 7.5 Reflections on Mixed Methods Research in a Performative Approach 7.5.1 Mixed Methods Research and Different Worlds

We have argued that the aim of mixed methods research is to explore and coordinate different worlds. The idea that mixed methods research interacts with differences is not new: It is the foundation of the dialectic stance (Greene, 2007, 2015; Greene & Hall, 2010) and dialectical pluralism (Johnson, 2015, 2023). It also plays an essential role in the transformative paradigm (Mertens, 2007, 2010). These scholars have emphasized that social science research

144

Judith Schoonenboom

should include various perspectives, dialogue with them, and treat them as being of equal value. A performative approach extends this view. We not only include worlds of different perspectives but also create different worlds. Splitting a whole world into different worlds is a powerful technique in which methods play an essential role. In the examples, splitting populations was performed using questionnaire scores (Example 1), subgroup analysis (Example 3), and case comparison analysis (Example 4). 7.5.2 Research as a Coordination Process

In a performative approach, research is perceived as a coordination process of both the research process and its outcomes. Because mixed methods research creates and explores different worlds, its coordination processes are numerous, diverse, and complex. Because worlds are coordinated using their end products rather than directly, a performative approach draws attention to how research worlds are made invisible; they disappear once they have delivered their end product. Thus, one task of mixed methods research could be to make those worlds visible by trying to rebuild them. Because worlds come into being and are maintained through interactions between human and nonhuman actors, it should be possible to rebuild such a world by reenacting the interactions that gave birth to it. Making disappeared worlds visible is an important aim of performative research – not necessarily mixed methods research – in the tradition of new materialism (Fox & Alldred, 2015, 2018; Schadler, 2019). Furthermore, the concept of coordination of different worlds supports Uprichard and Dawney’s (2019) idea that mixed methods research can result not only in a closure – that is, in bringing together worlds through an explanation or agreement – but also in difference. We have seen that differences that result from coordination statements (Statements 2–7) are often resolved through an explanation (Statements 8–11). Explanation, though, is only one possible form of coordination. In Statement (12), the different worlds of children and practitioners are brought together in practice by changing the outdoor environment so that it accommodates both perspectives – their difference is not explained. Even more, as clarified by Uprichard and Dawney (2019), the outcome of a mixed methods study can also be a difference between two different worlds that is not resolved. Thus, statements such as Statements (2) and (7), in which coordination results in a difference, could also be the outcome of a mixed methods study. If such revealed differences between different worlds had not been known before, such a study could provide a valuable contribution. One task of mixed methods research is coordinating statements from qualitative and quantitative worlds. An example of such “mixing” is (3), in which

A Performative Approach to Mixed Methods Research

145

the qualitative findings showed that language problems affect course performance, whereas this effect was not found in the quantitative findings. However, after exploring different forms of coordination, we can now draw the following conclusion: Although each mixed methods study coordinates statements from quantitative and qualitative worlds, a lot of statement coordination in a mixed methods study does not involve mixing. Statements (2)–(12) all stem from a mixed methods study, but several of these coordinating statements do not result from mixing. In Statement (2), the first outcome of the study is coordinated with statements from previous studies (“contrary to the previous literature”). In Statements (4) and (5), statements stemming from the same method – interviewing – are coordinated: Statement (4) coordinates statements from different stakeholder groups, whereas Statement (5) coordinates statements of students with different procrastination levels. In Statement (6), statements about students with and without high pretest scores that resulted from one quantitative analysis are coordinated. In Statement (11), the statement that “students with high pretest scores could read their textbooks” was not obtained through empirical study but through reasoning; thus, connecting this statement to other statements is not a case of mixing. In summary, a performative approach shows that mixing occurs in a study among other forms of coordination, including coordinating statements not based on empirical research (Hammersley, 2011). 7.5.3 The Distinction Between Worlds and Statements

A performative approach emphasizes a distinction between worlds and statements as the end products of research worlds. Distinguishing between “theory” and “practice” is not new, and various research approaches have emphasized the research process as an interaction (Ragin, 1992) or dance (Pickering, 1995) between theory and practice. Distinguishing between worlds and statements, however, has one crucial advantage: it enables us to discuss what happens with worlds as statements develop. As we saw earlier, what distinguishes mixed methods research from the TRF studies is that different worlds remain after their statements have been integrated. In Statement (1), the researchers, children, and parents agreed that the playhouse was an important resource for the children. Unlike in the TRF studies, this does not imply that their different perspectives vanished, and only one perspective remained. In Example 1, students with low, intermediate, and high procrastination levels still had different experiences after the differences between their worlds had been explained in the complex statement (Statement 8). These remaining worlds have important implications for the concept of integration. Common approaches to integration in mixed methods research implicitly assume that integration means integrating different realities into one reality. However, the examples show that, in mixed methods research,

146

Judith Schoonenboom

linguistic statements are integrated, while the different worlds remain and are not integrated. Thus, when Uprichard and Dawney (2019) stated that, in mixed methods research, realities are sometimes integrated and sometimes not, in my view, the authors misrepresented the mixed methods research process. I have tried to show that, contrary to the TRF studies, worlds in mixed methods research are not integrated. The problem that Uprichard and Dawney (2019) discussed should be reformulated. The real question is whether mixed methods researchers should always attempt to explain differences between statements from different worlds (as in Statements 8–11) or whether we could also let differences between statements stay as they are. 7.5.4 The Role of Controversial Statements in Mixed Methods Research

A performative approach includes an ontology of accepted, controversial, and tentative statements. Until now, we have discussed controversial statements as part of a competitive process that results in one world, a process that applies to the TRF studies but not to mixed methods research. But is there perhaps another role for controversial statements in mixed methods research? At first sight, controversial statements appear to play an important role in mixed methods research as well; Statements (2)–(7) all contain connectors indicating contradiction. Statement (2) contains a controversial statement, “providing textbooks raises average test scores,” which applies to studies elsewhere but not in Kenya; and in Statement (3), a controversial statement, “English skills affect students’ course performance,” applies to the qualitative findings but not to the quantitative findings. I have contributed to this view in various publications by marking such statements as “contradictions” (Schoonenboom, 2019, 2022, 2023b). But are they controversial statements? A moment of reflection shows that perhaps they are not. Could the outcomes of these studies really be – as in the TFT studies – that textbooks do not affect test scores (Example 3) or that language problems do not affect academic performance (Example 4)? This is highly unlikely. Thus, in a performative ontology, “controversial” statements (2) and (3) are accepted statements. Rather than trying to resolve a controversial statement, an attempt is made in Examples 3 and 4 to identify the circumstances under which an accepted statement does not apply. The result of these attempts is that the accepted statements do not apply when children cannot read their textbooks or when students with language problems use compensation strategies. Thus, instead of establishing one world in which only one competing statement applies, mixed methods research often identifies different worlds: worlds in which an accepted statement materializes and worlds in which it does not. Thus, our ontology of statements has implications for how we view the contribution of a mixed methods study. Looking closely at the examples, we

A Performative Approach to Mixed Methods Research

147

can see that the “controversial” statements are actually accepted statements. Many “explaining” statements are also accepted. In Example 1, both task aversion and intrinsic motivation were known to influence procrastination. Similarly, the fact that textbooks cannot affect children’s test scores when children cannot read them is not something we did not know or could not guess (Example 3). Furthermore, the fact that students can compensate for their language problems may have been known beforehand (Example 4). The value of these mixed methods studies is not in developing new statements. Instead, their contribution is that one or two of the many possible factors that could have prevented the accepted statement from materializing are decisive in distinguishing the different populations. Thus, of all the factors that could have played a role, the examples reveal the critical role of intrinsic motivation in situations of task aversion; of being able to read in a primary school intervention in rural Kenya; and of compensation strategies in dealing with language problems. In other words, statements may acquire the status of a priori knowledge, thereby changing the role of empirical research (Hammersley, 2011). A performative approach sheds new light on the role of whole-group effect testing in mixed methods research. In a performative approach to mixed methods research, different worlds and populations exist, and many “controversial” statements are actually accepted statements that fail to materialize in some worlds. Unlike the TRF studies, the aim of a mixed methods study is not to choose between two overall statements; consequently, the outcome of a whole-group effect test may not be that important. Therefore, we may as well not investigate the overall effect at all but, right from the beginning, explore the differences between groups, as was done in Example 1. Many interventions can be expected to have different effects on different groups of people. Thus, we could start by exploring the differences between children with and without high pretest scores (Example 3) and between students with different combinations of language skills and GPA (Example 4) without calculating the overall effect. This strategy aligns well with two traditions in the methodological literature. One is the realistic evaluation, in which, according to Pawson and Tilley (1997), we should find out “what works for whom under which circumstances.” The other tradition is represented by Turner (1948), who emphasized that quantitative researchers should first try to find groups that are homogeneous enough for a group effect to make sense. Performing quantitative analyses on a group that most likely consists of different populations does not make sense because the overall effect will hide the different effects that would emerge for different groups. Mixed methods research should directly engage with these different worlds, and a performative approach that recognizes these different worlds and their coming into being through methods provides a solid basis.

148

Judith Schoonenboom

7.6 Conclusion

If different methods create different worlds, mixed methods research is about exploring different worlds. Following dialectical pluralism, mixed methods research interacts with differences. In addition to the different worlds created by methods, mixed methods research explores the different perspectives from, for instance, different stakeholders and populations. In dialectical pluralism and the transformative approach, the emphasis has been on including different perspectives, especially those that have been excluded – for instance, perspectives from vulnerable groups. With its emphasis on the creation rather than the inclusion of different perspectives, a performative approach draws attention to the task of mixed methods research to split what had until then been considered one world into several different worlds and explore these separately to determine and assess their differences. A performative approach recognizes that coordination between research worlds is done using coordinating objects and not between worlds directly. Instruments are commonly used to coordinate the research process, and statements are commonly used to coordinate research outcomes. This recognition has two effects. First, it draws attention to what is lost when a research world is used to create an instrument or statement. Second, it opens up the possibility of rebuilding these lost worlds and exploring everything involved in creating the coordinating objects. Research worlds are fluid because statements develop. In mixed methods research, the researched worlds are fluid, multiple, and different as well. These characteristics of researched worlds have a significant impact on how statements develop. In the TRF studies, statements developed through competition between controversial statements. This competition results in one winner and the establishment of one world. Mixed methods research is different. Developing theory in mixed methods research is about integrating those statements resulting from the different worlds that remain. Thus, integration is not, contrary to a common view, combining different realities into one. Furthermore, controversial statements hardly play a role in mixed methods research. Because worlds are different, a final important implication is that mixed methods research should often start by splitting worlds instead of calculating effects in a world that consists of different populations. The performative approach is a monistic (Shan, 2022) position. The approach applies to all research. All research is assumed to create worlds that are all bounded, fluent, enacted, and re-enacted, and the task of all research is to coordinate these worlds. A performative approach recognizes that we can sensibly distinguish many different practices, experienced worlds, views of different stakeholder groups, different populations, and other types of worlds. This raises the question of which circumstances may justify ignoring these differences and conducting studies in which controversial statements compete

A Performative Approach to Mixed Methods Research

149

and one statement, and hence one world, wins. The research world in which one can rightly conduct TRF-like studies may be smaller than we think. Working with differences can be construed in terms of including, developing, creating, and rebuilding. Most other approaches to mixed methods research have emphasized inclusion. Thus, from a critical realist perspective, mixed methods research includes various perspectives on one independently existing world (Maxwell, 2012; Maxwell & Mittapalli, 2010). The transformative paradigm emphasizes the inclusion of perspectives of vulnerable groups (Mertens, 2007, 2010). The dialectical approaches emphasize inclusion and interaction with (presumedly existing) differences. Only the pragmatic approach emphasizes development: Previous ideas are updated based on interaction with reality (Morgan, 2007). A performative approach extends these approaches by emphasizing that research creates different worlds and that research worlds can be recreated or rebuilt. Worlds may be created by including assumedly different perspectives, but also by splitting a presumedly whole world – for instance, by splitting a population. We can also rebuild worlds that have become invisible after they delivered their end product. In this sense, a performative approach provides “strong” philosophical foundations that “justify a normative thesis that mixed methods research should be encouraged in (at least some) social scientific research” (Shan, 2022, p. 7). Mixed methods research is the standard, and we should define the circumstances under which we may rightly conduct studies that test controversial statements to arrive at one world. Acknowledgements

Many thanks to Cornelia Schadler for her remarks on an earlier draft of this chapter. All remaining errors are mine. References Austin, J. L. (1962). How to Do Things with Words. Oxford: Clarendon Press. Barad, K. (2007). Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning. Durham, NC: Duke University Press. Clark, A. (2005). Ways of seeing: Using the Mosaic approach to listen to young children’s perspectives. In A. Clark, A. T. Kjørholt, & P. Moss (Eds.), Beyond Listening: Children’s Perspectives on Early Childhood Services (pp.  29–49). Bristol: Policy Press. Clark, A., & Moss, P. (2005). Spaces to Play: More Listening to Young Children Using the Mosaic Approach. London: National Children’s Bureau. Coleman, R., & Ringrose, J. (2022). Introduction: Deleuze and research methodologies. In R. Coleman  & J. Ringrose (Eds.), Deleuze and Research Methodologies (pp. 1–22). Edinburgh: Edinburgh University Press. https://doi.org/doi:10.1515/97 80748644124-002.

150

Judith Schoonenboom

Deleuze, G., & Guattari, F. (1987). A Thousand Plateaus: Capitalism and Schizophrenia (B. Massumi, Trans.). Minneapolis, MN: University of Minnesota Press. Fox, N. J., & Alldred, P. (2015). New materialist social inquiry: Designs, methods and the research-assemblage. International Journal of Social Research Methodology, 18(4), 399–414. https://doi.org/10.1080/13645579.2014.921458. Fox, N. J., & Alldred, P. (2018). Mixed methods, materialism and the micropolitics of the research-assemblage. International Journal of Social Research Methodology, 21(2), 191–204. https://doi.org/10.1080/13645579.2017.1350015. Glewwe, P., Kremer, M., & Moulin, S. (2009). Many children left behind? Textbooks and test scores in Kenya. American Economic Journal: Applied Economics, 1(1), 112–135. https://doi.org/10.1257/app.1.1.112. Greene, J. C. (2007). Mixed Methods in Social Inquiry. San Francisco, CA: Jossey-Bass. Greene, J. C. (2015). Preserving distinctions within the multimethod and mixed methods research merger. In S. Hesse-Biber & R. B. Johnson (Eds.), The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry (pp. 606–615). Oxford: Oxford University Press. Greene, J. C., & Hall, J. N. (2010). Dialectics and pragmatism. In A. Tashakkori & C. Teddlie (Eds.), SAGE Handbook of Mixed Methods in Social & Behavioral Research (2nd ed., pp. 119–167). Los Angeles, CA: Sage. Halliday, T. (2022). Otherlands: A World in the Making. London: Penguin Books. Hammersley, M. (2011). On Becker’s studies of marijuana use as an example of analytic induction. Philosophy of the Social Sciences, 41(4), 535–566. https://doi. org/10.1177/0048393110367796. Johnson, R. B. (2015). Dialectical pluralism: A metaparadigm whose time has come. Journal of Mixed Methods Research, 11(2). https://doi.org/10.1177/1558689815607692. Johnson, R. B. (2023). Dialectical pluralism and integration in mixed methods research. In Y. Shan (Ed.), Philosophical Foundations of Mixed Methods Research: Dialogues Between Philosophers and Researchers. London: Routledge. Latour, B., & Woolgar, S. (1986). Laboratory Life: The Construction of Scientific Knowledge (2nd ed.). Princeton, NJ: Princeton University Press. Law, J. (2004). After Method: Mess in Social Science Research. London: Routledge. Lee, Y.-J., & Greene, J. (2007). The predictive validity of an ESL placement test: A mixed methods approach. Journal of Mixed Methods Research, 1(4), 366–389. https://doi. org/10.1177/1558689807306148. Maxwell, J. A. (2012). A Realist Approach for Qualitative Research. Los Angeles, CA: Sage. Maxwell, J. A., & Mittapalli, K. (2010). Realism as a stance for mixed methods research. In A. Tashakkori & C. Teddlie (Eds.), SAGE Handbook of Mixed Methods in Social & Behavioral Research (2nd ed., pp. 145–167). Los Angeles, CA: Sage. Mertens, D. M. (2007). Transformative paradigm: Mixed methods and social justice. Journal of Mixed Methods Research, 1(3), 212–225. https://doi.org/10.1177/1558 689807302811. Mertens, D. M. (2010). Transformative mixed methods research. Qualitative Inquiry, 16(6), 469–474. https://doi.org/10.1177/1077800410364612. Mol, A. (2002). The Body Multiple: Ontology in Medical Practice. Durham, NC: Duke University Press. Morgan, D. L. (2007). Paradigms lost and pragmatism regained: Methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research, 1(1), 48–76. https://doi.org/10.1177/2345678906292462. Pawson, R., & Tilley, N. (1997). Realistic Evaluation. London: Sage.

A Performative Approach to Mixed Methods Research

151

Pickering, A. (1995). The Mangle of Practice: Time, Agency, and Science. Chicago, IL: University of Chicago Press. Putnam, H. (1987). The Many Faces of Realism. Chicago, IL: Open Court Publishing Company. Ragin, C. C. (1992). Introduction: Cases of “what is a case?”. In H. S. Becker & C. C. Ragin (Eds.), What Is a Case? Exploring the Foundations of Social Inquiry (pp. 1–18). Cambridge: Cambridge University Press. Rescher, N. (1996). Process Metaphysics: An Introduction to Process Philosophy. New York, NY: SUNY Press. Schadler, C. (2019). Enactments of a new materialist ethnography: Methodological framework and research processes. Qualitative Research, 19(2), 215–230. https:// doi.org/10.1177/1468794117748877. Schoonenboom, J. (2019). Develop your case! How controversial cases, subcases, and moderated cases can guide you through mixed methods data analysis. Frontiers in Psychology, 10(1369). https://doi.org/10.3389/fpsyg.2019.01369. Schoonenboom, J. (2022). Developing the meta-inference in mixed methods research through successive integration of claims. In J. H. Hitchcock & A. J. Onwuegbuzie (Eds.), The Routledge Handbook for Advancing Integration in Mixed Methods Research (pp. 55–70). London: Routledge. https://doi.org/10.4324/9780429432828-6. Schoonenboom, J. (2023a). Mixed methods and multimethod research in education: Six design decisions for collecting, combining, and developing datasets. In R. Tierney, F. Rizvi, K. Ercikan,  & G. Smith (Eds.), International Encyclopedia of Education (4th ed., pp.  361–371). Amsterdam: Elsevier. https://doi.org/10.1016/ B978-0-12-818630-5.11034-6. Schoonenboom, J. (2023b). Ten mixed methods integration strategies for obtaining a detailed understanding. In R. Tierney, F. Rizvi, K. Ercikan, & G. Smith (Eds.), International Encyclopedia of Education (4th ed., pp. 450–461). Amsterdam: Elsevier. https://doi.org/10.1016/B978-0-12-818630-5.11045-0. Schoonenboom, J.,  & Johnson, R. B. (2021). The case comparison table: A  joint display for constructing and sorting simple tables as mixed analysis. In A. J. Onwuegbuzie & R. B. Johnson (Eds.), The Routledge Reviewer’s Guide to Mixed Methods Analysis (pp. 277–288). New York, NY: Routledge. https://doi.org/10.4324/ 9780203729434-24. Seibt, J. (2022). Process philosophy. In E. N. Zalta (Ed.), The Stanford Encyclopedia of Philosophy (Summer 2022 ed.). Stanford, CA: The Metaphysics Research Lab, Stanford University. https://plato.stanford.edu/archives/sum2022/entries/ process-philosophy/. Shan, Y. (2022). Philosophical foundations of mixed methods research. Philosophy Compass, 17(1), e12804. https://doi.org/https://doi.org/10.1111/phc3.12804. Turner, R. H. (1948). Statistical logic in social research. Sociology and Social Research, 32(3), 697–704. Uprichard, E.,  & Dawney, L. (2019). Data diffraction: Challenging data integration in mixed methods research. Journal of Mixed Methods Research, 13(1), 19–32. https://doi.org/10.1177/1558689816674650. Visser, L., Korthagen, F. A.,  & Schoonenboom, J. (2018). Differences in learning characteristics between students with high, average, and low levels of academic procrastination: Students’ views on factors influencing their learning. Frontiers in Psychology, 9(808). https://doi.org/10.3389/fpsyg.2018.00808.

8 A REALIST APPROACH FOR MIXED METHODS RESEARCH Joseph A. Maxwell

8.1

Introduction

In this chapter, I argue for the value of a realist perspective1 for mixed methods research. I’m using the term ‘realist’ to refer specifically to a significant view in philosophy and the social sciences, one that combines ontological realism – ‘the view that entities exist independently of being perceived, or independently of our theories about them’ (Phillips, 1987, p.  205) – with epistemological constructivism – the stance that our theories and perceptions of this world are inherently our own constructions, grounded in our perceptual and cognitive abilities, rather than any direct or infallible understanding of reality. This approach rejects the view that we can have any purely ‘objective’ knowledge of the world and accepts the possibility of alternative valid accounts of any phenomenon. All theories about the world are seen as grounded in a particular perspective and world view, and all knowledge is partial, incomplete, and fallible. I see no point in delving into the complex arguments in philosophy over realism. The Stanford Encyclopedia of Philosophy’s entry on realism (accessed online) stated at the outset: The question of the nature and plausibility of realism is so controversial that no brief account of it will satisfy all those with a stake in the debates between realists and non-realists. (Miller, 2021) My justification for bypassing these controversies is that the combination of ontological realism and epistemological constructivism has been endorsed by DOI: 10.4324/9781003273288-9

A Realist Approach for Mixed Methods Research

153

a substantial number of philosophers and social scientists, and I believe that this viewpoint has significant value for mixed methods researchers. A confusing array of terms, with somewhat varying meanings, have been used for such versions of realism. In philosophy, these include ‘critical’ realism (Bhaskar, 1989; Archer et al., 1998), ‘pluralistic’ (and ‘cautious’) realism (Dupre, 2016); ‘innocent’ realism (Haack, 1998, 2003), ‘experiential’ realism (Lakoff, 1987), ‘constructive’ realism (Wallner et  al., 1993), and ‘internal’ realism (Putnam, 1987), ‘natural’ realism (Putnam, 1999), and eventually ‘metaphysical realism’ (Putnam, 2012). In research methodology, they include ‘critical’ realism (Campbell, 1974, 1988, pp.  432, 447; Cook  & Campbell, 1979), ‘subtle’ realism (Hammersley, 1992), ‘emergent’ realism (Henry et al., 1998; Mark et  al., 2000), and simply ‘realism’ (Pawson, 2013; Pawson  & Tilley, 1997). In physics, they include ‘agential’ realism (Barad, 2007) and ‘model-dependent realism’ (Hawking & Mlodinow, 2010). Mir and Watson (2000), simply using ‘constructivism’ for this version of realism, argued: Constructivism occupies a methodological space characterized by ontological realism and epistemological relativism. Ontological realism is an important cornerstone of a field as applied as strategy, while epistemological relativism helps us explore the constructed nature of the field, where the researcher is an active participant rather than a reactor or information processor. (2000, p. 941) Lakoff stated the distinction between ‘objectivist’ and ‘realist’ views as follows: Scientific objectivism claims that there is only one fully correct way in which reality can be divided up into objects, properties, and relations. . . . Scientific realism, on the other hand, assumes that ‘the world is the way it is,’ while acknowledging that there can be more than one scientifically correct way of understanding reality in terms of conceptual schemes with different objects and categories of objects. (Lakoff, 1987, p. 265) This approach holds that there is no possibility of attaining a single ‘objective’ understanding of the world, what Putnam (1990, p. 114) described as a ‘God’s eye view’ that is independent of any particular viewpoint. He argued that ‘Internal realism is, at bottom, just the insistence that realism is not incompatible with conceptual relativity. One can be both a realist and a conceptual relativist’ (Putnam, 1987, p. 17; italics in original). Putnam later argued: [I]f we understand ‘metaphysical realist’ more broadly, as applying to all philosophers who reject all forms of verificationism and all talk of our

154

Joseph A. Maxwell

‘making’ the world, then I  believe it is perfectly possible to be a metaphysical realist in that sense and to accept the phenomenon I am calling ‘conceptual relativity’. (Putnam, 2012) None of these arguments reject the view that we can develop better understandings of reality. They simply assert that there is no possibility of attaining an ultimate ‘correct’ construction of the world that is immune to further change. Even in the physical sciences, what Thomas Kuhn (1963) called ‘paradigm shifts’, such as quantum theory in physics (Gamow, 1966), or plate tectonics in geology (McPhee, 1998), are likely to continue. Such views have been significant in the methodology of social research. Shadish, Cook, and Campbell, in their influential work on experimental designs (2002), argued that ‘all scientists are epistemological constructivists and relativists’ in the sense that they believe that both the ontological world and the worlds of ideology, values, etc. play a role in the construction of scientific knowledge (p. 29). Conversely, Schwandt, in his Dictionary of Qualitative Research (2007), stated: On a daily basis, most of us probably behave as garden-variety empirical realists – that is, we act as if the objects in the world (things, events, structures, people, meanings, etc.) exist as independent in some way from our experience with them. We also regard society, institutions, feelings, intelligence, poverty, disability, and so on as being just as real as the toes on our feet and the sun in the sky. (p. 256) This position has a much longer history than its label as a form of realism. A particularly clear statement was by Herbert Blumer, the leading figure in the symbolic interactionist approach to social research. In a classic paper, ‘The methodological position of symbolic interactionism’ (1969), Blumer asserted that symbolic interactionism is a perspective in empirical social science – ‘an approach designed to yield verifiable knowledge of human group life and human conduct’ (p. 21). He stated: I shall begin with the redundant assertion that an empirical science presupposes the existence of an empirical world. Such an empirical world exists as something available for observation, study, and analysis. It stands over against the scientific observer, with a character that has to be dug out and established through observation, study, and analysis. . . . ‘Reality’ for empirical science exists only in the empirical world. (pp. 21–22; italics in original)

A Realist Approach for Mixed Methods Research

155

However, Blumer combined this ontological realism with an epistemological constructivism (although, since this term was not available to him, he referred to this position as ‘idealism’). He asserted: [T]he empirical necessarily exists always in the form of human pictures and conceptions of it. However, this does not shift ‘reality,’ as so many conclude, from the empirical world to the realm of imagery and conception. .  .  . [This] position is untenable because the empirical world can ‘talk back’ to our pictures of it or assertions about it – talk back in the sense of challenging and resisting, or not bending to, our images or conceptions of it. (p. 22) Blumer summarised this argument by stating that ‘Fundamentally, empirical science is an enterprise that seeks to develop images and conceptions that can successfully handle and accommodate the resistance offered by the empirical world under study’ (pp. 22–23). In this chapter, I  explore the implications of this concept of realism for mixed methods research. This definition of realism assumes that our concepts and theories are our own constructions, but these refer to real phenomena, ones that can only be incompletely understood by means of those constructions. This concept has been promoted in both quantitative/experimental research (Campbell, 1974; Cook & Campbell, 1979) and qualitative research (Clark, 2008; Hammersley, 1992, 2008; Maxwell, 2008, 2009, 2012b, 2017a, 2017b). However, aside from articles exclusively presenting Bhaskar’s approach (e.g. Koopmans & Schiller, 2022; Modell, 2009), which differs in significant ways from what I’m proposing, it has received less attention in mixed methods research; one exception is Maxwell and Mittapalli (2010). In what follows, I address the relevance of realism for five key issues in mixed methods research: mind and mental phenomena, culture, diversity, causation, and research designs. I argue that all of these are real phenomena and that realism has important implications for each, ones that contrast with some prominent views in mixed methods research, and in social research generally. 8.2

Mind

The reality of mental phenomena2 has been a hotly debated issue in philosophy. To oversimplify somewhat, most historical approaches to this problem have been either versions of dualism (mind and the physical world, including the body, are separate and irreducible, with nothing in common) or physicalism (mental phenomena will eventually be reduced to physical – e.g.

156

Joseph A. Maxwell

neurological – entities and processes). The argument that mental properties and processes are just as real as physical ones, although they are understood using a different language and conceptual framework, has been strongly promoted by the philosopher Hilary Putnam (1990, 1999), who argued that both physiological theories of brain function, and mental/psychological theories involving concepts, emotions, and beliefs, are our constructions, but both theories refer to real entities, ones that can be understood using either (or both) of these conceptual frameworks; neither framework is reducible to the other. He stated: [T]he metaphysical realignment I  propose involves an acquiescence in a plurality of conceptual resources, of different and mutually irreducible vocabularies . . . coupled with a return not to dualism but to the ‘naturalism of the common man’. (Putnam, 1999, p. 38) Similar to Lakoff and Johnson (1999), he agreed that our minds are embodied, but insisted that this embodiment doesn’t require that we must somehow be able to reduce the vocabulary of ordinary psychological talk to chemistry, physics, neurology, or computer science (Putnam, 1999, pp. 148–149). However, he is not rejecting the results of neurological research on the brain structures and processes that are related to thought. For a more detailed presentation of a realist approach to mind (and culture), see Maxwell (2012b, chapter 2). Both quantitative and qualitative researchers typically employ the mental framework in seeking to understand the ideas, beliefs, and motives that inform people’s actions, and use the physical framework in describing people’s settings and behaviour. However, quantitative and qualitative methods have different, and complementary, strengths and limitations for studying mental phenomena. Quantitative methods are most useful for drawing aggregate conclusions about people’s beliefs and values; qualitative methods are most useful for understanding the relationships among an individual’s beliefs and values, and how these inform and influence their behaviour. I discuss this later in more detail under causation, in Section 8.5. 8.3

Culture

The realist approach to mind that I described earlier can also be applied to the concept of culture. If ‘mind’, understood as the ideas, beliefs, values, perceptions, and so on (broadly speaking, the thoughts and emotions) of an individual, can legitimately be seen as real phenomena, but ones that are conceptualised using a different framework than that used for physical phenomena, then culture can also be seen as real, consisting of the system of

A Realist Approach for Mixed Methods Research

157

ideas, values, etc. found in a social group. Individuals and collectives of individuals are conceptualised as physical entities, understood in terms of physical and biological processes, and observable groups and interactions among individuals. Mind, personality, and culture, in contrast, are understood in terms of mental concepts, including beliefs, values, and motives. This is consistent with the classic formulation by Kroeber and Parsons (1958) of the relationship among the four concepts of society or social system, culture, personality, and the biological organism. They theorised these as the intersection of two dimensions, forming a 2 × 2 table. The first dimension distinguished the individual from the collectivity of individuals; the second distinguished physical individuals and collectivities from personality and culture, the latter two seen respectively as individual and collective. They defined culture as ‘transmitted and created content and patterns of values, ideas, and other symbolic-meaningful systems as factors in the shaping of human behaviour and the artifacts produced through behavior’. Society, in contrast, was defined as ‘the specifically relational system of interactions among individuals and collectivities’. Of course, these two frameworks are interdependent; an individual’s biological (including neurological) processes aren’t disconnected from that individual’s beliefs, values, and so on. Putnam’s analysis of the relationship between brain and mind – that the two terms refer to the same broad domain of real phenomena, but using different conceptual frameworks – is equally applicable to the concepts of society (or social system) and culture. Culture, understood as the system of concepts, beliefs, and so on that exists in a social group, thus refers to a real phenomenon, although any theory of such systems is necessarily a construction (Maxwell, 2012, chapter 2). As is the case for mind and brain, the two frameworks are most useful for different aspects of human groups and interactions. Research questions focusing on behaviour are likely to emphasise physical interactions, but explaining this behaviour will generally require attention to individuals’ beliefs, goals, and values. However, for many years, the dominant view of culture, in anthropology and other social sciences, as well as in popular use, was that it consisted of the ideas, beliefs, values, and so on that are shared by members of a social group. This view was systematically challenged by Wallace (1970), who contrasted what he called the ‘replication of uniformity’ and ‘organization of diversity’ approaches to culture; he argued that the latter far better represented the actual interaction of the meanings, beliefs, and so on found in any social group. The ‘shared’ definition of culture was criticised by many other social scientists as well; see Maxwell (2012b, pp. 21–32) for a more detailed discussion. More recently, it has been attacked by Atran & Medin (2008), who argued that ‘for our purposes, it is just a nonstarter to treat or define cultures or groups in terms of shared properties’ (p. 265), and that ‘the study of culture is the study of variation within and across populations’ (p. 222).

158

Joseph A. Maxwell

As is the case for the concepts of body and mind, the concepts of society and culture, to a significant extent, are different frameworks for making sense of a single range of phenomena – in this case, of human (and other animals’) collective properties. 8.4

Diversity

As I argued in discussing culture, diversity is a real phenomenon, one that is often ignored or dismissed by researchers who aggregate their data to identify general patterns or characteristics of a group. However, the reality of diversity is much broader than culture; it also includes individual and social diversity. Unfortunately, social and political theories have often assumed that diversity, despite its moral and practical benefits, is necessarily in tension with solidarity and community – that individual or group differences are intrinsically a source of conflict, and need to be overcome or transcended through the recognition or creation of commonalities. (For a detailed argument against this position, see Maxwell, 1996; 2012b, chapter 4; 2020.) This tendency was substantially strengthened by the astronomer and statistician Adolphe Quetelet (1796–1874). Quetelet’s main influence on statistics was his emphasis on the concept of ‘average’. In astronomy, when multiple measurements of a single phenomenon (e.g. the observed time that it took for a planet to move a given distance) varied, the ‘true’ value was best approximated by the arithmetic mean (average) of these measurements; variations from this mean were treated as ‘error’. This is a legitimate strategy when it can be assumed that there is a ‘true’ value to which all observations are approximations. Unfortunately, Quetelet applied this logic to single measurements of multiple individuals or events, treating individual variation as ‘error’ (Mayr, 1982, p. 47; Rose, 2015, pp. 23–28). This emphasis on average values was central to Quetelet’s and others’ dream of ‘ “automating” the human sciences by substituting calculation for intuition’ (Weisberg, 2014, p. 4). Quetelet states: [T]he more advanced the sciences have become, the more they have tended to enter the domain of mathematics. . . . We can judge of the perfection to which a science has come by the facility, more or less great, with which it may be approached by calculation. (Hayek, 1952, p. 357, quoting a translation by Walker, 1929, p. 29) Quetelet’s focus on averages, as the true measure of some phenomenon, was profound. ‘Scholars and thinkers in every field hailed Quetelet as a genius for uncovering the hidden laws of society’ (Rose, 2015, p.  31). Weisberg (2014) argued that these influences led to ‘willful ignorance’ of the actual variability of phenomena, and a profound misunderstanding and mismeasure

A Realist Approach for Mixed Methods Research

159

of uncertainty, and have distorted the use of statistics for understanding variability. As a result, diversity is often ignored or ‘erased’ in the presentation of research findings, or reduced to a ‘standard deviation’ from the mean. This practice tends to emphasise shared beliefs and behaviours, and ‘main effects’, and neglect variability. Logical positivism, which was the dominant philosophy of science during the first half of the 20th century, largely retained this focus on mathematicisation and the goal of general laws, continuing to treat variability as a hindrance to these goals. However, logical positivism was shown in the 1950s to be largely unworkable, and many of its tenets were abandoned by philosophers. A. J. Ayer, whose book Language, Truth, and Logic (1936) basically introduced logical positivism to the English-speaking world, said much later, when asked what the problems were with logical positivism, ‘I suppose the greatest defect is that nearly all of it was false’ (1978). However, positivist ideas have continued to influence the thinking of many social researchers. Steinmetz (2005) wryly noted, ‘Despite repeated attempts by social theorists and researchers to drive a stake through the heart of the vampire, the [social science] disciplines continue to experience a positivistic haunting’ (p. 3). The point of this brief review is that diversity is still often treated as a hindrance to identifying ‘main effects’ of some intervention or process, and not explicitly recognised as a real phenomenon, one that needs to be acknowledged in order to adequately develop policies or assess their consequences, as well as to understand social and cultural processes in general. 8.5

Causation

One implication of the reality of mental phenomena, described earlier, is that such phenomena can legitimately be seen as causes. This is clearly recognised in our everyday accounting for people’s actions; we regularly explain these actions by referring to their beliefs, motives, and values. As the psychologist Albert Bandura (1986) argued, ‘What people think, believe, and feel affects how they behave. The natural and extrinsic effects of their actions, in turn, partly determine their thought patterns and affective reactions’ (p. 25). Similarly, Mohr stated: [P]eople’s motives or reasons for undertaking certain behaviors could not form part of general laws governing those behaviors, but . . . those same motives or reasons could, in the individual case, cause the behaviors to be carried out. (p. vi) However, causation has been a particularly fraught issue for the social sciences. Many quantitative researchers have tended to adopt a quasi-positivist

160

Joseph A. Maxwell

version of causation – that all we can understand about causation is the regularity with which y follows x. They have tended to treat the actual process of causation as a ‘black box’ and denied that we can get inside that box to understand the actual processes involved (an inherent limitation of quantitative and experimental methods). Qualitative researchers, in reaction, have often rejected the entire concept of causal explanation in qualitative research (or even more broadly, in the social sciences) as a vestigial remnant of logical positivism. A particularly influential statement of this position was by Lincoln and Guba (1985), who argued that ‘the concept of causality is so beleaguered and in such serious disarray that it strains credibility to continue to entertain it in any form approximating its present (poorly defined) one’ (p. 141). They later grounded this view in a constructivist stance, stating that ‘there exist multiple, socially constructed realities ungoverned by natural laws, causal or otherwise’ (Guba & Lincoln, 1989, p. 86) and that ‘ “causes” and “effects” do not exist except by imputation’ (p. 44). Despite this, a realist approach to causation has gained increasing acceptance in philosophy, particularly through the work of the philosopher Wesley Salmon (1984, 1989, 1998), who referred to this approach as the ‘causal/ mechanical’ view. In this view, ‘explanatory knowledge opens up the black boxes of nature to reveal their inner workings. It exhibits the ways in which the things we want to explain come about’ (Salmon, 1989, p. 182). In the philosophy of the social sciences, very similar views were developed, by realists and others. Sayer (1992) argued: [M]uch that has been written on methods of explanation assumes that causation is a matter of regularities in relationships between events, and that without models of regularities we are left with allegedly inferior, ‘ad hoc’ narratives. But social science has been singularly unsuccessful in discovering law-like regularities. One of the main achievements of recent realist philosophy has been to show that this is an inevitable consequence of an erroneous view of causation. Realism replaces the regularity model with one in which objects and social relations have causal powers which may or may not produce regularities, and which can be explained independently of them. In view of this, less weight is put on quantitative methods for discovering and assessing regularities and more on methods of establishing the qualitative nature of social objects and relations on which causal mechanisms depend. (pp. 2–3) These mechanisms are seen not as general laws, or as having invariant outcomes, but as situationally contingent; their specific context is inextricably involved in the causal processes (Cartwright, 1999, p. 73; Little, 1998, p. 197 ff.; Pawson & Tilley, 1997).

A Realist Approach for Mixed Methods Research

161

A particularly clear statement of this view is by Shadish et al. (2002), in their influential presentation of experimental and quasi-experimental methods for studying causation. They stated: [T]he unique strength of experimentation is in describing the consequences attributable to deliberately varying a treatment. We call this causal description. In contrast, experiments do less well in clarifying the mechanisms through which and the conditions under which that causal relationship holds – what we call causal explanation. (p. 9; emphasis in original) They argued elsewhere in their book that ‘qualitative methods provide an important avenue for discovering and exploring causal explanations’ (p. 389), and provided an example of how, in one study, the actual cause of the result was ‘revealed only by the qualitative examination of the actual operation of groups’ (p. 391). Mohr (1982, 1996) similarly distinguished between what he called variance theory and process theory. Variance theory is the traditional view derived from positivism, what Shadish et al. labelled ‘causal description’. Process theory, in contrast, deals with events and the processes that connect them; it is based on an analysis of the causal processes by which some events influence others. Process explanation, since it deals with specific events and processes, is less amenable to statistical or experimental approaches. It lends itself to the in-depth study of one or a few cases or a relatively small sample of individuals, and to textual forms of data that retain the chronological and contextual connections between events. Similar distinctions to that between variance theory and process theory include Blumer’s distinction between ‘variable analysis’ and the ‘process of interpretation’ (1956), Ragin’s between variable- and case-oriented approaches (1987), and Yin’s between factor theories and explanatory theories (1993, pp. 15 ff.). This distinction between two approaches to causation has received increasing support from philosophy. Illari and Russo, in their book Causality: Philosophical Theory Meets Scientific Practice (2014), provided a detailed argument for two different, and complementary, theories of causation, which they termed ‘difference-making’ and ‘production’ accounts. These closely correspond to Shadish et al.’s and Mohr’s distinctions. The main implication for mixed methods researchers from this literature is that regularity theories and process theories have complementary strengths and limitations (Illari & Russo, 2014, pp. 58, 75–134), and that these substantially correspond to the strengths and limitations of quantitative and qualitative research, respectively, as described by Becker (2017, pp. 19–35), Maxwell (2021), and Miles and Huberman (1994, p. 147). Such arguments assume that causal processes are real phenomena, ones that are responsible for the relationships that experimental and quantitative methods identify.

162

Joseph A. Maxwell

An important implication of this distinction is its relevance for extending causal conclusions to other settings than those initially studied. The philosopher Nancy Cartwright and the economist Jeremy Hardie, in their book Evidence-Based Policy: A Practical Guide to Doing It Better (2012), stated at the outset: You are told: use policies that work. And you are told: RCTs – randomized controlled trials – will show you what these are. That’s not so. RCTs are great, but they do not do that for you. They cannot alone support the expectation that a policy will work for you. . . . For that, you need to know a lot more. (Cartwright & Hardie, 2012, p. ix) They argue that the key issue in extending policies to new settings is context; how a different situation will affect the outcome of the policy. Although they never use the term ‘qualitative’, their argument is essentially that qualitative methods are necessary for understanding how the new context will affect the processes and outcomes of implementing a policy in a different setting. Cartwright has argued elsewhere (2016) that quantitative research has almost no useful tools for generalising causal conclusions to different contexts. For more detailed discussion, see Davidson (1993) and Maxwell (2012b, chapter 3; 2021).

8.6

Research Designs

The term ‘research design’ obviously refers to the conception of the research held by the researcher – how this researcher envisions the research and plans to carry it out. As discussed earlier, this conception is a real entity, part of the researcher’s mental framework. We invoke the researcher’s theories, purposes, and so on in explaining the decisions that result in the design. The dominant conception of research design in the self-identified mixed methods research community has been a typological one; designs (plural) are specific models for how the research should be planned and conducted, typically represented by a diagram (e.g. Creswell & Plano Clark, 2018; Mertens, 2018; Tashakkori et al., 2021); for critiques of this conception, see Maxwell (2012, chapter 5; 2013, pp. 79–82; in press). However, the actual implementation of the research – how it is negotiated and carried out, in interaction with the settings and participants involved – is also a real phenomenon, independent of how the researcher understands this, and may result in unforeseen and unintended (or even unrecognised) consequences. For this reason, I use ‘design’ broadly to refer to the realisation of the research design ‘on the ground’, as well as to the researcher’s conceptions of the design (Maxwell, 2011, 2013, pp. 2–3).

A Realist Approach for Mixed Methods Research

163

As a real entity, I see a research design as having five main components. Three of these – the researcher’s goals (personal and practical as well as academic) for the study, the conceptual framework (the researcher’s beliefs and theories about the setting and participants), and the specific research questions that the researcher wants to answer – are part of the researcher’s mental model of the research. However, the methods (planned and actual) for answering the research questions, and the validity issues that will need to be addressed, as well as the ways in which these change during the study, exist both as mental conception held by the researcher(s) and as the actual implementation of the research – how the conceptualised design is actually playing out, and potentially altered (possibly in ways that the researcher is not fully aware of), during the course of the research. During this process, the researcher’s goals, conceptual framework, and research questions may also change, the actual data collection and analysis may be significantly modified, and the potential validity threats, and the strategies for dealing with these, may differ from those initially intended. To avoid unintended and undesired consequences, researchers need to understand how their planned design is actually realised – what activities turn out to be difficult or impossible to implement as intended; how participants perceive and react to the research in ways that were unexpected and could affect the results – and how the intended design needs to be modified in order to address these issues. 8.7

Conclusion

In this chapter, I have tried to indicate how a realist approach to mixed methods research has important advantages for conceptualising and conducting this research. Realism has implications for a variety of issues in this area of research, for both quantitative and qualitative approaches as well as for their joint use; see Maxwell (2012b) for a more detailed discussion of realism in qualitative research. Here, I have addressed five areas in which I believe realism has a valuable role to play in designing and conducting mixed methods studies: mind and meaning, culture, diversity in social and cultural phenomena, causation, and research design. I’m not arguing that realism is the single ‘correct’ approach to mixed methods research, only that it has insights and advantages that other stances lack, and that it is thus a valuable conceptual tool in a researcher’s toolkit. My purpose in this chapter is to indicate areas where I think a realist perspective can be useful for mixed methods researchers. Notes 1 In previous work, I used the term ‘critical realism’ to denote this combination of ontological realism and epistemological constructivism, following Campbell’s use

164

Joseph A. Maxwell

of this term (1974, 1988, pp. 432, 447; Cook & Campbell, 1979, pp. 28–29). However, this term is widely used (particularly in the UK) to refer specifically to the views of Roy Bhaskar and his associates (e.g. Archer et al., 1998; Gorski, 2013; Lawani, 2021; McEvoy & Richards, 2006; cf. Wikipedia, 2022a). Bhaskar apparently independently developed this combination of ontological realism and epistemological constructivism, but originally used the term ‘transcendental realism’ for his views for the natural sciences (1975), and later extended this to the social sciences as ‘critical naturalism’ (1979). Others in the Bhaskarian tradition then adopted the term ‘critical realism’ to refer to both, and Bhaskar later accepted this (Wikipedia, 2022b). However, Bhaskar’s later development of what he termed ‘transcendental dialectical critical realism’, as a moral and spiritual as well as a scientific position (Bhaskar, 2011), departed in significant ways from the definition I have adopted, and has been criticised by some other proponents of realism. For example, Hammersley argued: [T]he critical realist argument for the emancipatory character of social science – the claim that social research necessarily involves a critique of society, pointing in the direction of desirable change – is not cogent. Bhaskar and Collier have not succeeded in undermining Hume’s demonstration that ought cannot be derived solely from is. (2002, p. 47) Pawson has specifically endorsed Campbell’s version of critical realism rather than Bhaskar’s, stating that Campbell’s version is about promoting criticism and counter-criticism in the community of scientists; . . . [Bhaskar’s] is about the possession of a privileged, normative standpoint with which to criticize other interpretations of the world. (2013, p. xviii) Thus, to avoid confusion, I’m simply labelling the combination of ontological realism and epistemological constructivism as ‘realist’, acknowledging that there is a diversity of realist views, but that many share these basic assumptions. 2 I want to emphasise, although it isn’t directly relevant to the human sciences, that the concept of ‘mind’, as I’ve used it, isn’t limited to humans. A great deal of research in biology is now using the concept of ‘mind’ to understand nonhuman animals (e.g. Heinrich, 1999; Chittka, 2022), a long-standing tradition in biology (Köhler, 1925; Yerkes, 1917). Shettleworth, in the introduction to the second edition of her highly regarded textbook Cognition, Evolution, and Behavior (2010, p. v), stated at the outset that ‘The study of the animal mind is one of the most exciting areas in the cognitive sciences’. There have even been arguments for the usefulness and validity of mental concepts in understanding plants (Chamovitz, 2012; Wohlleben, 2015).

References Archer, M., Bhaskar, R., Collier, A., Lawson, T.,  & Norrie, A. (Eds.). (1998). Critical Realism: Essential Readings. London: Routledge. Atran, S., & Medin, D., (2010). The native mind and the cultural construction of nature. Cambridge, MA: MIT Press. Ayer, A. J. (1936). Language, Truth, and Logic. London: Gollancz. Ayer, A. J. (1978). Interview accessed at www.basicincome.com/bp/ratherlike.htm (this contains a link to a video of the television interview in which Ayer said this). Ayer, A. J. (1982). Language, Truth, and Logic. London: Penguin Books.

A Realist Approach for Mixed Methods Research

165

Bandura, A. (1986). Social Foundations of Thought and Action: A  Social Cognitive Theory. Englewood Cliffs, NJ: Prentice-Hall. Barad, K. (2007). Meeting the Universe Halfway: Quantum Physics and the Entanglement of Matter and Meaning. Durham and London: Duke University Press. Becker, H. S. (2017). Evidence. Chicago, IL: University of Chicago Press. Bhaskar, R. (1975). A Realist Theory of Science. London: Verso. Bhaskar, R. (1979). The Possibility of Naturalism. A Philosophical Critique of the Contemporary Human Science. Atlantic Highlands, NJ: Humanities Press. Bhaskar, R. (2011). Critical Realism: A Brief Introduction. London: Routledge. Blumer, H. (1956). Sociological analysis and the “variable.” American Sociological Review 22, 683–690. Reprinted in Blumer, Symbolic interactionism: Perspective and method (pp. 127–139). Berkeley: University of California Press. Blumer, H. (1969). The methodological position of symbolic interactionism. In H. Blumer (Ed.), Symbolic Interactionism: Perspective and Method (pp. 1–60). Berkeley, CA: University of California Press. Campbell, D. T. (1974). Evolutionary epistemology. In P. A. Schilpp (Ed.), The Philosophy of Karl Popper (pp. 413–463). La Salle, IL: Open Court Publishing, Reprinted in Campbell (1988), pp. 377–388. Campbell, D. T. (1988). Methodology and Epistemology for Social Science: Selected Papers. Chicago: University of Chicago Press. Cartwright, N. (1999). The Dappled World: A  Study of the Boundaries of Science. Cambridge: Cambridge University Press. Cartwright, N. (2016). Where’s the rigor when you need it? Foundations and Trends® in Accounting, 10(2–4), 106–124. Cartwright, N., & Hardie, J. (2012). Evidence-Based Policy: A Practical Guide to Doing It Better. Oxford, UK: Oxford University Press. Chamovitz, D. (2012). What a Plant Knows. Oxford: Oneworld Publications. Chittka, L. (2022). The Mind of a Bee. Princeton: Princeton University Press. Clark, A. M. (2008). Critical realism. In L. A. Given (Ed.), The SAGE Encyclopedia of Qualitative Research Methods (vol. 1, pp. 167–169). Thousand Oaks, CA: Sage. Cook, T. D.,  & Campbell, D. T. (1979). Quasi-Experimentation: Design  & Analysis Issues for Field Settings. Boston, MA: Houghton Mifflin Co. Creswell, J. L., & Plano Clark, V. L. (2018). Designing and Conducting Mixed Methods Research (3rd ed.). Thousand Oaks, CA: Sage Publications. Davidson, D. (1993). Thinking causes. In J. Heil & A. Mele (Eds.), Mental Causation (pp. 3–17). Oxford: Clarendon Press. Dupre, J. (2016). Realism, pluralism and naturalism in biology. In N. Cartwright & K. Ward (Eds.), Re-Thinking Order After the Laws of Nature (pp. 99–118). London: Bloomsbury. Gamow, G. (1966). Thirty Years That Shook Physics. Garden City, NY: Doubleday. Gorski, P. S. (2013). What is critical realism? And why should you care? Contemporary Sociology, 4(25), 658–670. Guba, E. G., & Lincoln, Y. S. (1989). Fourth Generation Evaluation. Newbury Park, CA: Sage. Haack, S. (1998). Manifesto of a Passionate Moderate. Chicago, IL: University of Chicago Press. Haack, S. (2003). Defending Science – Within Reason. Amherst, NY: Prometheus Press. Hammersley, M. (1992). Ethnography and realism. In M. Hammersley (Ed.), What’s Wrong with Ethnography? Methodological Explorations (pp.  43–56). London: Routledge.

166

Joseph A. Maxwell

Hammersley, M. (2002). Research as emancipatory: The case of Bhaskar’s critical realism. Journal of Critical Realism, 1(1), 33–48. Hammersley, M. (2008). Questioning Qualitative Inquiry: Critical Essays. London: Sage Publications. Hawking, S., & Mlodinow, L. (2010). The Grand Design. London: Bantam Press. Hayek, F. A. (1952). The Counterrevolution of Science: Studies on the Abuse of Reason. Glencoe, IL: The Free Press. Heinrich, B. (1999). Mind of the Raven. New York: Harper Collins Publishers. Henry, G., Julnes, J., & Mark, M. (1998). Realist Evaluation: An Emerging Theory in Support of Practice. New Directions for Evaluation 78. San Francisco: Jossey-Bass. Illari, P., & Russo, F. (2014). Causality: Philosophical Theory Meets Scientific Practice. Oxford: Oxford University Press. Köhler, W. (1925). The Mentality of Apes. London: Routledge & Kegan Paul. Kroeber, A. L.,  & Parsons, T. (1958). The concepts of culture and of social system. American Sociological Review, 23(1), 582–583. Kuhn, T. (1963). The Structure of Scientific Revolutions. Chicago, IL: University of Chicago Press. Lakoff, G. (1987). Women, Fire, and Dangerous Things: What Categories Reveal About the Mind. Chicago, IL: University of Chicago Press. Lakoff, G., & Johnson, M. (1999). Philosophy in the Flesh: The Embodied Mind and Its Challenge to Western Thought. New York: Basic Books. Lawani, A. (2021). Critical realism: What you should know and how to apply it. Qualitative Research Journal, 21(3), 320–333. Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic Evaluation. Thousand Oaks, CA: Sage Publications. Little, D. (1998). Causal explanation in the social sciences. In D. Little (Ed.), Microfoundations, Method, and Causation (pp. 197–214). New Brunswick, NJ: Transaction Publishers. Mark, M. M., Henry, G. T., & Julnes, G. (2000). Evaluation: An Integrated Framework for Understanding, Guiding, and Improving Policies and Programs. San Francisco, CA: Jossey-Bass. Maxwell, J. A. (in press). Mixed methods design in historical perspective: Implications for researchers. To appear In C. Poth (Ed.), The SAGE Handbook of Mixed Methods Research Designs. Thousand Oaks, CA: Sage Publications. Maxwell, J. A. (1996). Diversity and methodology in a changing world. Pedagogia, 30, 32–40. Maxwell, J. A. (2008). The value of a realist understanding of causality for qualitative research. In N. Denzin (Ed.), Qualitative Research and the Politics of Evidence (pp. 163–181). Walnut Creek, CA: Left Coast Press. Maxwell, J. A. (2009). Evidence: A critical realist perspective for qualitative research. In N. Denzin & M. Giardina (Eds.), Qualitative Inquiry and Social Justice (pp. 108– 122). Walnut Creek, CA: Left Coast Press. Maxwell, J. A. (2011). Epistemological heuristics for qualitative research. In H. Soini, E.-L. Kronqvist, & G. Huber (Eds.), Epistemologies for Qualitative Research (pp. 10–27). Tuebingen, Germany: Center for Qualitative Psychology. Maxwell, J. A. (2012). A Realist Approach for Qualitative Research. Thousand Oaks, CA: Sage. Maxwell, J. A. (2013). Qualitative Research Design: An Interactive Approach (3rd ed.). Thousand Oaks, CA: Sage Publications.

A Realist Approach for Mixed Methods Research

167

Maxwell, J. A. (2020). The value of qualitative research for public policy. Qualitative Inquiry, 26(2), 177–186. Maxwell, J. A., & Mittapalli, K. (2010). Realism as a stance for mixed method research. In A. Tashakkori  & C. Teddlie (Eds.), Handbook of Mixed Methods in Social and Behavioral Research (2nd ed., pp. 145–167). Thousand Oaks, CA: Sage Publications. Mayr, E. (1982). The Growth of Biological Thought: Diversity, Evolution, and Inheritance. Cambridge, MA: Harvard University Press. McEvoy, P.,  & Richards, D. (2006). A  critical realist rationale for using a combination of quantitative and qualitative methods. Journal of Research in Nursing, 11(1), 66–78. McPhee, J. (1998). Annals of the Former World. New York: Farrar, Strauss, & Giroux. Mertens, D. (2018). Mixed Methods Design in Evaluation. Thousand Oaks, CA: SAGE Publications. Miles, M. B.,  & Huberman, A. M. (1994). Qualitative Data Analysis: An Expanded Sourcebook. Thousand Oaks, CA: SAGE Publications. Miller, A. (2021). Realism. In Edward N. Zalta (Ed.), The Stanford Encyclopaedia of Philosophy (Winter 2021 ed.). https://plato.stanford.edu/archives/win2021/entries/ realism/. Mir, R., & Watson, A. (2000). Strategic management and the philosophy of science: The case for a constructivist methodology. Strategic Management Journal,  21(9), 941–953. Modell, S. (2009). In defence of triangulation: A  critical realist approach to mixed methods research in management accounting. Management Accounting Research, 20, 208–221. Mohr, L. B. (1982). Explaining Organizational Behavior. San Francisco, CA: Jossey-Bass. Mohr, L. B. (1996). The Causes of Human Behavior: Implications for Theory and Method in the Social Sciences. Ann Arbor, MI: University of Michigan Press. Pawson, R. (2013). The Science of Evaluation: A Realist Manifesto. London: Sage. Pawson, R., & Tilley, N. (1997). Realistic Evaluation. London: Sage. Phillips, D. (1987). Philosophy, Science, and Social Inquiry: Contemporary Methodological Controversies in Social Science and Related Applied Fields of Research. Oxford: Pergamon Press. Putnam, H. (1987). The Many Faces of Realism. La Salle, IL: Open Court Press. Putnam, H. (1990). Realism with a Human Face. Cambridge, MA: Harvard University Press. Putnam, H. (1999). The Threefold Cord: Mind, Body, and World. New York: Columbia University Press. Putnam, H. (2012). Philosophy in an Age of Science: Physics, Mathematics, and Skepticism, Mario De Caro & David Macarthur (Eds.), Cambridge, MA: Harvard University Press. Ragin, C. (1987). The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies. Berkeley, CA: University of California Press. Rose, T. (2015). The End of Average. New York: HarperCollins. Salmon, W. C. (1984). Scientific Explanation and the Causal Structure of the World. Princeton, NJ: Princeton University Press. Salmon, W. C. (1989). Four decades of scientific explanation. In P. Kitcher & W. C. Salmon (Eds.), Scientific Explanation (vol. XIII, pp. 3–219). Minnesota Studies in the Philosophy of Science. Minneapolis, MN: University of Minnesota Press. Salmon, W. C. (1998). Causality and explanation. New York: Oxford University Press.

168

Joseph A. Maxwell

Schwandt, T. (2007). The SAGE Dictionary of Qualitative Research (3rd ed.). Thousand Oaks, CA: SAGE Publications. Shadish, W. R., Cook, T. D.,  & Campbell, D. T. (2002). Experimental and QuasiExperimental Designs for Generalized Causal Inference. Boston, MA: Houghton Mifflin. Shettleworth, S. (2010). Cognition, Evolution, and Behavior (2nd ed.). Oxford: Oxford University Press. Steinmetz, G. (2005). Introduction: Positivism and its others in the social sciences. In G. Steinmetz (Ed.), The Politics of Method in the Human Sciences. Durham, NC: Duke University Press. Tashakkori, A., Johnson, R. B., & Teddlie, C. B. (2021). Foundations of Mixed Methods Research: Integrating Quantitative and Qualitative Approaches in the Social and Behavioral Sciences (2nd ed.). Thousand Oaks, CA: SAGE Publications. Walker, H. M. (1929). Studies in the History of Statistical Method. Baltimore, MD: Williams & Wilkins. Wallace, A. F. C. (1970). Culture and Personality (2nd ed.). New York: Random House. Wallner, F., Schimmer, J., & Costazza, M. (1993). Grenzziehungen zum konstruktiven realismus. Wien, Austria: WUV Universitatsverlag. Weisberg, H. I. (2014). Willful Ignorance: The Mismeasure of Uncertainty. Hoboken, NJ: John Wiley & Sons. Wikipedia. (2022a). Critical realism (philosophy of the social sciences) (accessed 18 July 2022). Wikipedia. (2022b). Roy Bhaskar (accessed 18 July 2022). Wohlleben, P. (2015). The Hidden Life of Trees: What They Feel, How They Communicate. Vancouver and Berkeley: Greystone Books. Yerkes, R. M. (1916). The Mental Life of Monkeys and Apes: A  Study in Ideational Behavior. Behavior Monographs Volume 3, Number 1, 1916. Serial Number 12. Edited by John B. Watson. Baltimore, MD: The Johns Hopkins University Press. Yin, R. (1993). Applications of Case Study Research. Thousand Oaks, CA: Sage.

PART II

Thus Spoke Philosophers

9 MIXED METHODS RESEARCH AND DEWEYAN PRAGMATISM RECONSIDERED Gert Biesta

9.1

Introduction

Over the past 25 years, ‘mixed methods’ has become a popular and widely used research approach in a range of different academic disciplines and fields. In its most basic form, mixed-methods research entails a combination of ‘qualitative’ and ‘quantitative’ approaches,1 with the ambition to generate a more accurate and adequate depiction of social phenomena that would be possible by using only one of these. The rise of mixed methods research should be understood against the background of the so-called ‘paradigms wars’ which dominated the field of social research in the second half of the 20th century (see e.g. Gage, 1989; Hammersley, 1992). During this period, researchers often held strongly opposing views about what would count as good and worthwhile research, not seldom characterising the views of their opponents as unscientific, irrelevant, or not being research at all. The advance of mixed methods approaches thus has brought about a degree of pacification in the field (see Denzin, 2008), acknowledging that both ‘qualitative’ and ‘quantitative’ approaches have their strengths and weaknesses, so that a combination of the two might be the more fruitful option. Proponents of mixed methods approaches thus advocate a pragmatic rather than a principled approach, arguing that decisions about design and methods should be driven by research aims, objectives, and questions and not by the a priori choice for a particular research ‘paradigm’ (see also Biesta, 2015). Johnson and Onwuegbuzie (2004, p. 17) thus have argued that one should ‘choose the combination or mixture of methods and procedures that works best for answering your research questions’. Tashakkori and Teddlie (1998,

DOI: 10.4324/9781003273288-11

172

Gert Biesta

p. 20), in a rather strong formulation, have called this idea the ‘dictatorship of the research question’ (see also Bryman, 2006). In discussions about the virtues of mixed methods research, authors have not only suggested that mixed methods research entails a pragmatic approach to the selection of methods, data, and design, but have also regularly argued that pragmatism might be the most appropriate philosophical ‘paradigm’ for mixed methods research (for an early exploration, see Maxcy, 2003; see also Greene & Hall, 2010). Tashakkori and Teddlie (1998, p. 167), for example, have suggested that ‘the paradigm of pragmatism can be employed as the philosophical underpinning for using mixed methods and mixed models’ (see also pp. 20–39; for a similar view see e.g. Johnson & Onwuegbuzie, 2004; Morgan, 2007; and for a more cautious approach, see Gorard & Taylor, 2004, p. 144). It is also the suggestion made by Greene (2008, p. 8), who identifies pragmatism as a ‘leading contender for the philosophical champion of the mixed methods arena’. In his recent ‘concise introduction’, Creswell (2022, pp. 10, 138) refers to pragmatism as a ‘popular worldview’ in mixed methods research, describing it as ‘an American philosophy focusing on the importance of the research question, collecting multiple forms of data to address the question, and applying the findings in a “real-world” practical way’, or, more precisely, ‘looking for the consequences of research, and seeing what works in real-world practice’ (on problems with the very idea of ‘what works’, see Biesta, 2007, 2010b, 2020). Pragmatism is a philosophical movement or ‘school’ that emerged in the late 19th and early 20th century in North America. Some have argued that it is the first original philosophical approach emerging from North America, and some even see it as a typically ‘American’ approach. The main proponents of philosophical pragmatism are Charles Sanders Peirce (1839–1914), William James (1842–1910), John Dewey (1859–1952), and George Herbert Mead (1863–1931). There are many rather simplistic ideas about what pragmatism entails, such as the suggestion that pragmatists would hold that ‘truth’ is about ‘what works’ or that ‘what works’ simply is ‘true’. In the field of mixed methods research, some authors have even claimed that pragmatism ‘is not committed to any sort of philosophical stance’ at all. This is obviously not correct, and in a sense a rather odd claim to make about a philosophical tradition. While the ideas of Peirce, James, Dewey, and Mead have much to offer to social research (and research in other domains as well), particularly because of their innovative ideas about the relationship between knowledge and action, the suggestion that pragmatism is the paradigm for mixed methods research is, as I  will discuss in more detail in this chapter, not correct and actually unhelpful. This partly has to do with the fact that the whole idea of a research ‘paradigm’ is itself a rather unhelpful notion. It also has to do with the fact that the labels ‘qualitative’ and ‘quantitative’ are rather unhelpful in

Mixed Methods Research and Deweyan Pragmatism Reconsidered

173

distinguishing between different research approaches. And it has to do with the fact that answers to the question as to what actually is being ‘mixed’ or combined in mixed methods research are often rather superficial and lacking precision. My main ambition with this chapter is to provide a better understanding of what pragmatist philosophy may have to offer for mixed methods research. To do this, I  will focus on the work of John Dewey who, in my view, has developed the most detailed account of how we might best understand what it means to know, where Dewey suggests that all of our knowledge is the outcome of our interactions with the natural and social world. Dewey therefore rejects the so-called ‘spectator theory of knowledge’ – knowledge as observation of the world outside of us – in favour of what he characterises as a ‘transactional’ account of knowledge. This approach has important consequences for our understanding of what it means to know, but also for how we judge the value of different claims to knowledge, including those generated by (scientific) research, and it is here that pragmatism has something to contribute to the field of mixed methods research. In order to provide a context for my discussion of Dewey’s pragmatism and its relevance for mixed methods research, I will, in the next section of this chapter, say a few things about the question what actually is being mixed in mixed methods research. This will not just make clear why ‘qualitative’ and ‘quantitative’ are unhelpful labels but will also bring into view where and how ‘mixing’ is rather unproblematic and where and how it raises rather fundamental problems. These, as I will suggest, have much to do with assumptions about knowledge (the domain of epistemology) and assumptions about reality (the domain of ontology), which is one reason why pragmatism has something to contribute to these discussions. Against this background, I will then suggest that rather than approaching the question of research in terms of data or methods, and even less so in terms of paradigms and paradigmatic ‘positions’, it is more fruitful to focus on the purposes of research. Here I will argue for the importance of the distinction between explanation, understanding, and emancipation as three key purposes for research and will suggest that any discussion about mixing should always be made with reference to such purposes. In the fourth section of this chapter, I will present a reconstruction of Dewey’s views on knowledge and action. I conclude this chapter with a discussion about the degree to which and the way in which Dewey’s pragmatism may be helpful for the field of mixed methods research. I will reject the claim that pragmatism is the paradigm for mixed methods, first and foremost because I think that thinking about research in terms of paradigms is actually unhelpful and untenable. I will try to show in what ways Deweyan pragmatism has a role to play in the field of mixed methods research, arguing for a more modest role than what some envisage, but nonetheless a meaningful one. Dewey’s pragmatism is particularly helpful

174

Gert Biesta

because of its emphasis on the fact that knowing is a truly human endeavour. In that way, Dewey is a strong critic of ‘scientism’, which is the idea that science would provide us with the only valid kind of knowledge and the only true and valid access to reality. In this regard, Dewey sides with those who reject the idea that ‘quantitative’ research is the only valid kind of research, which, as I will argue, is probably one of the most important ways in which there is a connection between mixed methods research and pragmatism. I wish to highlight that, with regard to the more fundamental questions surrounding mixed methods approaches, there are some enduring issues that deserve attention. I have explored some of these in earlier publications (including Biesta & Burbules, 2003; Biesta, 2010a, 2015, 2020) and will, in this chapter, be relying on some insights developed there. I nonetheless hope that it is helpful to bring these together in the context of this chapter and for the purposes of this book. 9.2

What Is Being Mixed in Mixed Methods Research?

A quick and popular way to characterise mixed methods research is to present it as a combination of ‘qualitative’ and ‘qualitative’ approaches. Indeed, Johnson et  al. (2007, p.  123) have suggested defining mixed-methods research as ‘the type of research in which a researcher or team of researchers combines elements of qualitative and quantitative research approaches (e.g. use of qualitative and quantitative viewpoints, data collection, analysis, inference techniques) for the broad purposes of breadth and depth of understanding and corroboration’. They distinguish this from a mixed-method study which involves ‘mixing within a single study’ and from a mixed-method programme which involves ‘mixing within a program of research [where] the mixing might occur across a closely related set of studies’ (Johnson et  al., 2007, p. 123). In discussions about methods and methodologies, it is indeed common to refer to qualitative and quantitative research as two different research approaches. Considering this, it seems obvious, then, to characterise mixed methods research as a combination of the two. While ‘qualitative’ and ‘quantitative’ are commonly being used as shorthand labels for particular approaches to research, it is important to bear in mind, however, that research itself can neither be qualitative nor be quantitative. The only entities to which these labels can accurately be applied are research data. Data can either be quantities – expressed in numbers – or qualities – usually expressed in words (albeit that there is research that makes use of other modalities of representation, such as images). If we want to use the words ‘quantitative’ and ‘quantitative’ in relation to research, the correct way would be to talk about research that uses quantitative data and research that used qualitative data.

Mixed Methods Research and Deweyan Pragmatism Reconsidered

175

Creswell (2022, p. 1) does indeed provide a definition of mixed methods research that focuses on data, explaining that he sees mixed methods research ‘as a method with a focus on data collection, analysis, and interpretation in response to research questions’. His more detailed definition is to see mixed methods research as a methodology and method to research in the social, behavioral, and health sciences in which the investigator gathers both quantitative (closedended) and qualitative (open-ended) data, integrates or combines the two, and then draws inferences (called ‘metainferences’) from the integration that provides insight beyond what can be learned from the quantitative or qualitative data. (ibid., p. 2) A ‘core assumption’ for him is that this ‘provides a better understanding of the research problem than either types of data alone’ (ibid.). What is helpful about Creswell’s approach is that he stays close to the actual practice of research and thus provides a definition of mixed methods research that many researchers will recognise. More generally, he takes a rather down-to-earth approach to research, suggesting that ‘the general process of research’ for both qualitative and quantitative researchers consists of the following steps: ‘identify a problem, determine research questions, collect data, analyse data, and interpret research’ (ibid., pp.  4–5). While Creswell is correct in highlighting the combination of different kinds of data within mixed methods research, his account glosses over the fact that the whole question of what is and what can be mixed or combined in mixed methods research is actually a bit more complicated than just using words and numbers to ‘draw inferences’ or gain ‘a better understanding of the research problem’.2 Elsewhere (Biesta, 2010a), I  have suggested that it might be helpful to distinguish between seven levels at which mixing and combining in mixed methods research might take place, also in order to ask more precise questions about where and when mixing is simple and uncontroversial and where and when there might be problems. The seven levels I have distinguished are: 1. Data – Is it possible to combine text and numbers? 2. Methods – Is it possible to combine different methods of data collection and/or data analysis? 3. Designs – Is it possible to combine experimental/interventionist and naturalist/non-interventionist designs? 4. Epistemologies – Is it possible to combine different views about knowledge? 5. Ontologies – Is it possible to combine different views about (social) reality?

176

Gert Biesta

6. Research purposes – Is it possible to combine the intention to generate causal explanation of social phenomena with the intention to generate interpretive understanding? 7. Practical orientations – Can research be orientated both towards the production of solutions, techniques, and technologies, and the development of critical understanding, analysis, and theory? While it is relatively uncontroversial to combine different data and different data-collection and data-analysis strategies within the same study (levels 1 and 2), and while it is also relatively uncontroversial to combine different designs within the same programme (level 3), things become more complicated when one reaches questions about what knowledge ‘is’ and what kind of knowledge research is able to generate, or how we should understand the reality that we see as the object of our investigation (levels 4 and 5). This, in turn, may have an impact on the extent to which different purposes of research can be combined (level 6) – something which is perhaps more feasible within a research programme than within a single study. (I will return to this in the next section.) And all this is also connected with the overarching orientation of one’s research, that is, whether one focuses more on contributing to the improvement of fields of practice or sees one’s contribution first and foremost to advancing knowledge and understanding in itself. To acknowledge that the question of mixing is more multi-layered and more complicated than what is often assumed is important for researchers who opt for a mixed approach, as it allows for more precision with regard to claims about what is actually being combined or mixed. And perhaps it can also counter a degree of ‘inflation’ of the notion of mixed methods research, because too many claims about mixed methods research tend to go not much further than saying that one uses words and numbers.3 9.3

The Importance of the Question of Purpose in Mixed Methods Research

The suggestion that in research everything starts with ‘the problem’ – or in Creswell’s formulation: the research problem – and that research is in some way about collecting and analysing information to ‘address’ the problem (which in such accounts often means something like solving the problem) – takes a rather simplistic view of what research is or can be. And it takes a rather simplistic view of what researchers actually do when they conduct research. Of course, research can play a role in and can be a form of problem-solving, and quite often, the word ‘research’ is invoked when there are complex problems that not just need a solution but also need a more detailed understanding of what the problem actually is, whose problem it is, what actually brought the problem about, and what avenues there are for addressing the problem. In

Mixed Methods Research and Deweyan Pragmatism Reconsidered

177

those situations, it does make sense to collect information and analyse the information in a careful manner. However, what distinguishes more or less complex forms of systematic problem-solving from research is the fact that research always also has a cognitive ambition. Research always also wishes to contribute to knowledge and understanding and some would argue that this is actually the main reason for doing research in the first place. How the knowledge and understanding we gain through research can contribute to addressing and solving practical problems is seen as an important but nonetheless secondary matter. It is with regard to the cognitive dimensions of research – research as a process of generating knowledge and understanding – that there are a small number of distinctively different options for and views about what research can achieve and should aim for. I find it most helpful to think of these different options and views in terms of different purposes of research. I also think that the question of purpose, understood in this way, is often absent or marginal in discussions about mixed methods research, which is unfortunate. With regard to the cognitive ambitions of research, I wish to suggest that it is helpful to make a distinction between three different overarching purposes, namely explanation, understanding, and emancipation. The idea that the task of research is to explain has its roots in the modern natural sciences, where explanation is generally understood in causal terms, that is, as the identification of connections between causes and effects – and in ‘strong’ interpretations of causality, as necessary connections between causes and effects; a way of thinking we can find, for example, in the idea of laws of nature. The ambition behind explanatory research is that once we are able to identify necessary connections between causes and effects – that is, if we are able to generate perfect explanations – we are, in principle, in a position to predict future events based on what is happening currently and, to the extent to which the causes can be manipulated, we are also able to control future events. Explanatory knowledge thus forms an important basis for the development of techniques and technologies. The idea of explanation – and perhaps we should add: the ambition of explanation – rests on a particular assumption about reality, namely that reality itself is made up of causal connections between elements or events. This ontology has its history of the rise of the so-called scientific worldview which sees reality ultimately as a perfect clockwork, where everything hangs together and functions according to the laws of nature. While the development of explanatory knowledge is an incremental process, with steps forward and steps backward, the overarching ambition of modern science has always been to come to a complete explanation of ‘everything’, which is for example expressed in the ambition that ultimately research should discover the laws according to which nature functions. While this ambition still plays a role in the natural sciences, it is also clear that not all domains and aspects of natural

178

Gert Biesta

reality operate in a mechanistic, law-like manner. Strong causality – the idea that event A will always lead to event B – is not universally present at subatomic level, for example, whereas also many biological processes do not operate in a mechanistic-causal way (something that has been theorised in complexity theory for example). More important for the field of social research, however, is the question whether human phenomena and processes can be understood in the same way. Put differently: can we assume that in the domain of human action and interaction things function in the same causal was as they do in the physical world? This question goes back to a much older discussion which is often framed in terms of the question whether human action is (externally) caused or (internally) motivated. Are human beings ultimately natural beings that function as stimulus-response machines – which would make it possible, in principle, to discover the laws of human behaviour? Or do human beings have agency and the capacity to act according to their own motivation and will. Proponents of the latter view thus argue, using ideas from the seminal work of Wilhelm Dilthey (1833–1911), that human action is not caused by external forces, but that human beings have reasons for their actions. If this is the case, then any research that wants to take human beings and their actions and interactions seriously should not aim for explanation of causes but should seek to understand the reasons human beings have for their actions. Such research thus seeks to understand why human beings – individually or collectively – act as they act and do as they do. And in order to generate such understanding, we need to talk to people themselves, in order to get a sense of their reasons and motivations, but also of their perceptions and interpretations. Some believe that explanatory knowledge is the only kind of proper knowledge or the only kind of scientific knowledge and would see research that aims for understanding as potentially interesting but not scientific. This issue was at the heart of the so-called ‘paradigm wars’, where some would argue that there is only one kind of scientific knowledge and hence only one kind of scientific method, whereas others made the case that to really grasp the distinctive character of human existence, we need a different approach. It also explains why the labels ‘quantitative’ and ‘qualitative’ play a role here. After all, to identify causal connections and laws in natural phenomenon, one needs to model the relationships between variables, which often amounts to using statistical procedures and processes. But when we seek to understand why human beings act as they act and do as they do, we obviously need to talk to them, and thus need to use words and language. Rather than to suggest that such work is unscientific – which would be based on a very narrow view of science – it makes more sense to acknowledge that different phenomena need approaches that can do justice to the specific nature

Mixed Methods Research and Deweyan Pragmatism Reconsidered

179

of such phenomena, and here the life and actions of human beings are fundamentally different from how the planets move or chemical reactions take place.4 The idea that the purpose of social research should be to generate understanding of the experiences, interpretations, and motivations of actors in order to make plausible why they act in the way they do, does, however, raise one further important question, which is whether the interpretations people give of their own actions, perceptions, and motivations can be taken as a true or correct account of what is going on. It is here that Marxist philosophy and theory has raised the possibility that our understandings can actually be distorted as a result of the way in which social power structures operate on our understandings and interpretations. This is the problem of ideology, where not only ideological thought is thought that is socially determined – that is, thought that is ‘produced’ by social forces – but also ideological thought is thought which, in the words of Karl Marx, ‘denies this determination’ (Marx, quoted in Eagleton, 2007, p. 80). There is therefore always the risk that when we ask people why they act as they do, we may get a distorted account of what is really going on. This is not just a methodological or methodical issue for social research. Marxist scholars have actually argued that this leads to a different task for social researchers, namely to help people to understand how their own thinking is influenced by and, more strongly, distorted by the social processes and forces around them. Here a different purpose for social research comes into view, namely that of emancipation, that is, of liberating individuals and groups from the hidden influences of social power on their thinking and doing. While some see explanation, understanding, and emancipation as three different options for research, others see possibilities for integration and for making a place for explanatory research with regard to human beings and their actions and interactions. This has, for example, been proposed by Jürgen Habermas, most notably in his books Erkenntnis und Interesse (Habermas, 1968; translated as Knowledge and Human Interests and published in 1971) and Zur Logik der Sozialwissenschaften (Habermas, 1970; translated as On the Logic of the Social Sciences and published in 1990). Habermas suggests that explanation has a role to play in social research – it can be quite interesting to try to explain the dynamics of individual and collective actions in terms of mechanisms and causes – but, when research operates exclusively in an explanatory mode, it misrepresents the specific nature of social reality. That is why explanation always needs to be embedded in research that aims for understanding, so that the interpretations of human actors can have ‘control’ over explanations generated about (parts of) their actions. Yet Habermas also argues that interpretative research needs, in turn, be embedded within modes of critical research that can make visible how power operates on people’s

180

Gert Biesta

interpretations so that ultimately the whole research effort can contribute to emancipation. For Habermas, the emancipatory ambition of social research is therefore not an approach that is different and separate from research aiming at explanation or understanding. He argues for a ‘nested’ model where explanation is nested within understanding, and understanding is nested within critical forms of research so that the total effort can contribute to emancipation. Looking at research in terms of its cognitive ambitions is, in my view, more helpful for our understanding of the potential of mixed methods research than just in terms of combining data or data-collection and data-analysis strategies. One could even argue that both when research seeks to explain and when it seeks to understand it can make use of qualitative and quantitative data and methods for analysing such data, so that the presence of different kinds of data in itself doesn’t say anything about the overarching ambitions and purposes of the research. What is also interesting is that, in the work of Habermas, we can already find an argument for a mixed approach to research, and I think that his work – including his emphasis on the potential of research to increase human freedom (emancipation) – remains an important reference point for contemporary discussions about the vices and virtues of mixed methods approaches. 9.4

Knowing as a Way of Doing: The Contribution From Pragmatism

Having said all this, I now turn to pragmatism and more specifically the work of John Dewey in order to gain a sense of what Dewey’s work may have to contribute to mixed methods research. I hope that by now it is clear that there is no point in looking at Dewey if we haven’t got an understanding of what mixed methods research is and isn’t about. The line of thought presented in this chapter so far is intended to do this, so that we can evaluate what pragmatism may have to offer to mixed methods research. 9.4.1

(How) Is Knowledge a Problem?

In modern philosophy, the question of knowledge is generally phrased as how the human mind can acquire knowledge of a world outside of itself. Robert Nozick put the challenge most succinctly when he asked whether we can ever know that we are not a brain suspended in a vat full of liquid, wired to a computer which is feeding our current experiences (see Nozick, 1981, pp. 161–171). Nozick is part of a long tradition in which the nature of knowledge is examined from a sceptical point of view, starting from the assumption that knowledge may not be possible, because we may not be able to get ‘outside’ of our own mind.

Mixed Methods Research and Deweyan Pragmatism Reconsidered

181

The first philosopher to place scepticism at the heart of modern epistemology was René Descartes. In the Second Meditation, he used the ‘method of doubt’ to arrive at the conclusion that although we can doubt everything, we cannot, when doing so, doubt that we are engaged in a process of doubting. Whereas this provided Descartes with certainty about the existence of the thinking self, it did not provide any certainty about the existence of a world beyond our experience, and this issue has troubled modern epistemology ever since. It eventually led David Hume to the conclusion that the existence of an external world of enduring object is a ‘very useful hypothesis’, but not something that can ever be proven. What unites the ideas of Nozick, Descartes, and Hume is their reliance on a dualistic view of reality. They assume that reality consists of two totally different ‘substances’, mind and matter, and that the question of knowledge has to begin with the mind in order then to ask how the mind can get in touch with the material world ‘outside’ of itself. The dualism between mind and matter has not only set the agenda for modern epistemology by giving it the task to answer the question how the mind can get ‘in touch’ with the world (see e.g. Dancy, 1985). The dualism has also provided the framework for the distinction between objectivity and subjectivity and, related to this, for distinctions such as between absolutism and relativism, or between realism and idealism. After all, on the basis of the dualism between knowing subjects and objects to be known, knowledge is objective if it depicts how objects ‘really’ are in themselves, whereas, if this is considered not to be possible, then the only other option is for knowledge to be subjective, that is, produced by the activities of the human mind. The mind-world scheme does indeed only offer two options: either knowledge is objective or it is subjective; either it is about the world ‘out there’ or it is made up by the activities of the human mind. The crucial question, however, is not which option to choose. The far more important question is whether the mind-world scheme is itself inevitable or whether it is possible to think about knowledge and reality in a different way. John Dewey’s theory of knowing does precisely this. It offers an understanding of knowing that does not start from what he saw as the ‘impossible question’ as to how ‘a knower who is purely individual or ‘subjective’, and whose being is wholly psychical and immaterial . . . and a world to be known which is purely universal or ‘objective’, and whose being is wholly mechanical and physical’ can ever reach each other (Dewey, 1911, p. 441). Instead, Dewey put forward a framework which starts with interactions – or as he later preferred to call it: transactions – taking place in nature and in which nature itself is understood as ‘a moving whole of interacting parts’ (Dewey, 1929, p. 232). This is Dewey’s self-confessed ‘Copernican turn’, in which ‘(t)he old center was mind’ while ‘(t)he new center is indefinite interactions’ (Dewey, 1929, p. 232). The key concept in this Copernican turn is ‘experience’.

182

9.4.2

Gert Biesta

A Transactional Theory of Knowing

While transaction refers to interactions taking place in nature more generally, experience refers to the transactions of living organisms and their environments. What is distinctive about these transactions is that they constitute a double relationship. The organism acts in accordance with its own structure, simple or complex, upon its surroundings. As a consequence the changes produced in the environment react upon the organism and its activities. The living creature undergoes, suffers, the consequences of its own behavior. This close connection between doing and suffering or undergoing forms what we call experience. (Dewey, 1920, p. 129) Experience is therefore the way in which living organisms are implicated in their environment. Contrary to what is suggested in the mind-world scheme, Dewey thus argues that experience, so understood, is not ‘a veil that shuts man off from nature’, but rather ‘a means of penetrating continually further into the heart of nature’ (Dewey, 1925, p. 15). Dewey saw knowing as the mode of experience that in some way ‘supports’ action. It is concerned with grasping the relationship between our actions and their consequences. It is because of this that knowing can help us to get more control over our actions, at least more than in the case of blind trial and error. It is important to see that ‘control’ here does not mean complete mastery, but the ability to intelligently plan and direct our actions. This ability is first of all important in those situations in which we are not sure how to act – which is expressed in one of Dewey’s definitions of knowing as having to do with ‘the transformation of disturbed and unsettled situations into those more controlled and more significant’ (Dewey, 1929, p. 236). Knowing is also important in order to achieve more control, a more intelligent approach in the other domains of experience, which is expressed in Dewey’s claim that knowing ‘facilitates control of objects for purposes of non-cognitive experience’ (Dewey, 1929, p. 79). The framework for Dewey’s theory of knowing lies in his theory of action, the outlines of which he developed early on in his career in a landmark paper called ‘The Reflex Arc Concept in Psychology’ (Dewey, 1896). One way to summarise Dewey’s theory of action is to say that it amounts to a theory of experimental learning if, that is, we think of learning as the way in which living organisms interactively ‘adapt’ to their environments (which in itself is a rather truncated conception of learning, of course; on this see Biesta, 2013). Dewey characterises living organisms – including human organisms – as capable of establishing and maintaining a dynamic coordination with their

Mixed Methods Research and Deweyan Pragmatism Reconsidered

183

environment. Through this process, the predispositions – or ‘habits’ as Dewey preferred to call them – of the organism become more focused and more specific, more attuned to ever-changing environing conditions, which is another way of saying that through the tentative, experimental way in which living organisms maintain coordinated transaction with their environment they learn. This learning, however, is not the acquisition of information about how the world ‘out there’ is. It is a learning process through which living organisms acquire a complex and flexible set of predispositions-for-action. On this view, learning is therefore basically a process of trial-and-error, and in one sense, this is indeed how Dewey argues that living organisms learn. But there is a difference between blind trial and error and what Dewey called ‘intelligent action’. The difference has to do with the intervention of thinking, which he defines as ‘dramatic rehearsal (in imagination) of various competing possible lines of action’ (Dewey, 1922, p. 132). The choice for a specific line of action should be understood as ‘hitting in imagination upon an object which furnishes an adequate stimulus to the recovery of overt action’ (Dewey, 1922, p. 134). Whether this choice will actually lead to coordinated transaction will only become clear when the organism actually acts. This is why thinking can never guarantee that our actions will result in coordinated transactions. But what it can do is to make the process of choosing more intelligent than would be the case with ‘blind’ trial-and-error. 9.4.3

Knowledge and Action

In Dewey’s view, the question of knowledge – or to be more precise: the issue of knowing – arises because of the appearance of incompatible factors within the empirical situation. . . . Then opposed responses are provoked which cannot be taken simultaneously in overt action, and which accordingly can be dealt with, whether simultaneously or successively, only after they have been brought into a plan of organized action. (Dewey, 1916, p. 326) The problem here is one of the meaning of the situation – and for Dewey ‘situation’ always refers to organism and environment in transaction. The only way to solve the problem in an intelligent manner and not by simple trialand-error is by means of a systematic inspection of the situation. On the one hand, we need to identify and state the problem. On the other hand, we need to develop suggestions for addressing the problem, for finding a way to act, and hence to find out what the meaning of the situation actually is. While thought or reflection must play an important part in this process, they will, in themselves, not result in knowledge. It is only when action follows that the

184

Gert Biesta

value of both the analysis of the problem and the suggested solution can be established. For Dewey, therefore, we need overt action in order to determine the worth and validity of our reflective considerations. Otherwise, we have, at most, a hypothesis about the problem and a hypothesis about its possible solution. This means that, in order to get knowledge, we need action. But although action is a necessary condition for knowledge, it is not a sufficient one. We also need thinking or reflection. It is the combination of reflection and action which leads to knowledge. From this, it follows that knowing, the acquisition of knowledge, is not something which takes place somewhere deep down inside the human mind. Knowing is itself an activity, it is ‘literally something which we do’ (Dewey, 1916, p.  367). The meaning which emerges from the restoration of coordinated action is a meaning ‘which is contemporaneously aware of meaning something beyond itself’ (Dewey, 1906, p. 113). This ‘beyond’ is not simply present or will not simply become present in the future. It will only become present ‘through the intervention of an operation’ (Dewey, 1906, pp. 113–114), that is, through what we do. A potato becomes edible when we cook it, so after the intervention of the act of cooking – and perhaps we can say: after the discovery that when we cook potatoes we can eat them – the potato means something different in our field of action – it has become ‘potentially edible food’. Therefore, when experience is ‘cognitional’, as Dewey puts it, it means that we perceive something as meaning-somethingelse-which-we-will-experience-when-we-act-in-a-specific-way. It is along these lines that knowledge is intimately connected to the possibility of control. ‘In knowledge’, Dewey argued, ‘causes become means and effects become consequences, and thereby things having meaning’ (Dewey, 1929, p.  236). Knowledge has, in other words, to do with inference: a reaction to something which is distant in time or place. Because inference is a step into an unknown future, it is a precarious journey. Inference always involves uncertainty and risk. A stone, Dewey argued, can only react to stimuli of the present, not of the future, and for that reason cannot make mistakes. Since inference entails the possibility of mistake, it introduces truth and falsity into the world. 9.4.3.1

Experience, Reality, and Knowledge

One important implication of Dewey’s transactional definition of experience is that it puts an end to the idea that it is only through knowledge that we can obtain a hold on reality. For Dewey, all modes of experience are equally real, since they are all modes of the transaction of living organisms and their environments. From this, Dewey concluded that ‘things – anything, everything, in the ordinary or non-technical use of the term “thing” – are what they are experienced as’ (Dewey, 1905, p. 158). This first of all means that everyone’s

Mixed Methods Research and Deweyan Pragmatism Reconsidered

185

experience is equally real. It also implies that what is experienced is itself real. If someone is frightened by a noise, so Dewey’s argument is, then that noise is fearsome. This claim must be understood transactionally. If someone is frightened by a sound, then the fear is the immediate response of the organism. The sound is frightening because the organism reacts to the sound as being-afrightening-sound. This implies, however, that being-frightened is not the same as knowing-that-one-is-frightened. Knowing what caused the fearsome noise is a different experience. While the latter experience may be more true than the former, it is in Dewey’s view not more real. ‘The question of truth is not as to whether Being or Non-Being, Reality or mere Appearance is experienced, but as to the worth of a certain concretely experienced thing’ (Dewey, 1905, p. 163). One important implication of this is that experience in itself does not provide us with any knowledge. Dewey rejected, in other words, the view that experience provides us with elementary ‘bits’ of knowledge which, when put together in a systematic of logical manner, result in knowledge.5 For Dewey, the difference between experience and knowledge is that knowledge is concerned with the occurrence of experience. The ‘office’ of knowledge signifies a search ‘for those relations upon which the occurrence of real qualities and values depends’ (Dewey, 1929, p. 83). In this respect, knowledge is intimately and necessarily connected with action, because – and this is the most crucial point in Dewey’s theory of knowing – the discovery of the conditions and consequences of experience ‘can take place only by modifying the given qualities in such ways that relations become manifest’ (Dewey, 1929, p. 84; emphasis added). The shift from understanding knowledge as being concerned with the world ‘as it is’ to understanding knowledge as being concerned with conditions and consequences is a very important element of Dewey’s approach. It represents a shift from a concern with things as they are to a concern with ‘the history to which a given thing belongs’ (Dewey, 1925, p. 243). It is a shift from knowing as an aesthetic enjoyment of the properties of nature as a world of divine art, to knowing as a means of secular control – that is, a method of purposefully introducing changes which will alter the direction of the course of events. (Dewey, 1929, p. 81) This implies that, for Dewey, knowledge is concerned with the relations between actions and consequences. This introduces the dimension of time into Dewey’s theory of knowing – a reason for arguing that Dewey has a temporal conception of knowing.

186

9.4.4

Gert Biesta

The Objects of Knowledge

Dewey’s approach also has implications for how we understand the objects of knowledge. Whereas in the dualistic approach, the objects of knowledge are seen as ‘things’ that exist in a world ‘out there’ and are there for us to discover and depict, Dewey’s transactional view sees the objects of knowledge as the outcomes of processes of inquiry. Since the habits we acquire through such processes provide us with more specific predispositions for action, habits in a sense embody the ways in which our environment becomes more meaningful for us. The experimental transformation of organism-environment transactions transforms the environment in which and through which we act into what Dewey referred to as ‘a figured framework of objects’ (Dewey, 1922, p. 128). This is the reason why Dewey referred to objects of perception not as things but as ‘events with meaning’ (Dewey, 1925, p. 240). In the case of spoken language, it is relatively easy to see that words – or ‘sound-events’ – do not have a meaning of their own, but that they have become meaningful over time. It is far more difficult to draw the same conclusion with respect to physical objects, such as chairs, tables, trees, stones, hills, and flowers, ‘where it seems as if the union of intellectual meaning with physical fact were aboriginal’ (Dewey, 1933, p.  231). Yet chairs and tables are as much events with meaning as words are. And their meaning has a strictly transactional origin, in that it has to be understood as the outcome of the specific ways in which successful relationship between our actions and their consequences have been established over time. It is not, therefore, that, through a process of inquiry, we can find out what the possible meanings of, for example, a chair are. Rather, a chair specifies a particular way in which the transaction with the environment has become meaningful. It is for this reason that Dewey argued that we should think of objects as tools. ‘The character of an object is like that of a tool . . .; it is an order of determination of sequential changes terminating in a foreseen consequence’ (Dewey, 1925, p. 121).

9.4.5

The Question of Truth

The final element of Dewey’s theory of knowing has to do with the question of truth. We have already seen that, for Dewey, there is no sense in asking about the truth of our immediate experience. Immediate experience simply is what it is. Truth and falsity only enter the scene when we raise questions about the meaning of experience. Truth and falsity are not properties of any experience or thing, in and of itself or in its first intention; but of things where the problem of assurance consciously enters in. Truth and falsity present themselves as significant

Mixed Methods Research and Deweyan Pragmatism Reconsidered

187

facts only in situations in which specific meanings are intentionally compared and contrasted with reference to the question of worth, as to the reliability of meaning. (Dewey, 1906, p. 118; emphasis in original) Truth and falsity are therefore not concerned with things as such but with the relationship between our experience of a thing on the one hand and our possible actions or responses on the other. This not only means that ‘truth’ is always contextual and related to action. It also means that truth is itself temporal. Truth does not refer to an alleged correspondence between a proposition and reality. It has to do with the correspondence between suggested meaning and realised meaning, that is, meaning ‘put into practice’. ‘The agreement, correspondence, is between purpose, plan, and its own execution, fulfilment’ (Dewey, 1907, p. 84). This does not mean that truth becomes disconnected from reality. The contrary is the case, not only because of the transactional framework that informs Dewey’s theory of knowing but also because of the indispensable role of action in the process that results in knowledge. The upshot of this is that that knowledge is not a passive registration of reality ‘out there’. Our intervention, our action, is a crucial, necessary, and constitutive part of knowledge. In this sense, we can say that knowledge is always a human construction just as the objects of knowledge are. But it does not mean that anything is possible. We always intervene in an existing course of events and although our intervention introduces change, it will always be change of an existing course of events. We cannot create out of nothing. For Dewey, the only possible construction is a reconstruction. 9.4.5.1

Consequences of Pragmatism

One of the most important implications of Dewey’s transactional approach is that knowledge does not provide us with a picture of reality as it is in itself – an idea to which Dewey referred as the ‘spectator theory of knowledge’. For Dewey, knowledge always concerns the relationship between (our) actions and (their) consequences. This, in essence, is what a transactional conception of knowledge implies. It means that knowledge is a construction; or, to be more precise, that the objects of knowledge are constructions. But contrary to how constructivism is often understood under the mind-world scheme (viz., as purely mental and hence subjective), Dewey’s constructivism is a transactional constructivism, a constructivism which holds that knowledge is at the very same time constructed and real. This is why we can call Dewey’s position a form of realism, albeit transactional realism (see Sleeper, 1986). Given that knowledge concerns the relationship between (our) actions and (their) consequences, knowledge will only ever offer us possibilities but not

188

Gert Biesta

certainty. The conclusions we draw on the basis of careful observation of what follows from how we act upon the world, show what has been possible in this particular transactional situation. Sometimes what was possible in one situation turns out also to be possible in another situation, but in other situations, the transactional determinants of the situation are different, so that what was possible in one case is no longer possible in another case (see also Biesta, 2007 on the implications of this idea for the discussion about ‘what works’). This is why Dewey preferred to refer to the outcomes of inquiry and research as ‘warranted assertions’ rather than truth. The assertions we make about the consequences of our actions are warranted on the basis of careful observation and control. But they are only warranted in relation to the particular situation in which they were ‘produced’ and we shouldn’t make the mistake – for example by putting the label ‘true’ on them – to think that they will be warranted for all time and all similar situations. This does not mean that conclusions from one situation cannot be useful for other situations. But the way in which knowledge from one situation transfers to another situation is in that it can guide our observation and perception and can suggest possible ways for resolving problems, for finding ways forward. Whether these possibilities will address the specific problems in the specific, new transactional situation can only be discovered when we act. A more general feature of Dewey’s transactional approach to knowing is that, contrary to mainstream modern philosophy, his approach is not a sceptical one. For Dewey, there is no gap between human beings and the world. This does not mean that everything we experience is simply ‘true’. While Dewey does hold that things are what they are experienced as, there is a crucial difference between experience and knowledge. While experience simply ‘occurs’, knowledge, because it has to do with inference, can always be fallible. In this respect we have to conclude that Dewey’s transactional theory of knowing is a form of fallibilism. But it is important to see that for Dewey knowledge is not fallible because of an alleged gap between ourselves and the world, but because we can never be sure what the future will bring, not in the least because what the future will look like depends also on our own ongoing actions. According to the transactional approach, we are not spectators of a finished universe, but participants in an ever-evolving, unfinished universe. Dewey’s transactional approach also cuts across the either/or of objectivism and subjectivism. From a transactional point of view, ‘the world’ always appears as a function of what we do. Objectivity, understood as a depiction of a world completely independent of and untouched by us, is therefore simply impossible. If we want to know the world, we must interact and, as a result, we will only know the world in the way in which it responds to us. The world we construct emerges out of the doing-undergoing-doing dynamics of what Dewey calls ‘experience’. One could argue – and many critics of Dewey had

Mixed Methods Research and Deweyan Pragmatism Reconsidered

189

done so – that although Dewey rejects objectivism, he thus ends up in a situation of complete subjectivism. Dewey simply acknowledges that this is the case – but he adds that there is no problem with this at all, as long as we see that the worlds we construct are constructed for our own individual purposes, for our own attempts to address the problems we are faced with. It is only when we start to interact with others that the need for some form of coordination of our subjective worlds with the subjective worlds of others arises. What happens in this case is that, through interaction, co-operation, coordination, and communication, we construct an intersubjective world out of our individual, subjective worlds. By showing that objectivity is simply not possible, that subjectivity is not always a problem, and that intersubjectivity addresses those instances where the subjectivity of knowledge does become a problem, Dewey not only presents us with a position that helps us to overcome the stalemate between objectivism and subjectivism. He also shows that we do not have to give up the world when we want to acknowledge that knowledge is always plural, changing, and open, and that knowing, most importantly, is always a thoroughly human endeavour. 9.5

Discussion: Reconsidering Pragmatism and Mixed Methods Research

In this chapter, I have (re)considered the question whether pragmatism can be seen as an appropriate and perhaps even as the best paradigm for mixed methods research. The short answer to this question is: no. This is first of all because I think that the language of paradigms remains unhelpful for discussions about social research. One important impetus for the development of mixed methods approaches has precisely been to overcome the stalemate over the ‘paradigm wars’, so already from that angle it would be odd to revert to the language of paradigms for mixed methods research and, moreover, to claim a particular philosophical ‘school’ as its ‘own’ paradigm. Creswell’s suggestion to talk about ‘worldviews’ rather than paradigms (see Creswell, 2022, p. 10) is to a certain degree helpful, because it offers a different language than that of paradigms. But even here we should be mindful that worldviews are not things that we can simply pluck from the air or choose because we like them. The question what kind of assumptions we rely on in our research are less a matter of choice and much more a matter of asking careful questions about what research is and what it is for. This is why in this chapter I wasn’t able to go straight to pragmatism, but first had to consider two other matters, one being the whole question of ‘mixing’ in mixed methods research and the other the question of the cognitive ambitions of research, so as to move beyond the too simple idea that research is just a matter of more or less sophisticated problem-solving. With regard to the question of mixing, I have tried to show that there are quite a

190

Gert Biesta

lot of different aspects to what can be combined or mixed in mixed methods research. This already reveals that just talking about mixed methods research as the combination of qualitative and quantitative approaches or the utilisation of qualitative and quantitative date is too superficial. Hence, there is the need to emphasise the importance of the different cognitive ambitions for research, as it is only from that perspective that we can begin to understand what we are mixing and why we are mixing it. The distinction between explanation and understanding as two different ambitions for research remains an important one, also because the so-called paradigm wars were to a large extent about the question whether (causal) explanation sets the standard for all (scientific) research, or whether a case can be made that human action requires a different approach, one that focuses on understanding of reasons, motives, and perceptions of human beings. This remains an important question for mixed methods research. At the heart of it lies the question whether researchers can and should treat human beings in the same way as they treat planets, molecules, and machines, that is as objects. If that is the case, then mixed methods research can at most be a more sophisticated way of conducting explanatory research, leading to more refined ways of explaining, predicting, and ultimately controlling the objects of study. If, on the other hand, mixed methods research does acknowledge that human beings are beings with agency and the capacity to make up their own minds, then mixed methods research may start working on the side of emancipation rather than control. In this regard, we might say that the more fundamental questions for mixed methods research remain paradigmatic, not in terms of what kind of data are worthy of inclusion, but first of all in terms of the overarching cognitive ambitions for research. It is interesting to see that the distinction between explanation and understanding, and thus between treating human action as something that can be explained from the outside or should be understood from the inside, is not a major theme in John Dewey’s views about knowledge and action. More generally, Dewey’s pragmatism is not a methodology, that is, not a theory about how to collect data and draw inferences and conclusions from the data. Dewey rather provides important insights about what it means to know and what it means to gain knowledge. One thing that is relevant in Dewey’s work is his critique of scientism, that is, of the idea that science would provide us with the only valid kind of knowledge and, moreover, with the ultimate account of reality as it really us. Dewey rejects the idea that knowledge can give us the ultimate account of how reality really us, because he shows that the ways in which we encounter reality and gain knowledge about it, is entirely connected to our actions. Everything we know, so Dewey argues, is about relationships between actions and consequences and not about an independent world ‘out there’. Knowledge thus provides us with

Mixed Methods Research and Deweyan Pragmatism Reconsidered

191

insights in possible relationships between actions and consequences, which is of course tremendously useful. But knowledge will never give us 100% certainty – Dewey rather argues that the ‘quest for certainty’ is a mistaken quest. Dewey thus present knowing and coming to know as thoroughly human endeavours and in this regard would see a continuity between everyday knowledge gathering and the ways in which researchers gain knowledge. Research knowledge is not fundamentally different from everyday knowledge for Dewey, which also means that ‘scientific’ is not an epistemological label – it is not a label that denotes knowledge of a special quality – but should rather be understood as a social category. It refers to knowledge generated by those who spend more time and resources and thoughtfulness to their investigations than what individuals can do in their everyday lives. The difference is gradual, not fundamental. In all this, Dewey thus sides with those who reject the idea that ‘quantitative research’ is the only valid kind of knowledge or the only scientific kind of knowledge. Dewey thus provides important arguments for putting the scientific research enterprise into perspective – not talking it down but also not seeing it as anything more than it is. In this way, Dewey’s work can help in overcoming the hegemony of so-called quantitative approaches and to the extent to which this is at stake in mixed methods research, mixed methods researchers can find Dewey on their side. Notes 1 I put ‘qualitative’ and ‘quantitative’ in quotation marks because I will argue below that these are rather problematic and misleading labels where it concerns the question of ‘mixing’ in mixed methods research. 2 The formulation itself is a bit odd as well, as one could argue that the point of research is not to gain a better understanding of the research problem, but rather to solve the research problem (which may, of course, sometimes lead to the insight that the research problem as articulated was actually not the problem that needed addressing). 3 For a helpful list of what mixed methods research is not, see Creswell (2022, pp. 3–4). 4 The distinction between explanation and understanding as two different ambitions for research emerged in the 19th century in the context of discussions about the status of the natural sciences and the humanities (in German: ‘Geisteswissenschaften’), first of all with regard to the question of the status of historical knowledge. The discussion re-emerged in the 1960s in Germany, particularly between Karl Popper and Theodor Adorno (see Adorno, 1969; see also Apel, 1982; Habermas, 1970), and has remained an important topic in discussions about the humanities and social sciences up to the present day, also in relation to discussions about the role of ‘evidence’ (see also Biesta, 2020). 5 The latter view was the one put forward by logical positivism and, although philosophically discredited, still lives on in the idea that knowledge acquisition is an inductive process starting from the collection of ‘basic facts’ and working ‘upwards’ towards general statements (see Ayer, 1959; Achinstein & Barker, 1969).

192

Gert Biesta

References Achinstein, P., & Barker, S. F. (1969). The Legacy of Logical Positivism: Studies in the Philosophy of Science. Baltimore: Johns Hopkins Press. Adorno, T. W. et al. (1969). Der Positivismusstreit in der deutschen Soziologie. Neuwied and Berlin: Luchterhand. Apel, K.-O. (1982). The Erklären-Verstehen controversy in the philosophy of the natural and human sciences. In G. Fløistad (Ed.), Contemporary Philosophy: A  New Survey (vol. 2, pp.  19–49). Dordrecht: Springer. https://doi.org/10.1007/97894-010-9940-0_2. Ayer, A. J. (1959). Logical Positivism. Glencoe, IL: Free Press. Biesta, G. (2007). Why ‘what works’ won’t work. Evidence-based practice and the democratic deficit of educational research. Educational Theory, 57(1), 1–22. Biesta, G. (2010a). Pragmatism and the philosophical foundations of mixed methods research. In A. Tashakkori & C. Teddlie (Eds.), The SAGE Handbook of Mixed Methods in Social and Behavioral Research (2nd ed., pp. 95–118). Thousand Oaks, CA: Sage. Biesta, G. (2010b). Why ‘what works’ still won’t work. From evidence-based education to value-based education. Studies in Philosophy and Education, 29(5), 491–503. Biesta, G. (2013). Interrupting the politics of learning. Power and Education, 5(1), 4–15. Biesta, G. (2015). No paradigms, no fashions, and no confessions: Why researchers need to be pragmatic. In A. B. Reinertsen & A. M. Otterstad (Eds.), Metodefestival og Øyeblikksrealisme (pp. 133–149). Bergen: Fagbokforlaget. Biesta, G. (2020). Educational Research: An Unorthodox Introduction. London: Bloomsbury. Biesta, G.,  & Burbules, N. (2003). Pragmatism and Educational Research. Lanham, MD: Rowman and Littlefield. Bryman, A. (2006). Paradigm peace and the implications for quality. International Journal of Social Research Methodology, Theory and Practice, 9(2), 111–126. Creswell, J. W. (2007). Qualitative Inquiry and Research Design: Choosing among Five Approaches. London: Sage. Creswell, J. W. (2022). A Concise Introduction to Mixed Methods Research (2nd ed.). London: Sage. Dancy, J. (1985). An Introduction of Contemporary Epistemology. Oxford: Basil Blackwell. Denzin, N.K. (2008). The new paradigm dialogs and qualitative inquiry. International Journal of Qualitative Studies in Education, 21(4), 315–325. Dewey, J. (1896). The reflex arc concept in psychology. In Jo Ann Boydston (Ed.), The Early Works (1882–1898) (vol. 5, pp.  224–243). Carbondale and Edwardsville: Southern Illinois University Press. Dewey, J. (1905). The postulate of immediate empricism. In Jo Ann Boydston (Ed.), The Middle Works (1899–1924) (vol. 3, pp. 158–167). Carbondale and Edwardsville: Southern Illinois University Press. Dewey, J. (1906). The experimental theory of knowledge. In Jo Ann Boydston (Ed.), The Middle Works (1899–1924) (vol. 3, pp. 107–127). Carbondale and Edwardsville: Southern Illinois University Press. Dewey, J. (1907). The control of ideas by facts. In Jo Ann Boydston (Ed.), The Middle Works (1899–1924) (vol. 4, pp. 78–90). Carbondale and Edwardsville: Southern Illinois University Press. Dewey, J. (1911). Epistemology. In Jo Ann Boydston (Ed.), The Middle Works (1899– 1924) (vol. 6, pp. 440–442). Carbondale and Edwardsville: Southern Illinois University Press.

Mixed Methods Research and Deweyan Pragmatism Reconsidered

193

Dewey, J. (1916). Introduction to Essays in experimental logic. In Jo Ann Boydston (Ed.), The Middle Works (1899–1924) (vol. 10, pp.  320–369). Carbondale and Edwardsville: Southern Illinois University Press. Dewey, J. (1920). Reconstruction in philosophy. In Jo Ann Boydston (Ed.), The Middle Works (1899–1924) (vol. 12, pp. 77–201). Carbondale and Edwardsville: Southern Illinois University Press. Dewey, J. (1922). Human nature and conduct. In Jo Ann Boydston (Ed.), The Middle Works (1899–1924) (vol. 14). Carbondale and Edwardsville: Southern Illinois University Press. Dewey, J. (1925). Experience and nature. In Jo Ann Boydston (Ed.), The Later Works (1925–1953) (vol. 1). Carbondale and Edwardsville: Southern Illinois University Press. Dewey, J. (1929). The quest for certainty. In Jo Ann Boydston (Ed.), The Later Works (1925–1953) (vol. 4). Carbondale and Edwardsville: Southern Illinois University Press. Dewey, J. (1933). How we think. A restatement of the relation of reflective thinking to the educative process. In Jo Ann Boydston (Ed.), The Later Works (1925–1953) (vol. 8, pp. 105–352). Carbondale and Edwardsville: Southern Illinois University Press. Eagleton, T. (2007). Ideology: An Introduction. New and Updated Edition. London and New York: Verso. Gage, N. (1989). The paradigm wars and their aftermath: A “historical” sketch of research on teaching since 1989. Educational Researcher, 18(7), 4–10. Gorard, S.,  & Taylor, C. (2004). Combining Methods in Educational and Social Research. Maidenhead: Open University Press. Greene, J. (2008). Is mixed methods social inquiry a distinctive methodology? Journal of Mixed Methods Research, 7(2), 7–22. Greene, J., & Hall, J. (2010). Dialectics and pragmatism: Being of consequence. In A. Tashakkori & C. Teddlie (Eds.), The SAGE Handbook of Mixed Methods in Social and Behavioral Research (2nd ed., pp. 119–144). Thousand Oaks, CA: Sage. Habermas, J. (1968). Erkenntnis und Interesse. Frankfurt am Main: Suhrkamp. Habermas, J. (1970). Zur Logik der Sozialwissenschaften. Frankfurt am Main: Suhrkamp. Hammersley, M. (1992). The paradigm wars: Reports from the front. British Journal of Sociology of Education, 13(1), 131–143. doi:10.1080/0142569920130110. Johnson, R. B.,  & Onwuegbuzie, A. J. (2004). Mixed methods research: A  research paradigm whose time has come. Educational Researcher, 33(7), 14–26. Johnson, R. B., Onwuegbuzie, A. J.,  & Turner, L. A. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1(2), 112–133. Maxcy, S. (2003). Pragmatic threads in mixed methods research in the social sciences: The search for multiple modes of inquiry and the end of the philosophy of formalism. In A. Tashakkori & C. Teddlie (Eds.), Handbook of Mixed Methods in Social and Behavioral Research (pp. 51–89). Thousand Oaks, CA: Sage. Morgan, D. L. (2007). Paradigms lost and pragmatism regained. Methodologies implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research, 1(1), 48–76. Nozick, R. (1981). Philosophical Explanations. Oxford: Oxford University Press. Sleeper, R. W. (1986). The Necessity of Pragmatism. John Dewey’s Conception of Philosophy. New Haven, CT: Yale University Press. Tashakkori, A., & Teddlie, C. (1998). Mixed Methodology: Combining Qualitative and Quantitative Approaches. Thousand Oaks, CA: Sage.

10 MIXED METHODS RESEARCH AND CRITICAL REALISM RECONSIDERED Rosa W. Runhardt

10.1

Introduction

As we have seen in Joseph A. Maxwell’s chapter (this volume), Maxwell argues that realists ought to adopt a mechanist view of causation in the social sciences. Qualitative, mechanism-based research therefore plays a major role in realist social scientific research (Maxwell, 2012a). While realist methodologists like Maxwell accept the possibility of mixed methods research which combines qualitative and quantitative approaches to causation, one of the consequences of their realist ontology is that they deem reference to causal mechanisms to be necessary for causal explanation. Qualitative research is a vital part of realist mixed methods research. In this chapter, I  will show that a realist approach to mixed methods research faces two key problems. First, I will show that realists emphasise the situational contingency of causal mechanistic claims, while social scientists focused on population-level causal claims generally support such claims with causal mechanisms that transcend situational differences. Therefore, there is a tension between realism and mixed methods practice. Second, if one were to go against this practice and assume that causal mechanisms are inescapably situationally contingent, then one cannot also justify why mixed methods ought to be used in social scientific causal inquiry. In summary, the critical realist approach to mixed methods research is not a suitable partner for methodological pluralists and an appeal to Maxwell’s realist mechanisms in mixed methods research is unsustainable. This chapter is set up as follows. In Section 10.2, I briefly summarise the current state of the art in realist theories of causal mechanisms, including their methodology for causal analysis. To highlight a relevant alternative, DOI: 10.4324/9781003273288-12

Mixed Methods Research and Critical Realism Reconsidered

195

I  compare the realist theory to an alternative social scientific methodology which is also focused on mechanisms, viz., process tracing. In Section  10.3, I  address the realist attitude towards mixed methods research, responding both to Maxwell’s chapter in this volume as well as his earlier work on the topic. I uncover a tension between realists’ emphasis on the situational contingency of mechanisms on the one hand, and their arguments for mixed methods on the other. In Section 10.4, I attempt to solve this tension by measuring realist mixed methods research against the most prominent candidate to support mixed methods research from the philosophy of evidence, Evidential Pluralism. I show that the causal mechanisms that realists like Maxwell defend cannot pull the evidential weight that Evidential Pluralism requires of them. Instead, I suggest that the best candidate for mechanistic evidence comes from process tracing methodology, given its lack of commitment to situationally contingent mechanisms. In the conclusion, I  draw out ontological and methodological questions for further research. Before diving into these issues, I will give a short caveat. In his chapter in this volume, and indeed in his earlier work, Joe Maxwell has argued that a realist approach to mixed methods research has many different forms, functions, and purported benefits (cf. Maxwell & Mittapalli, 2010; Maxwell, 2011; Maxwell & Chmiel, 2012). However, in this chapter, I will limit myself to discussing the purported benefits of the realist approach for finding evidence of causal relations only. Moreover, I will accept without further questioning Maxwell’s assumption that meanings, beliefs, values, and intentions can play a causal role (Maxwell, 2012a, p. 40). My main focus will be on possible foundational frameworks for combining qualitative evidence of causal mechanisms and processes with quantitative evidence of regularities, regardless of whether either evidence concerns mental phenomena (e.g. beliefs, values, intentions, meanings and/or culture, to name a few; see Maxwell, this volume), physical phenomena, or both.

10.2

Realist Theories of Causation

To start, this section provides a short analysis of realist theories of causal mechanisms. I highlight the realist assumption that causation is ‘local’ and that causal mechanisms are ‘situationally contingent’ and draw conclusions from this assumption for the modern realists’ methodology for causal analysis in the social sciences. Subsequently, I compare the realist methodology to a prominent, but less metaphysically committed mechanist methodology viz., process tracing. I show that process tracing methodologists do not make the same assumptions of locality and contingency.

196

Rosa W. Runhardt

10.2.1

Causal Mechanisms

Causal mechanisms have been central to ‘critical realist’ theories of causation and causal explanation since Roy Bhaskar’s early work. Bhaskar argued that mechanisms are real entities, which ‘endure and operate independently of our knowledge, our experience and the conditions which allow us access to them’ (Bhaskar, 1998, p. 19). Mechanisms, according to Bhaskar and later realists, underpin the events and processes we observe, but are not directly accessible. In Bhaskar’s view: The world consists of mechanisms not events. Such mechanisms combine to generate the flux of phenomena that constitute the actual states and happenings of the world. They may be said to be real, though it is rarely that they are actually manifest and rarer still that they are empirically identified. (Bhaskar, 1998, p. 34) Since mechanisms are essential to causation, seeking knowledge about those mechanisms responsible for a given phenomenon of interest is essential to critical realist science. The method for this centres around a technique Bhaskar called ‘retroduction’, that is, ‘to posit a mechanism (typically at a different level to the phenomenon being explained) which, if it existed and acted in the postulated manner, could account for the phenomenon singled out for explanation’ (Lawson, 1998, p. 156). Modern realists working in the philosophy and methodology of social science have adopted some of the early central principles from Bhaskar’s work.1 Most importantly for this chapter, modern realists in the social sciences also emphasise the use of mechanisms for causal explanation. This influences which social scientific methods they consider to be worthwhile, including which methods they believe social scientists should use for causal explanation.

10.2.2

Realist Methods

To understand modern realist methodology, it is especially relevant to note their emphasis on causal mechanisms as ‘situationally contingent’ (Maxwell, this volume), that is, inextricably connected to contextual factors in individual cases. The realists argue that social scientists cannot separate the context in which a case or process under investigation takes place from the mechanisms underlying that case or process. As such, we cannot understand causal mechanisms behind a phenomenon in isolation from the social and cultural context of that phenomenon (cf. Maxwell, 2017; Maxwell & Mittapalli, 2010). In short, in this realist view, causation is considered ‘fundamentally local’ (Maxwell, 2012a, p. 40). It is no wonder therefore that realists in the social sciences have defended the use of qualitative research methods over quantitative research methods.

Mixed Methods Research and Critical Realism Reconsidered

197

Moreover, Maxwell has argued that it is possible to establish causal relationships in single case studies (cf. Maxwell, 2012a). To establish singular causal claims, a qualitative researcher must collect data that are rich enough to explain exactly how events are connected, that is, to lay bare the processes connecting a putative cause and effect of interest. Given the realists’ focus on local context-dependent causal mechanisms, Maxwell sees narrative analysis as especially helpful for explanation (Maxwell, 2012a). Narratives lay out the specific causal chains and the interactions with contextual factors that led to a particular event or outcome and thus Maxwell believes them to be explanatory. The realist qualitative research process may also involve careful, long-term observation and/or interviewing (Maxwell, 2004, 2012a). Given their emphasis on local, situationally contingent causal mechanisms, realists criticise the ‘Humean’ or ‘regularity’ theory of causation, which they believe underpins quantitative research, in particular regression analysis (cf. Mohr, 1996). Maxwell defines the regularity theory as that which ‘holds that causality consists simply of regular associations between events or variables, patterns in our data, and denies that we can know anything about supposed “hidden” mechanisms that produce these regularities’ (Maxwell, 2012a, p.  9).2 In contrast, as we have seen, realists like Maxwell assume that the mechanisms underpinning reality are discoverable, but also that these mechanisms are so context-laden that to focus on regular association in causal explanation is arguably a fundamental error.3 According to realists, we cannot straightforwardly generalise the local, process-based evidence collected in individual case studies. To illustrate the realist approach to causality, consider educational research. Maxwell has rejected a wholly regularity-based approach to causation in education, and particularly disparages the assumption that quantitative research and randomised-controlled trials ought to be the gold standard in educational research (Maxwell, 2004, 2012b). Such an approach, he has argued, cannot do justice to the complex, context-dependent causal processes in educational settings. Instead, Maxwell defends a prominent role for realist qualitative methods in education, which he believes do more justice to such case-specific, contextually dependent processes, claiming that ‘educational research desperately needs qualitative approaches and methods if it is to make valid and useful claims about what works’ (Maxwell, 2012b, p. 655). 10.2.3

Modern Approaches to Finding Processes in Social Scientific Research

Let us step back for a moment. The argument that quantitative research typically, and unjustifiably, treats causation as a ‘black box’ (Maxwell, this volume) will be familiar to readers, even if the realist position itself is not. Since

198

Rosa W. Runhardt

the 1980s, leading qualitative methodologists in the social sciences have defended process-based approaches, which also promise to open the ‘black box’ of case study causation, and they similarly argue that social scientists should lay bare the causal mechanisms behind observed correlations (cf. Bennett & Checkel, 2015; Bennett & George, 1997; Brady & Collier, 2010; George & Bennett, 2005; Hall, 2013; Hedström & Ylikoski, 2010; Mahoney, 2001, 2010; Mohr, 1982). Some of the recent qualitative methodological literature is reminiscent of the realists’ emphasis on observing processes.4 The distinction between ‘correlational causal analysis’ and ‘the analysis of causal mechanisms and processes’ has become a hot topic in political science and international relations methodology in particular (Kincaid, 2009, pp. 739–740). To illustrate, consider Andrew Bennett and Jeffrey Checkel’s elaborate best practices guide for ‘process-tracing’, Process Tracing: From Metaphor to Analytic Tool (Bennett & Checkel, 2015). Bennett and Checkel sum up their version of the method as ‘the examination of intermediate steps in a process to make inferences about hypotheses on how that process took place and whether and how it generated the outcome of interest’ (Bennett & Checkel, 2015, p. 6). The authors urge researchers to investigate the potential observable implications of their own hypothesised mechanism (reminiscent of the ‘retroduction’ method mentioned in Section  10.2.1), as well as the potential observable implications of existing alternative hypotheses from the literature. Bennett and Checkel’s work contains many such concrete, process-focused rules of thumb for working social scientists.5 Note that while the critical realists postulate a rich metaphysics of real but rarely manifested or directly observed mechanisms, modern process tracing theorists come from a variety of philosophical traditions, if any, in defining the term ‘causal mechanism’ (cf. Hall, 2013; Rohlfing  & Zuber, 2021). The philosophical foundations of process tracing are not necessarily realist. More specifically, social science researchers who utilise process tracing are not necessarily committed to the situational contingency of mechanisms. Consider a well-known and highly acclaimed process tracing study, viz., political scientists Doug McAdam, Sidney Tarrow, and Charles Tilly’s analysis of the mechanisms behind contentious politics (McAdam et  al., 2001). They claim that four of the same mechanisms (brokerage, category formation, object shift, and certification) produce a variety of events in contentious politics (including social movements, strikes, and revolutions), arguing that ‘similar mechanisms of change combine differently with varying environmental conditions in distinctive trajectories of historic change’ (McAdam et al., 2001, p. 83). This use of the term ‘mechanism’ to indicate a causal term that is very general in scope is not a one-off, as these quotations, all taken from the authors’ introduction to their method, indicate:

Mixed Methods Research and Critical Realism Reconsidered

199

We . . . make a bet on how the social world works: that big structures and sequences never repeat themselves, but result from differing combinations and sequences of mechanisms with very general scope. (McAdam et al., 2001, p. 30) [We search] for causal mechanisms and processes that produce similar effects in a wide variety of contentious politics . . . by matching obviously different sorts of episodes, then showing that identical mechanisms and processes play significant parts in those episodes. (McAdam et al., 2001, p. 35) [We] examine how . . . causal mechanisms combine into longer chains of political processes, for example how identity shift and brokerage combine in episodes of nationalism. From identification of such processes, [we] create not general theories of contention but partial theories corresponding to these robust causal similarities. (McAdam et al., 2001, p. 33) Causal mechanisms, then, seem to play a different role in this prominent example of process tracing than they do in the realists’ suggested methodology. As Daniel Little has argued: Take brokerage as a mechanism of social contention – isn’t this really an umbrella term that encompasses a number of different kinds of negotiation and alliance-formation? .  .  . Brokerage is rather a ‘family-resemblance’ term that captures a number of different instances of collective behaviour and agency. (Little, 2011, pp. 277–278)6 In the remainder of this chapter, I  will argue that mixed methods research requires causal mechanisms that transcend situational differences and that therefore the causal mechanisms in process tracing are suitable for mixed methods research while realist causal mechanisms are not. First, however, let us step back to consider why a critical realist may defend mixed methods research in the first place. 10.3 10.3.1

Realist Approaches to Mixed Methods Research Combining Evidence of Mechanisms and Evidence of Regularities

In this section, I  will focus on the realist arguments for using mixed methods research in causal explanation.7 Besides Maxwell’s introduction to realist

200

Rosa W. Runhardt

mixed methods research in this volume, his earlier joint work with Kavita Mittapalli (Maxwell & Mittapalli, 2010)8 forms a key realist contribution to mixed methods research. Given the realists’ emphasis on process-oriented, case-based research, and their rejection of research focused only on discovering regularities, it is unsurprising that realists construe mixed methods as the complementing of ‘mere’ evidence of correlations by evidence of mechanisms (Maxwell & Mittapalli, 2010, pp.  155–156). Citing Pawson and Tilley, Maxwell and Mittapalli see mechanisms as ‘an account of the makeup, behaviour and interrelationship of those processes which are responsible for the regularity [found in quantitative research]’ (Pawson & Tilley, 1997, pp. 67–68, as cited in Maxwell & Mittapalli, 2010, p. 155). So, Maxwell and his realist colleagues like Mittapalli do not reject a combination of quantitative and qualitative approaches. In some situations, they argue, such an approach may even be the best one; in some of his work, Maxwell defends a case-by-case position about evidence and validity, arguing that what counts as evidence of causation is necessarily context-dependent (cf. Maxwell & Chmiel, 2012). Having said this, however, Maxwell has made clear that qualitative, mechanism-based approaches are essential to causal explanation. For example, he claims that if educational researchers wish their research to be useful, they cannot do without qualitative approaches: ‘[w]e need qualitative methods and approaches in order to understand “what works” and why’ (Maxwell, 2012b, p. 659). Relatedly, Maxwell asserts that in the best examples of mixed methods approaches in educational research, ‘the insights gained by the qualitative researchers were essential to the causal conclusions that emerged from the study’ (Maxwell, 2012b, p. 658). 10.3.2

Tensions in Realist Mixed Methods Research

This brings up a tension in realist mixed methods research. We have seen in the previous section that realists consider causation to be ‘fundamentally local’, and mechanisms to be ‘situationally contingent’. As a result, they believe that we cannot simply generalise process-based evidence. Given these realist assumptions, we may wonder what the added benefit of regularity-based research (and its associated evidence of correlations) could be for authors like Maxwell. What can a combination of regularity-based and process-based research deliver, if indeed mechanisms are inescapably context-dependent and, as a result, difficult to aggregate or generalise?9 The reason that Maxwell does not recognise the tension between fundamentally local mechanisms on the one hand and correlational evidence on the other as a problem for mixed methods research is because he sees qualitative and quantitative research as having fundamentally different underlying logics of inference (Maxwell, 2004, p. 3).10 Because of their different ‘logics’,

Mixed Methods Research and Critical Realism Reconsidered

201

they each have different strengths and limitations (Maxwell, 2004, p. 8). Consequently, Maxwell maintains: [I]t is [not] generally appropriate or useful to attempt to synthesize different philosophical approaches or assumptions into a single, logically consistent paradigm for mixed methods research. Different situations and research problems may require different sets of assumptions and models, as well as different combinations of methods. (Maxwell, 2011, p. 29) Maxwell instead defends a ‘bricolage’ approach to mixed methods.11 Following the work of Andrew Abbott, Maxwell thinks of qualitative and quantitative positions ‘as heuristics, conceptual and practical tools that are used to solve specific problems in theory and research’ (Maxwell, 2011, p. 28, emphasis in original). Mixed methods research, considered in this way, becomes ‘a flexible toolkit of different methods and “lenses” for understanding the phenomena we study’ (Maxwell, 2011, p. 29). The picture that emerges is of an extremely lenient methodology: ‘mixed method research has . . . the fewest limitations on the sorts of methods and evidence that you can use to gain an understanding of the things you study and to address alternative explanations and interpretations of these’ (Maxwell & Chmiel, 2012, p. 7). Research is no longer focused on ‘triangulation and testing’, but rather on allowing ‘different approaches to develop different understandings of the phenomena studied, and to put these in dialog with one another, allowing researchers to deepen and complexify’ (Maxwell  & Chmiel, 2012, p. 8). 10.3.3

What If We Did Focus on Testing?

However, what if we did limit ourselves to a focus on testing causal claims, as is the aim of this chapter and indeed of many working social scientists and social policy makers? In that case, a bricolage approach is less defensible. The realist picture that has emerged so far is that causal mechanisms ought to be central to (or even dominant in) causal explanation in the social sciences. Mechanisms connect in idiosyncratic ways with local, contextual factors, resulting in difficult to generalise processes that need to be judged on a caseby-case basis using rich data. Qualitative methods, such as narrative analysis, are therefore essential to discovering and explaining causal relations in the social world. If we follow the bricolage approach, the only reason we might include quantitative research in mixed methods work is to provide ‘a different lens’ for understanding the phenomena under study, but it is difficult to see what they might add to causal testing for the realist, given their disparagement of generalisation.

202

Rosa W. Runhardt

Compare this to process tracing researchers like McAdam, Tarrow, and Tilly, whose aim instead is to find recurring mechanisms, mechanisms of general scope. In short, those researchers who are interested in testing more general causal claims may be sold short by the realist approach. Realism does not provide a clear justification for using mixed methods in social scientific research aimed at causal analysis.12 10.4

Philosophy of Evidence Meets Realist MMR

In this section, I  attempt to solve the issues mentioned so far in this chapter by measuring realist mixed methods research against the most prominent candidate to support mixed methods research from the philosophy of evidence, Evidential Pluralism. I will show that the causal mechanisms that realists like Maxwell defend cannot pull the evidential weight that evidential pluralists require of them. Instead, I suggest that the best candidate for mechanistic evidence comes from the more permissible process tracing methodology discussed in Section  10.2.3, which does not commit to context-heavy metaphysics. Evidential pluralists argue that a causal claim should normally be corroborated by finding both a statistically significant relation and a suitable mechanism complex connecting the putative cause and effect (Russo & Williamson, 2007; Shan, 2021; Shan & Williamson, 2021). While most initial discussions of Evidential Pluralism were limited to biomedical research, Yafeng Shan and Jon Williamson (2021) have recently argued that Evidential Pluralism supports the use of mixed methods in causal analysis in the social sciences as well, that is, that pluralism urges social scientists to consider the results of both quantitative association studies and qualitative mechanistic studies (Shan & Williamson, 2021, p. 16). However, unlike in the realist methodology of the previous sections, in the evidential pluralist framework, mechanisms are construed broadly, as any relation by which a putative cause produces an outcome (cf. Runhardt, 2022). Which particular conceptualisation of mechanisms a researcher has in mind does not affect the argument for pluralism: Evidential Pluralism makes no direct claims about the nature of mechanisms, although it does appeal to the concept of mechanism. From the point of view of the epistemology of causality, it is important to distinguish evidence of mechanisms from evidence of correlation regardless of whether mechanisms themselves are ultimately reducible to correlations, laws, dispositions, low-level causal relations, or none of these. (Shan & Williamson, 2021, p. 5)

Mixed Methods Research and Critical Realism Reconsidered

203

In short, unlike the realists discussed earlier, the evidential pluralists make no metaphysical commitment: they argue that pluralism is ‘a purely epistemological thesis that makes no specific metaphysical claims’ (Shan & Williamson, 2021, p. 10). What matters to Shan and Williamson instead is that the mechanistic aspects of mixed methods research clearly corroborate that the putative cause and effect under study are indeed causally linked. A mechanistic study can do so in two ways: either it uncovers the ‘component links or parts of the mechanism complex’ (Shan & Williamson, 2021, p. 4) which connect cause and effect, and shows that these are not nullified by other causal relations which interact with them, or it rules out alternative explanations of a correlation between the putative cause and effect. Consider the application of Shan and Williamson’s mixed methods research to causal explanations at a higher level than that of individual case studies, for example, causal explanations of a variety of episodes of contentious politics rather than one such episode. In that context, Shan and Williamson argue, ‘it is not enough to show the purported mechanism merely exists in some individuals – it needs to be present in enough individuals to be able to account for the extent of the observed correlation’ (Shan & Williamson, 2021, p. 7). In other words, in order to make a general causal claim, the mechanisms under study must occur in more than one of the cases in some population of interest (e.g. the statistical population). Shan and Williamson admit that this is difficult, as social settings can ‘vary . . . widely in their social mechanisms’ (Shan & Williamson, 2021, p. 9). The thesis that social settings vary widely has some parallels to the realist argument that causation is local, but we should not equate the two. After all, Maxwell has argued that causal mechanisms are inextricably linked to context, to such extent that generalisation and extrapolation are problematic. Shan and Williamson, on the other hand, require that mechanisms be ‘extractable’ if they are to function as support for a general causal claim. For Shan and Williamson, in the study of general causal relations, mechanisms must be able to recur in multiple cases under analysis, explaining the correlations observed at the population level (a higher level than individual case studies). It seems clear, then, that the realist approach to causal mechanisms is not suitable as a type of ‘mechanistic study’ as described by Shan and Williamson. The realist ontology of causation is arguably fundamentally incompatible with the ‘extraction’ of general causal claims. A different picture emerges if we consider the more permissible process tracing methodology from Section  10.2.3. McAdam, Tarrow, and Tilly aimed at mechanisms with general scope, that is, at robust causal similarities between episodes of contention, while granting that processes of contention never repeat across cases in the exact same way. As we have seen, their

204

Rosa W. Runhardt

process tracing makes no ontological commitment; as such, the approach is intuitively more compatible with Evidential Pluralism as it lacks realism’s complex contextual dependencies. And indeed, process tracing has been somewhat successfully combined with quantitative approaches already. In recent developments in the process tracing literature, methodologists attempt to combine the method with statistical research in specific ways, for example, using a whole population’s worth of case studies to find the causal mechanisms behind statistical patterns (for a comprehensive overview of this approach, see Goertz, 2017). In summary, Evidential Pluralism and process tracing are a stronger match for justifying mixed methods research than Evidential Pluralism and realism. To illustrate how one may accept contextual distinctions without committing to a full-blown localism, consider one of the examples in Shan and Williamson’s article, that is, Dale Copeland’s study of the relationship between economic interdependence between great powers and the risk of conflict between those powers. Copeland’s study contains the general causal claim that ‘economic interdependence between states influences the expectation that a state has of future trade with other states, which eventually influences the chances of military conflict’ (Shan & Williamson, 2021, p. 12, see also Copeland (2015)), which according to Copeland’s work holds for all great power conflicts since 1790. While Copeland believes this general causal claim to be supported by the evidence, and indeed a fruitful level of analysis, the mechanistic evidence he uses to support it comes from case studies of individual great power conflicts. Copeland finds his evidence through process tracing of the observed implications of an underlying hypothesised causal mechanism, the ‘trade expectations mechanism’. Crucially, Copeland does not argue that the trade expectations mechanism repeats in a structurally identical way across these cases, but accepts individual differences; nevertheless, unlike the realists, he believes that the generalisation that the trade expectations mechanism influenced all great power conflicts since 1790 is accurate and fruitful. In his work, we ought to treat the trade expectation mechanism as a claim of family resemblance between great power conflicts, just like we treated brokerage as a family resemblance claim in the McAdam, Tarrow, and Tilly example. 10.5

Conclusion

In this chapter, I have uncovered a tension between realist methodologists’ emphasis on the situational contingency of mechanisms on the one hand, and their arguments for mixed methods on the other. In particular, I argued that mixed methods research requires causal mechanisms that transcend situational differences; the individual mechanisms in each case study in a population must be somehow integrated with one another if one is to support a

Mixed Methods Research and Critical Realism Reconsidered

205

general causal claim for that population. To put this argument differently: if one assumes that causal mechanisms are always situationally contingent, then one cannot also provide a clear justification for using mixed methods research in social scientific causal inquiry. I have argued that an evidential pluralist combination between process tracing and association studies is a stronger candidate for supporting general causal claims, given its lack of commitment to a metaphysics that focuses on situational contingency. Before ending this chapter, we may wonder at the ontological status of the evidential pluralists’ and process tracers’ conclusions. Going all the way back to Bhaskar’s initial discussion of causal mechanisms, we find that he believed causal mechanisms to be the best possible explanation for why scientific research seems to work: a ‘full and consistent realism . . . is the only position that can do justice to science’ (Bhaskar, 1998, p. 19). Bhaskar believed that we must distinguish between constant conjunctions on the one hand, which are created in experimental settings, and causal mechanisms on the other hand, which are a real part of the world independent of our perception. We have seen this mirrored in the modern realists’ emphasis on mechanisms over regularities. A realist critic might at this point interject that ‘brokerage’ as defined by McAdam, Tarrow, and Tilly is just not a ‘real’ mechanism, and that it is misguided to use it as a counterexample here (and, along similar lines, that Copeland’s ‘trade expectations mechanism’ is not a ‘real’ mechanism either). To this, I have three responses. First of all, given the high acclaim of McAdam, Tarrow, and Tilly’s and Copeland’s work, as well as their usage of ‘causal mechanism’, arguably a practice-engaged, descriptive philosophy of social science theory that does not take the work seriously is problematic. Second, I would argue that it is by no means clear that we can only explain the success of social science if social scientific causal mechanisms are real. To repeat, the causal mechanism brokerage is explanatory successful for McAdam, Tarrow, and Tilly; it can help explain parts of what happened in many of the episodes of contention they have analysed. Yet ‘brokerage’ and ‘trade expectations’ are family resemblance terms, an indication of causal similarity rather than causal sameness. It is therefore hard to see McAdam, Tarrow, and Tilly’s or Copeland’s work as an ontological thesis about how the social world is organised. A ‘realist’ ontological status of a mechanism like brokerage is difficult to defend. Finally, and relatedly, there exist theories in the literature that may explain why, for example, brokerage is explanatory that are not realist (cf. Williamson, 2005, 2006). Based on those theories, we may treat social causal mechanisms like ‘brokerage’ and ‘trade expectations’ as theoretical constructs by social scientists which license them to group together political episodes of interest, and thereby explain larger patterns. Such theories could form the jumping-off point for future philosophical research into the use of social scientific, general causal mechanisms.

206

Rosa W. Runhardt

By hinting at the compatibility between process tracing and general causal claims, as I have done in this chapter, we are not yet done with building a fruitful philosophical framework for mixed methods research. An important question for further research is more methodological and concerns how one may best combine statistical research with process tracing.13 As I have argued in other work (Runhardt, 2022), process tracing evidence of mechanisms is often found at the level of individual case studies, while difference-making claims are more general. It is by no means yet settled what a correct ‘concatenation’ of multiple case studies into the development and testing of general causal claims would look like. Realism, however, does not seem a likely source of advice. Notes 1 As we can see in Maxwell (this volume), they diverge on other matters. I will here adopt Maxwell’s terminology and, from this point in the article, use the slightly broader term ‘realism’ (rather than ‘critical realism’) to refer to his theory, which ‘assumes that our concepts and theories are our own constructions, but that these refer to real phenomena, ones that can only be incompletely understood by means of those constructions’ (Maxwell, this volume, 3). 2 It is an oversimplification to equate modern quantitative approaches in the social sciences with a focus on regular associations. The current trend towards structural causal modelling is arguably compatible with a much richer view of causation that includes mechanisms (cf. Rohlfing & Zuber, 2021). To maintain the chapter’s focus, however, I will not discuss these additional complications to the realist position here. 3 Another version of these claims, in this volume, is Maxwell’s argument that theories based on regular association ignore the essential ‘diversity’ in the social world (Maxwell, this volume, p. 159). 4 Maxwell himself has called process tracing a good example of a method for realist, process-based causal explanation: in Maxwell (2017), for instance, he contends that the realist approach to causation has been developed particularly well in process-based political science. In what follows in this section, as well as in Sections 10.4 and 10.5, I will show that this connection is not as tight as Maxwell has claimed. 5 Some of the other advice in Bennett and Checkel is also reminiscent of what Maxwell considers to be good practice in qualitative research (cf. Maxwell, 2012a, 2017). For instance, Maxwell also advises to look for evidence ‘as to whether particular processes were operating and if they had the causal influence hypothesized’ (Maxwell, 2012a, p. 47). 6 Daniel Little does not draw the same conclusions from McAdam, Tarrow, and Tilly work as I do in this chapter. Instead, he defends a version of methodological localism that goes against the emphasis I will put on generality in Sections 10.3 and 10.4. I  will not discuss that position any further here; his interpretation of McAdam, Tarrow, and Tilly remains apt even against the backdrop of his different conclusions. 7 Like Maxwell (this volume), I  will define mixed methods research broadly, as a combination of qualitative and quantitative methods in general. Where more specificity is needed, I will follow Creswell and Plano Clark’s focus on mixed methods research as gathering both qualitative and quantitative data in order to answer

Mixed Methods Research and Critical Realism Reconsidered

8 9

10

11 12 13

207

research questions and test hypotheses (Creswell & Plano Clark, 2018). Note that, as stated in the Introduction, I will limit myself to discussing the benefits of mixed methods research for causal analysis, rather than any of the other benefits Maxwell has discussed. Another argument for a realist interpretation of mixed methods research, which fits well with Maxwell’s position, can be found in Mingers and Standing (2017). Given the similarities to Maxwell and Mittapalli, I will not discuss this article further here. We may also ask what the ontology of mechanisms is, if indeed they are as inextricable from context as Maxwell describes. For early critical realists like Bhaskar, as we have seen, mechanisms were unobservable entities which combined in diverse ways to produce a variety of potentially observable processes. I will briefly return to this question in the conclusion to the chapter. Here Maxwell pushes back against King, Keohane, and Verba (1994), who claimed that ‘the differences between qualitative and quantitative traditions are “only stylistic and . . .  methodologically and substantively unimportant” ’ (Maxwell, 2004, p. 7, citing King et al., 1994, p. 4). Maxwell takes this term from the work of Claude Lévi-Strauss. For an analysis of Lévi-Strauss’s term as applied to qualitative research, and criticism thereof, see Hammersley (2008). For a similar but much more brief argument against realism, see Shan (2021). Relatedly, we may ask whether process tracing as discussed by Bennett and Checkel (2015) is extendable to other social sciences, like Maxwell’s area of educational research. I see no reason for thinking Bennett and Checkel’s guidelines are inapplicable elsewhere; we can find process tracing in a wide variety of social scientific literature.

References Bennett, Andrew,  & George, Alexander L. (1997). Process Tracing in Case Study Research. MacArthur Foundation Workshop on Case Study Methods. Bennett, Andrew,  & Checkel, Jeffrey T. (2015). Process Tracing: From Metaphor to Analytic Tool. Cambridge, UK: Cambridge University Press. Bhaskar, Roy. (1998). Philosophy and scientific realism. In Roy Bhaskar, Margaret Archer, Andrew Collier, Tony Lawson, & Alan Norrie (Eds.), Critical Realism: Essential Readings (pp. 16–47). London: Routledge. Brady, Henry E.,  & Collier, David. (2010). Rethinking Social Inquiry: Diverse Tools, Shared Standards (2nd ed.). Lanham: Rowman and Littlefield Publishers. Copeland, Dale C. (2015). Economic Interdependence and War. Princeton: Princeton University Press. Creswell, J. W., & Plano Clark, V. L. (2018). Designing and Conducting Mixed Methods Research. Thousand Oaks: Sage. George, Alexander L., & Bennett, Andrew. (2005). Case Studies and Theory Development in the Social Sciences. Cambridge: MIT Press. Goertz, Gary. (2017). Multimethod Research, Causal Mechanisms, and Case Studies: An Integrated Approach. Princeton and Oxford: Princeton University Press. Hall, Peter A. (2013). Symposium: Tracing the progress of process tracing. European Political Science, 12, 20–30. Hammersley, Martyn. (2008). The dadaist alternative: “Postmodernist” qualitative research. In Questioning Qualitative Inquiry: Critical Essays (pp.  128–144). London: Sage.

208

Rosa W. Runhardt

Hedström, Peter, & Ylikoski, Petri. (2010). Causal mechanisms in the social sciences. Annual Review of Sociology, 36, 49–67. Kincaid, Harold. (2009). Causation in the social sciences. In Helen Beebee, Christopher Hitchcock,  & Peter Menzies (Eds.), The Oxford Handbook of Causation (pp. 726–743). Oxford: Oxford University Press. King, Gary, Keohane, Robert O., & Verba, Sidney. (1994). Designing Social Inquiry. Princeton: Princeton University Press. Lawson, Tony. (1998). Economic science without experimentation. In Roy Bhaskar, Margaret Archer, Andrew Collier, Tony Lawson, & Alan Norrie (Eds.), Critical Realism: Essential Readings (pp. 144–186). London: Routledge. Little, Daniel. (2011). Causal mechanisms in the social realm. In Phyllis McKay Illari, Federica Russo, & Jon Williamson (Eds.), Causality in the Sciences (pp. 273–295). Oxford: Oxford University Press. Mahoney, James. (2001). Beyond correlational analysis: Recent innovations in theory and method. Sociological Forum, 16(3), 575–593. Mahoney, James. (2010). After KKV: The new methodology of qualitative research. World Politics, 62(1), 120–147. Maxwell, Joseph A. (2004). Causal explanation, qualitative research, and scientific inquiry in education. Educational Researcher, 33(2), 3–11. Maxwell, Joseph A. (2011). Paradigms or toolkits? Philosophical and methodological positions as heuristics for mixed methods research. Mid-Western Educational Researcher, 24(2), 27–30. Maxwell, Joseph A. (2012a). A Realist Approach to Qualitative Research. Thousand Oaks: Sage. Maxwell, Joseph A. (2012b). The importance of qualitative research for causal explanation in education. Qualitative Inquiry, 18(8), 655–661. Maxwell, Joseph A. (2017). Realism in Qualitative Inquiry. Presented at the Qualitative Methods Master Class Webinar, International Institute for Qualitative Methodology (IIQM), 16 February. www.youtube.com/watch?v=Nk7_D1j59nA (accessed 29 July 2022). Maxwell, Joseph A.,  & Chmiel, Margaret. (2012). What Makes Mixed Methods Research “Scientific”? Presented at the Annual Meeting of the American Educational Research Association, Vancouver, BC, 13 April. www.researchgate.net/publi cation/270219691_Mixed_methods_education_research_Is_it_science (accessed 29 July 2022). Maxwell, Joseph A., & Mittapalli, Kavita. (2010). Realism as a stance for mixed methods research. In Abbas Tashakkori  & Charles Teddlie (Eds.), SAGE Handbook of Mixed Methods in Social & Behavioral Research (pp. 145–168). Thousand Oaks: Sage. McAdam, Doug, Tarrow, Sidney,  & Tilly, Charles. (2001). Dynamics of Contention. Cambridge: Cambridge University Press. Mingers, John, & Standing, Craig. (2017). Why things happen – developing the critical realist view of causal mechanisms. Information and Organization, 27, 171–189. Mohr, Lawrence B. (1982). Explaining Organizational Behavior. San Francisco: Jossey-Bass. Mohr, Lawrence B. (1996). The Causes of Human Behavior: Implications for Theory and Method in the Social Sciences. Ann Arbor: University of Michigan Press. Pawson, Ray, & Tilley, Nicholas. (1997). Realistic Evaluation. London: Sage.

Mixed Methods Research and Critical Realism Reconsidered

209

Rohlfing, Ingo, & Isabel Zuber, Christina. (2021). Check your truth conditions! Clarifying the relationship between theories of causation and social science methods for causal inference. Sociological Methods and Research, 50(4), 1623–1659. Runhardt, Rosa W. (2022). Limits to evidential pluralism: Multi-method large-n qualitative analysis and the primacy of mechanistic studies. Synthese, 200(171). Russo, Federica,  & Williamson, Jon. (2007). Interpreting causality in the health sciences. International Studies in the Philosophy of Science, 21(2), 157–170. Shan, Yafeng. (2021). Philosophical foundations of mixed methods research. Philosophy Compass, 17(1). Shan, Yafeng, & Williamson, Jon. (2021). Applying Evidential Pluralism to the social sciences. European Journal for Philosophy of Science, 11(96). Williamson, Jon. (2005). Bayesian Nets and Causality: Philosophical and Computational Foundations. Oxford: Oxford University Press. Williamson, Jon. (2006). Dispositional versus epistemic causality. Minds and Machines, 16, 259–276.

11 MIXED METHODS AND CAUSAL ONTOLOGY Christopher Clarke

Ontological issues have played less of prominent role in methodological discussions of mixed methods research than epistemological issues (Tashakkori  & Teddlie, 2010, p.  4).1 But ontology matters for epistemology, I  will argue. In particular, different ontological assumptions about the nature of causation entail different conclusions about what mixed methods research needs to do in order to deliver successful causal inferences. My examples in this chapter are mostly drawn from political science, where mixed methods research is often called multi-method research. Nevertheless, I expect that my argument will probably extend to many of the other social sciences. Here’s the plan. Section 11.1 distinguishes between quantitative sources of evidence and modes of inference (on the one hand) and qualitative sources of evidence and modes of inference (on the other hand). Section 11.2 builds a taxonomy of five ways in which quantitative and qualitative evidence and inference modes might be genuinely integrated – the simplest of which is cross-checking triangulation. While other taxonomies are built for practitioners who design mixed methods research, my taxonomy is tailor-made for the methodologist who is interested in understanding the rationale behind mixed methods research. Section  11.3 then analyses an exemplar of triangulation in political science, namely Hummel et al.’s (2021) study on whether political finance subsidies reduce corruption. I offer one way of reconstructing the logic of this study so that it counts as a genuine instance of cross-checking triangulation. My interpretation requires three controversial assumptions: (1) ‘translation assumptions’ in which qualitative descriptions are translated into the values of quantitative variables; (2) the assumption of causal homogeneity; and (3)

DOI: 10.4324/9781003273288-13

Mixed Methods and Causal Ontology

211

the ontological assumption that causal claims are claims about counterfactual dependence. Section  11.4 then examines some reasons to doubt the latter ontological assumption. In light of these reasons, one ought to be open to the idea that causal claims in qualitative research are not simple claims of counterfactual dependence. One alternative ontology, offered by Beach and Pedersen (2016), is that these causal claims are instead claims about causal production – where causal production has absolutely nothing to do with counterfactual conditionals. Contrary to Beach and Pedersen, I  argue that their notion of causal production is ill-defined, and I argue that it is unclear what the logic of triangulation would be, if it were true that causal claims in qualitative political science are about causal production. In summary, there are problems with interpreting causation in Hummel et al. (2021) in terms of counterfactual dependence and in terms of causal production. This motivates the search for a third ontology of causation – one in which causal claims are neither claims of counterfactual dependence nor claims of causal production. Section  11.5 offers such an ontology: causal claims in qualitative political science are claims of counterfactual sufficiency. This ontology of causation is a close relative of Ragin’s (1987) ontology in terms of set theory and of Mackie’s ([1974] 1980) ontology in terms INUS conditions. Section  11.6 then shows how one might reconstruct the logic of Hummel et al.’s mixed method study in line with this alternative ontology of causation as counterfactual sufficiency. But this interpretation in terms of causal sufficiency is not without problems of its own, I show. I conclude that, for each of the three ontologies of causation that I’ve considered in this chapter – causation as counterfactual dependence or as causal production or as counterfactual sufficiency – there are serious problems with reconstructing the logic of Hummel et al.’s study as a successful instance of cross-checking triangulation. Each causal ontology gives rise to different problems. Since Hummel’s study is an exemplar of excellent mixed methods research, I contend that it is much more difficult to perform successful mixed methods research than is currently acknowledged. Once one pays attention to causal ontology, these difficulties become apparent. (What’s more, each of the three ontologies of causation entails different conclusions about what Hummel’s study needs to do in order to successfully establish its causal conclusion.) 11.1

Sources of Evidence and Modes of Inference

To explain what it means to employ a mixed-methods research design, one first has to appreciate the diversity of sources of evidence in political science, as well as the diversity of modes of inference in political science.

212 Christopher Clarke

Political researchers use many sources of evidence in support of their conclusions and in support of their narratives. These sources of evidence include undertaking ethnographic fieldwork (Brodkin, 2017), facilitating focus group discussions (Cyr, 2019), interviewing key actors, studying archives, commissioning surveys, running field experiments (Morton  & Williams, 2010; Druckman et  al., 2011), and manipulating theoretical models such as a rational-choice models or game-theoretic models (Bates, 1998). Some of these sources of evidence are mainly used by political researchers working in the qualitative tradition – namely ethnography, focus groups, interviews, and archival work. Some of these sources are mainly used by political researchers working in the quantitative tradition – namely surveys, field experiments, and theoretical modelling. So the qualitative and quantitative traditions differ in how they collect their data. They differ in their sources of evidence. The qualitative and quantitative traditions also seem to employ different modes of inference. That is to say, they seem to differ in how they draw conclusions from the evidence available to them. Researchers in the quantitative tradition typically use statistical modes of inference (King et al., 1994). Statistical modes of inference are characterized by two features. The first feature is that one draws a conclusion about a broad population of cases on the basis of what one observes about a large sample of cases selected from that broader population. By looking at a sample of 50 democracies, for example, a quantitative researcher might ask: to what extent do lax campaign finance laws, in democracies, make a causal contribution to corruption? So quantitative researchers are ultimately interested in hypotheses about a population of cases – such as the population of all democracies for example. The second feature of statistical modes of inference is that one tries to distinguish genuine patterns in the data from patterns that arise merely by statistical fluke (random co-incidence). In contrast, researchers in the qualitative tradition use non-statistical modes of inference (Brady & Collier, 2010). First, some qualitative researchers compare two cases with each other in order to draw causal conclusions (Marx et al., 2013). For example, one might ask: what factor was present in the first case (Venezuela for instance), but absent in the second case (Uruguay for instance), such that this factor caused corruption to be high in the first case (Venezuela) and low in the second case (Uruguay)? Here the qualitative researcher is still interested in a population of cases (Latin American states for example), and she is still interested in causal hypotheses. But she uses a non-statistical mode of inference, one that uses data from two cases only. She does not use a statistical mode of inference, which would require using data from a much larger number of cases. Second, some qualitative researchers focus on a single case only (Paraguay for instance). For example, one might show that some event A in Paraguay was a cause of a second event B, which in turn was a cause of a third event

Mixed Methods and Causal Ontology

213

C, and so on. Thus, one produces a causal narrative in which A caused B, B caused C, and so on. (By a causal narrative, I mean a series of causal claims that are presented in the way that people normally present stories. For example, a narrative will usually be more or less in chronological order, and be fairly easy to understand.) Qualitative political scientists often say that they produce this causal narrative by using a distinct mode of inference known as process-tracing (Clarke, 2023a). And process-tracing supposedly produces this causal narrative about Paraguay by studying evidence from Paraguay alone without any explicitly comparing Paraguay to any other cases (Bennett & Elman, 2006; Mahoney, 2010; Beach & Pedersen, 2016; Beach, 2017). Third, other qualitative researchers are less interested in producing causal narratives. Instead they aim to describe, as richly and comprehensively as possible, some of the meanings that events A, B, and C had for those who experienced those events (Bevir & Blakely, 2018). I will call such ‘interpretivist’ modes of inference hermeneutic modes of inference. In short, the qualitative and quantitative traditions seem to employ different modes of inference – with the quantitative tradition employing statistical modes of inference, while the qualitative tradition employs comparative process-tracing or hermeneutical modes of inference. Although the primary distinction between the quantitative tradition and the qualitative tradition is that the former uses statistical modes of inference and the latter does not, this is a simplification. Manipulating theoretical models such as a rational-choice model does not involve statistical modes of inference (Bates, 1998). Indeed, the same is true of many researchers who use machinelearning tools to find patterns in the data – for example, in vast quantities of interview data, or newspaper materials (Chatsiou & Mikhaylov, 2020; Grimmer et al., 2021). Often these machine-learning tools do not involve statistical inference, in that they do not involve inferences from sample to population, and in that they pay much less attention to distinguishing genuine patterns from statistical flukes. Instead, these tools are more exploratory; they search for prima-facie patterns in overwhelmingly large datasets. Nevertheless, most researchers see such methods as belonging to the quantitative tradition, rather than to the qualitative tradition. A second way in which what I said earlier was simplistic is that there are some sources of evidence drawn on by both qualitative and quantitative researchers. Newspaper reports, for example, of the number of people killed each month in a particular armed conflict could be used either as evidence in a process-tracing inference, or it could be used as part of a data-set that is analysed using statistical modes of inference. In summary, there are a diverse range of sources of evidence in political science (ethnography, focus groups, interviews, archival work, surveys, field experiments, theoretical modelling), and there are diverse range of modes of inference (statistical, comparative, process-tracing, and hermeneutics).

214

11.2

Christopher Clarke

Five Logics of Mixed Methods Research

As a first approximation, mixed-method research designs are research designs in which one draws on multiple sources of evidence and/or multiple modes of inference. Although this definition of mixed methods seems plausible at first glance, a little probing shows that this definition is naive. That’s because real mixed-methods research needs to genuinely integrate quantitative and qualitative sources of evidence and/or modes of inference (Teddlie & Tashakkori, 2006; Moran-Ellis et al., 2006; Wolf, 2010; Humphreys & Jacobs, 2015). This contrasts with quasi mixed-method research, which involves different sources of evidence/modes of inference but does not integrate them. This section will catalogue five different ways in which genuine integration might occur. Before doing so, I will use the following example of quasi mixed-methods research to make the issue as vivid as possible. Imagine a biochemist who uses X-ray crystallography to study the structure of a protein and imagine a political scientist who is using a hermeneutic mode of inference to study Russian identity in Moldova. Instead of each researcher publishing a separate paper – one in a biochemistry journal, the other in a political science journal – they make the bizarre decision to stick their two papers together (without any further editing) and submit this joint paper to the Journal of Mixed Methods Research. This joint paper includes both biochemical evidence sources and inference modes alongside hermeneutical evidence sources and inference modes. So the joint paper does indeed draw on multiple sources of evidence, and does engage in multiple modes of inference. And so the joint paper counts as mixed-methods research according to the naive definition of mixed methods that I presented earlier. However, it is obvious that this joint paper should not count as an example of genuine mixed methods research. This is because this joint paper fails to integrate the biochemical evidence sources and inference modes (on the one hand) with the political evidence sources and inference modes (on the other hand). Mere juxtaposition is not genuine integration. This shows that the naive definition of mixed methods research needs to be improved. In particular, any improved definition of mixed methods needs to insist that the two sources of evidence and two modes of inference be genuinely integrated with each other. But what does it mean to genuinely integrate these things? There are a large number of different taxonomies for what such genuine integration might look like (Greene, 2007; Morse, 2010; Tashakkori et al., 2010; Tashakkori & Teddlie, 2010, chap. 1). These taxonomies are designed to be useful to practicing social scientists when designing their research. I want to offer a different taxonomy. This taxonomy is instead meant for methodologists, not for practitioners. This taxonomy illuminates the different epistemological rationales behind mixed methods research. My taxonomy says that there are at least five ways in which genuine integration can occur: (1) quantitative research guides

Mixed Methods and Causal Ontology

215

qualitative questions; (2) qualitative research guides quantitative questions; (3) qualitative research justifies quantitative assumptions; (4) quantitative research justifies qualitative assumptions; and (5) cross-checking triangulation. I will now discuss each in turn. Quantitative Research Guides Qualitative Questions. The results of a quantitative study guide a qualitative researcher on what particular case or cases would be most interesting to study and/or what sort of qualitative research questions would be most interesting to ask about those cases. For example, a quantitative study could rule out some causal hypotheses as unlikely, and thus narrow down the field of plausible causal hypotheses. A qualitative study could then test hypotheses from this narrower field. Or a quantitative study could identify which cases are likely to offer the clearest exemplars of a general trend, and then a qualitative study could examine those cases in detail.2 Qualitative Research Guides Quantitative Questions. The results of one or more qualitative studies guide a quantitative researcher on what population of cases would be most interesting to study and/or what causal relationships would be most interesting to measure using quantitative methods. For example, it is not uncommon in the literature review section of a quantitative paper to cite qualitative research in order to motivate one’s research question.3 Qualitative Research Justifies Quantitative Assumptions. Quantitative studies draw conclusions about the causal contribution that one or more treatment variables X1, X 2 , X3 ... make to an outcome variable Y . To justify the conclusions they draw, all quantitative studies appeal to some assumptions. These assumptions are known as the study’s econometric model. For example, a quantitative study might assume that the causal contribution of the treatment variables to the outcome variable is described by the regression equation Y = b1X1 + b 2 X 2 + ‹. In this equation, b1 and b 2 are unknown coefficients that describe the causal contributions of X1 and X 2 respectively to the outcome variable Y . Thus, b1 and b 2 are what the study is trying to measure. In this equation, ‹ is the ‘error term’, which is introduced to take account of the fact that the relationship between the treatment variables and the outcome variable may not be a deterministic one. In other words, ‹ represents the component of outcome Y that arises through random chance, plus the component of outcome Y that arises through causes other than the treatment variables – causes that are not explicitly part of the regression equation. Quantitative studies always assume that this error term does not correlate with any of the treatment variables X. This key assumption is known as the exogeneity assumption, or as the zero conditional mean assumption (Wooldridge, 2016), or as the conditional independence assumption (King et al., 1994, chap. 3.3.2). Thus the econometric model is constituted by the regression equation plus various assumptions about the error term (such as the exogeneity assumption).

216

Christopher Clarke

The role of the econometric model is to justify the causal conclusions that the study draws. How does the econometric model do this? Assuming that a particular econometric model is true, one can often prove mathematically that a particular strategy for measuring the b values is a reliable one. Thus, one can prove that a given study’s strategy (for measuring the causal contributions b ) is a reliable one (Wooldridge, 2016). But this mathematical proof raises the question: what justifies the econometric model itself? Critics of quantitative methods point out that social scientists’ choice of econometric model often does not appear to have any justification (Freedman, 1991; McKim & Turner, 1996; Berk, 2004; Achen, 2005; Romer, forthcoming). One solution to this problem is to use a qualitative study to lend credence to one’s econometric model. For example, one might use a qualitative study to establish that the treatment variables X do not share any ‘common causes’ with the outcome variable Y . It follows from this that the exogeneity assumption is true, one can show.4 Another way of putting this is that the qualitative study allows one to identify any omitted variables that would lead to ‘omitted variable bias’. This illustrates how one might hope to use qualitative studies to secure knowledge of one’s econometric model, and thereby to support the inference that the quantitative study makes from the data to its conclusions about b . That is to say, its conclusions are about the causal contribution that the treatment variables X make to outcome variable Y . Examples of this are becoming increasingly frequent.5 Quantitative Justifies Qualitative Assumptions. As I’ve already noted, process-tracing is a method that belongs to the qualitative tradition in political science. In process-tracing, the study notes that some factor A caused another factor B, and that some factor B caused another factor C, and that factor C caused factor D. Often the process tracer does this in order to draw the conclusion that factor A was a cause of factor D (Waldner, 2012; Mahoney, 2015, p.  213; Rohlfing, 2012, p.  152). To put it metaphorically, one notes each link in a causal chain, and from these links one infers that the beginning of the chain is linked to the end of the chain. But this raises the question of how each link in the causal chain is itself to be established (Runhardt, 2015; Clarke, 2023a). In the past, some methodologists suggested that each link in the causal chain could be directly observed to hold (Brady, 2010). But most methodologists now reject this claim (Beck, 2006; see also my discussion in Section 11.4). In light of this, one idea is that some of these causal links are to be established using quantitative modes of inference (Crasnow, 2012); although this is a controversial suggestion (Bennett & Checkel, 2015). If this is indeed how process tracers establish their causal links, this use of quantitative modes of inference must be of an informal and implicit kind, because most process tracers do not perform explicit statistical analyses of data. One notable exception is Wood’s (2003) use of formal quantitative models for the purposes of qualitative process-tracing (see Waldner, 2015 for discussion). Thus, Wood uses the conclusions established by quantitative studies in order

Mixed Methods and Causal Ontology

217

to justify some of the assumptions made in this qualitative mode of inference (process-tracing). Cross-Checking Triangulation. A  quantitative study uses quantitative sources of evidence and a quantitative mode of inference to address a particular research question. But sometimes the very same research question can also be addressed by a qualitative study that uses qualitative sources of evidence and a qualitative mode of inference. If both studies suggest the same answer to the research question, this provides more evidential support for this answer, compared to if only one study had been performed on its own. If each study suggests a different answer, this provides less evidential support. I call this cross-checking triangulation: looking to see whether multiple sources of evidence/modes of inference give the same answer to the same question. I use the phrase ‘cross-checking triangulation’ to acknowledge that the term ‘triangulation’ itself is often used quite loosely to refer to any kind of integration of the quantitative and qualitative (Hammersley, 2008; Bryman, 2012). Examples of cross-checking triangulation are abundant in political science.6 This concludes my catalogue of five ways in which quantitative and qualitative sources of evidence and modes of inference might be genuinely integrated. There may well be more ways, I acknowledge. And indeed these ways themselves can be combined with each other. To take one example, Lieberman (2005) proposes a method that both involves ‘Quantitative Research Guides Qualitative Questions’ and also ‘Cross-Checking Triangulation’. To take another example, a qualitative study might both (a) suggest some interesting questions for a quantitative researcher to ask and (b) provide information about which econometric assumptions hold true. In particular, a qualitative study might suggest the inclusion of the variable X 2 in one’s econometric model Y = b1X1 + b X 2 + ‹ for two reasons. First, the qualitative study might suggest that it’s interesting to study the causal relationship between X 2 and the outcome variable Y . This counts as ‘Qualitative Research Guides Quantitative Questions’. Second, the qualitative study might suggest that X 2 is a common cause of X1 and Y , and so one needs to include X 2 in one’s econometric model in order to satisfy one’s econometric assumption that the X variables not correlate with the error term ‹ (that is to say, to avoid ‘omitted variable bias’, as discussed earlier). This counts as ‘Qualitative Research Justifies Quantitative Assumptions’. In summary, there are at least five ways in which quantitative sources of evidence and modes of inferences can be integrated with qualitative sources of evidence and modes of inference. 11.3

Triangulation and Counterfactual Dependence

The last section discussed, in brief, the question of how quantitative and qualitative evidence/modes of inference might be genuinely integrated. In the rest of this chapter, I  will show that one cannot address this question

218

Christopher Clarke

more concretely without taking a position on the ontology of causation. To show this, I will focus on an example of cross-checking triangulation. In this section, I will explore how cross-checking triangulation works according to a simple ontology of causation – an ontology in which all causal claims are understood as claims of counterfactual dependence. Then in later sections, I will contrast this logic with the logic according to a more complex ontology of causation. On this basis, I will show how different ontological assumptions about the nature of causation entail different conclusions about what crosschecking triangulation needs in order to deliver successful causal inferences. (And I contend that something very similar could be said for the other four ways of integrating the quantitative and the qualitative that I described in the last section.) My example of cross-checking triangulation will be Hummel et al. (2021) – a mixed-methods design that looks at the causal relationship between political finance subsidies and corruption. The qualitative part of the study focused on Paraguay alone. The authors interviewed various key actors about the results of the 1996 political finance reforms in Paraguay, which introduced public funding for politicians, clarified what political financing was lawful and unlawful, and introduced penalties for unlawful political financing. This qualitative part of the study asks: did these finance reforms reduce the financial burdens on politicians and thereby make them less corruptible? The causal conclusion suggested by the qualitative study is that the political finance subsidies that Paraguay introduced in 1996 reduced corruption. Let’s call this proposition ‘Single’ to mark that it refers to a single case, namely Paraguay. In contrast, the quantitative part of the study collected a dataset of 175 countries. And it measured the political finance subsidies in a given year on a quantitative scale from 0 to 5. For each country and for each year, the authors aggregate these political finance subsidies over the previous year into a ‘political finance stock’ variable – which measures the degree to which a country has experienced political finance subsidies over the previous years, stretching back decades.7 The theoretical maximum value that this stock variable can take is 268.8 As it happens, the authors find it more convenient to work with log subsidy stock (the natural logarithm of this stock variable), whose theoretical maximum value is 5.6 units. Now, the causal conclusion suggested by the quantitative study is the following: for all countries and for any given year, for each extra unit of log subsidy stock, this reduces corruption five years later by .02 units – where corruption is measured on a quantitative scale from 0 to 1.9 Notice that this conclusion says that the causal contribution of each unit of log subsidy stock is the same for all countries, namely .02 units of corruption. In this respect, it’s worth noting that this conclusion entails a claim of causal homogeneity across all countries (King et  al., 1994, chap.

Mixed Methods and Causal Ontology

219

3.3.1). Let’s call this proposition ‘General’ to mark that it refers to a population of cases (namely all countries). What is the relationship between Single (the proposition established by the qualitative study) and General (the proposition established by the quantitative study)? In other words, what is the nature of the causal relation posited by the qualitative study, and what is the nature of the causal relation posited by the quantitative study, and how (if at all) does the nature of these two relation differ? One answer to this ontological question is that both Single and General express counterfactual conditionals – conditionals of the form ‘if it hadn’t been the case that X, then it would not have been the case that Y ’ or of the form ‘if it were not the case that X, then it would not be the case that Y ’. To be specific, this ontology of causation says that Single means: if Paraguay hadn’t introduced political finance subsidies in 1996, then corruption a few years later would have been considerably greater than it actually was. Now, let’s stipulate that by ‘considerably greater’ we mean at least .05 units greater than its actual value, and let’s stipulate that by ‘a few years later’ we mean the year 2015. So Single means: if Paraguay hadn’t introduced political finance subsidies in 1996, then corruption in the year 2015 would have been at least .05 units greater than its actual value. Now, to translate Single into a quantitative claim that can be compared with the claims made by the quantitative study, we also need to translate it into a claim that relates log subsidy stock to corruption five years later. Given this, let’s assume that Paraguay’s not introducing political finance subsidies in 1996 meant the log subsidy stock remaining at its previous level (namely 0 units) until at least 2010 (rather than increasing as it actually did so that the log subsidy stock by 2010 was 4 units).10 So translated, Single becomes the claim: Single Depend. If in 2010 Paraguay’s log subsidy stock had been 4 units less than it actually was, then corruption in the year 2015 would have been at least .05 units greater than it actually was. I call this claim ‘Single Depend’ to mark that is a claim about the counterfactual dependence of corruption on finance subsidies. Compare this with General, which according to our ontology is also a claim about the counterfactual dependence of corruption on finance subsidies: General Depend. For any given country and for any given year and for any value you choose for x : if log subsidy stock had been x units greater than it actually was, then corruption five years later have been .02x units less than it actually was. By considering the year 2010 and by choosing the value x = -4, one can easily observe that General Depend entails Single Depend. In light of this, it is

220

Christopher Clarke

useful to consider General Depend to be the conjunction of two sub-claims – the first sub-claim is that General Depend applies to Paraguay in 2010, and the second sub-claim is that General Depend applies to cases other than Paraguay in 2010. Let’s call this second sub-claim General* Depend, to mark that it isn’t quite a fully general claim, in that it excludes the case of Paraguay in 2010. But we’ve already seen that the first sub-claim is more or less equivalent to Single Depend.11 So we can treat General Depend as the conjunction of Single Depend and General* Depend. In light of this, it is easy to see the logic behind Hummel et al.’s study is that depicted in Figure 11.1. Namely, the qualitative evidence about Paraguay supports Single Depend, and the quantitative evidence about all countries supports Single Depend and supports General* Depend also. Since General Depend is more or less the conjunction of Single Depend and General* Depend, it follows that the qualitative evidence supports General Depend (at least to some small degree) and the quantitative evidence supports General Depend.12 In summary, each of Single Depend and General Depend is supported by both the qualitative evidence and the quantitative evidence. So here we have a genuine case of cross-checking triangulation. To draw this conclusion about the logic of triangulation, however, I relied on a number of assumptions, which I will divide into three types: (1) variable translation assumptions, (2) the assumption of causal homogeneity, and (3) the ontological assumption that causation is counterfactual dependence. The variable translation assumptions were the following: (a) by a ‘few years later’ we mean the year 2015; (b) by ‘considerably greater’ corruption we mean corruption being at least .05 unit greater than its actual value; (c) Paraguay’s not introducing political finance subsidies in 1996 means log subsidy stock remaining at its previous level (0 units) until at least 2010, rather than increasing as it actually did to 4 units in 2010. I call these translation assumptions, because they translate the descriptions used by the qualitative study into particular values of the quantitative variables examined by the

FIGURE 11.1

The Logic of Cross-Checking Triangulation

Mixed Methods and Causal Ontology

221

quantitative study. Though they are not without their complications, I will set these translation assumptions aside for the moment as the least problematic of our three types of assumptions. (I will return to these translation assumptions later.) The second assumption I made was that of causal homogeneity. Everything I said relied on the assumption that General Depend is the conclusion of the quantitative study. And General Depend says that – for all countries – the causal contribution of each extra unit of log subsidy stock is .02 units of corruption. Indeed, to get this conclusion as an output of standard quantitative methods, one needs to assume that the causal contribution of each extra unit of log subsidy stock (whatever it is) is the same for all countries. Formally, one’s econometric model needs to treat b as constant across countries. This is the causal homogeneity assumption (King et al., 1994, chap. 3.3.1). Although this assumption is very common in applied work, some methodologists deny that it is true (Morgan & Winship, 2007; Manski, 2007; Mahoney, 2010; Crasnow, 2012). If one denies causal homogeneity, then one must interpret the conclusion of the quantitative study as a much more modest one. Although causal contribution b can vary considerably from country to country, it’s average value across all countries is b = .02 units of corruption. Let’s call this alternative conclusion ‘Average General Depend’. Now, if there is considerable causal heterogeneity – in that causal contribution b varies considerably from country to country – then this average value b = .02 tells us very little about the value of causal contribution b for Paraguay in particular. So Average General Depend does not entail Single Depend. Indeed, Average General Depend doesn’t even make Single Depend probably true. This illustrates the role of my second assumption (causal homogeneity) in ensuring that this mixed methods study has the triangulation logic that is depicted in Figure 11.1. Without this assumption of causal homogeneity, this logic breaks down. Indeed, I suspect that this is the fundamental reason why Rohlfing and Zuber (2019, p. 26) denies that cross-checking triangulation can work. The third assumption I made was an ontological one: both Single and General are to be interpreted in terms of counterfactual dependence. However, as we will see in the next section, some methodologists deny that the causal conclusions of the qualitative studies are to be interpreted in terms of counterfactual dependence. In summary, when I  presented my story about the logic of triangulation I  made assumptions of three types: (1) variable translation assumptions, in which the descriptions given in quantitative studies are translated into the values taken by quantitative variables, (2) causal homogeneity assumptions, and (3) the ontological assumption that the causal claims established by the qualitative researcher are claim of simple counterfactual dependence. I’ve noted that the causal homogeneity assumption is open to question. And in

222

Christopher Clarke

the next section, I  will present some reasons to deny the latter ontological assumption – that the causal claims established by the qualitative researcher are claims of counterfactual dependence. 11.4

Ontological Assumptions in Qualitative Research

Some methodologists deny that the causal conclusions of qualitative studies are to be interpreted in terms of counterfactual dependence (Beach  & Pedersen, 2016). One reason that Beach and Pedersen offer for this claim is the following. The most straightforward way of establishing counterfactual dependence is by comparison. To see whether the improvement in a patient’s health counterfactually depended on her taking a drug, for example, one might compare her health with the health of someone very similar to her who did not take the drug. Or, more commonly, one might compare the average health of the members of a treatment group (all of whom take the drug) with a control group (none of which take the drug). But typical qualitative research does not seem to perform such comparisons – at least not of this straightforward treatment versus control variety. But this is the only way of establishing counterfactual dependence, Beach and Pedersen seem to assume. So typical qualitative research cannot establish counterfactual dependence. So any charitable interpretation of this qualitative research should interpret its causal claims as claims about something other than simple counterfactual dependence. So qualitative research does not typically aim to establish counterfactual dependence. That is Beach and Pedersen’s argument, as I reconstruct it. However, this argument is too quick in my view. After all, perhaps there are some other methods for establishing counterfactual dependence that don’t require comparisons of the straightforward treatment versus control variety. So this argument isn’t that powerful, I’d say. Nevertheless, there is a more powerful argument that qualitative research does not study counterfactual dependence (but instead some other conception of causation). Take the First World War as an example. One of the causes of the First World War was the assassination of the Austro-Hungarian archduke by Serbian nationalists. Nevertheless, some scholars think that there still would have been a war between the European powers in the 1910s, even without the assassination. This is because tensions between the countries were already running high, because of (i) French distrust of Germany, (ii) German isolation as a result of the Triple Entente between the French, British and Russian empires, (iii) the isolation of the Ottoman empire, and the (iv) conflict over Morocco, among other things. As a result, even if the archduke had not been assassinated, there would very probably have been some other crisis that precipitated a war between the European powers in the 1910s, these scholars suggest. In summary, these scholars say that (1) the war did not counterfactually depend on the assassination, but (2) the assassination was

Mixed Methods and Causal Ontology

223

one of the causes of the war (more accurately: of there being a war between the European powers in the 1910s). However, according to the ontology of causation as counterfactual dependence, claims 1 and 2 are incompatible. So any charitable interpretation of these qualitative researchers will interpret their causal claims as claims about something other than counterfactual dependence. So, insofar as this qualitative research on the First World War is typical, qualitative research does not typically make claims about counterfactual dependence. In summary, there are two reasons (one more powerful one less powerful) to think that typical qualitative research does not make claims about simple counterfactual dependence. I don’t claim that this more powerful reason is absolutely decisive. After all, this reason appealed to the principle of charity: interpret qualitative scholars in such a way as to making their reasoning as reasonable as possible. And, for all I’ve said, it’s possible that there will turn out to be no way of making sense of this qualitative research. 11.5

Causal Production

If causation in qualitative political science isn’t counterfactual dependence, then what is it? Beach and Pedersen (2016) contend that there are two types of causation (see also Hall, 2004). On the one hand, there is causation as understood in terms of counterfactual conditionals. On the other hand, there is a type of causation that has absolutely nothing to do with counterfactual conditionals. This latter sort of causation they call ‘mechanistic’ causation or ‘generative’ causation or causal ‘production’. In short, Beach and Pedersen have a radically dualist concept of causation: there is counterfactual dependence, and there is causal production, and they bear very little relation to each other. Beach and Pedersen claim that qualitative studies in political science are mostly interested in causal production and quantitative studies in counterfactual dependence. What’s the difference between counterfactual dependence and causal production? Imagine, for example, a hiker who strikes a match and then places the burning match next to some kindling. At the point of contact between match and the kindling, the kindling starts to char and then to smoke and then bursts into flame. This is strong evidence that the burning of the match causally produced the kindling burning, one might intuitively feel. But this evidence is very weak evidence that the kindling’s burning counterfactually depended upon the burning of the match. After all, had the match been damp and therefore not burned, the hiker might have instead used a cigarette lighter to ignite the kindling. If so, the kindling’s burning was not counterfactually dependent upon the match’s burning. Nevertheless, one might intuitively feel that the match’s burning causally produced the kindling’s burning. This shows that establishing counterfactual dependence requires greater knowledge

224

Christopher Clarke

about the surrounding circumstances, as it were, while establishing causal production seems to require somewhat less of this knowledge (Paul & Hall, 2013). Indeed, one might intuitively feel that there is a sense in which one can directly see that the match’s burning causally produced the kindling’s burning (Brady, 2010) – even though one cannot directly see that that the kindling’s burning counterfactually depended upon the match’s burning. Whatever one ultimately makes of these intuitions, they are at very least prima facie evidence that there is some important or useful concept of causal production that differs from counterfactual dependence, I’m happy to agree. In light of this, Beach and Pedersen contend that qualitative scholars are interested in causal production (so understood) rather than counterfactual dependence. 11.6

Problems with Causal Production

When one goes beyond this prima facie evidence offered by intuition, however, problems begin to emergence for the ontology of causation as causal production. There are (i) problems for this ontology of causal production in social settings, and there are (ii) problems with using this ontology for the purposes of interpreting mixed-methods research. I will now give an example of i and ii. For another example of i, see Clarke (2023b), in which I show that our intuitions about causal production are incoherent. These problems will motivate me, in the next section, to look at a third ontology of causation as an alternative to the ontology of causal production and the ontology of counterfactual dependence. Now, Beach and Pedersen tell us that, when one event causally produces another event, this involves one entity performing an activity upon another entity. For example, when the match’s burning causally produces the kindling’s burning, this involves the match igniting the kindling. Here ignition is the activity; and the match and the kindling are the entities. Beach and Pedersen’s talk of entities and activities invokes a philosophical tradition from Anscombe (1971) to Cartwright (1999) to the recent philosophical literature on causal mechanisms (Machamer et al., 2000). In this tradition, causal production is understood as general concept that encompasses various more specific activities – activities such as pushing, carrying, knocking over, scraping, and squashing. The idea seems to be that this general concept gets its meaning from these more specific concepts, in the following way. First, we have some intuitive idea about what pushing means for example, and what carrying means for example, and so on. Second, since causal production is just a general concept that encompasses these more specific concepts, the general concept of causal production itself becomes a meaningful one. That’s the idea. Unfortunately, this story does not help us understand the concept of causal production in the context of the social sciences, I think. Why not? To make

Mixed Methods and Causal Ontology

225

the concept of causal production a meaningful one, one must – at the very least I  assume – say what would count as evidence that an event C causally produced an event E. Of course, one doesn’t need to provide an infallible method of establishing that C causally produced E. One just needs to give a rough idea about what sort of evidence would warrant a researcher to increase their confidence that C causally produced E. In the case of scraping and pushing and burning and the like, I’m happy to grant that researchers can more or less directly see that one entity scraped another entity, or that one entity pushed another entity, or that one entity burned another entity. So in these contexts, one can more or less directly see that C causally produced E. And so, in these contexts, the claim that C causally produced E is indeed a meaningful one. The general concept of causal production does indeed get its meaning from these more specific concepts, I’m happy to agree. However, in the context of political science, for example, the activities that qualitative research studies are activities such as: actor A successfully persuades actor B to perform action C; actor A successfully threatens actor B not to perform action C; actor A successfully warns actor B that proposition C is true; actor A successfully organizes actors B to perform action C; and so on. Take for example the claim that Blair persuaded Bush to seek a UN mandate for the invasion of Iraq. Now, I’m happy to grant that one can directly see from their recorded correspondence that Blair attempted to persuade Bush. And one can directly see that Bush did in fact subsequently seek a UN mandate. But, I insist, one cannot directly see whether Bush’s seeking a UN mandate was causally produced by Blair’s attempt to persuade him – as opposed to it being causally produced by something else, by some other person or consideration for example. One can of course get evidence for and against this hypothesis (that Bush’s behaviour was causally produced by Blair) but one cannot directly see whether or not this hypothesis is true or false. In short, in order to work out whether Blair successfully persuaded Bush, one needs inference, not direct observation. (In a moment, I will respond to the objection that one can indeed directly observe that Blair successfully persuaded Bush.) If I’m right here, one cannot directly observe most of the key causal activities in qualitative research in political science – activities such as successful persuasion, successful threat, successful warning, and the like. But, as I’ve already noted, the meaningfulness of causal production as a general concept is supposed to rest on the meaningfulness of these more specific activities. So Beach and Pedersen need to tell us how one might get evidence that Blair successfully persuaded Bush, for example. Until they do this – and until they do the same for activities such as successful threat and successful warning and the like – the concept of causal production is not a meaningful one in the context of political science. This is one problem for Beach and Pedersen’s ontology of causal production: causal production as it stands is ill-defined, at least in social science settings. Or so I’d argue. (Of course, a relatively easy

226

Christopher Clarke

way to make successful threat and successful warning and the like meaningful is to define them in terms of counterfactual conditionals. But this is precisely what Beach and Pedersen do not want to do.) There is also a second problem for the ontology of causal production. This problem arises when one tries to interpret what is going on in mixed-methods research. The problem is this: cross-checking triangulation requires that one’s qualitative study and one’s quantitative study each provide evidence for both Single and General (see Section 11.3). But Beach and Pedersen’s ‘radically dualistic’ ontology says that Single is to be interpreted in terms of causal production, which has absolutely nothing to do with counterfactual conditionals, they claim. But everyone agrees that the results of quantitative studies, such as General, are to be interpreted in terms of counterfactual conditionals. It follows that there is a wide ontological gulf between these two types of causation – a large difference in meaning between Single and General. But the logic of triangulation requires there to be a closer relationship between Single and General, such that evidence for Single can be evidence for General too, and such that evidence for General can be evidence for Single. Therefore, the prospects of genuine cross-checking triangulation look dim, according to Beach and Pedersen’s ‘radically dualistic’ ontology. Of course, this is not a problem with Beach and Pedersen’s ontology itself. Instead it’s a problem for anyone who, like Johnson et al. (2019), is committed both to (a) a radically dualistic ontology of causation, or indeed a radically pluralistic ontology of causation and to (b) the possibility of genuinely integrated mixed methods research. For the two reasons I’ve just explored, and for the third reason given in Clarke (2023b), I think it is worth looking at an alternative ontology of causation in qualitative political science, other than causation as counterfactual dependence and other than causation as causal production. 11.7

An Objection

One response on behalf of Beach and Pedersen is that a qualitative researcher can simply ask Bush whether he was successfully persuaded by Blair (or indeed discover a secret diary of Bush’s in which he says ‘it was Blair who persuaded me’). This is relatively direct evidence that Blair successfully persuaded Bush, one might claim. In my view, this response does not solve the problem at hand. Of course, it’s true that Bush can report his own beliefs, hopes, desires, preferences, values, and the like to the qualitative researcher. And I’m happy to grant that Bush can do so more or less reliably (if he chooses). I’m also open to the idea that Bush can reliably report various counterfactual conditionals of the form: if Blair hadn’t tried to persuade me (Bush), then I would have acted differently and not gone to the UN. However, Beach and Pedersen claim that causal

Mixed Methods and Causal Ontology

227

production (whatever it is) has nothing whatsoever to do with counterfactual conditionals. To emphasize this, let’s call this type of causal production BP causal production. So it does not follow Bush can reliably report whether Blair successfully persuaded Bush, when we interpret successful persuasion in terms of BP causal production rather than in terms of counterfactual conditionals. In short, it does not follow that Bush can report on what BP causally produced his own actions. So Beach and Pedersen’s reply simply assumes without argument that Bush can reliably report on what BP causally produced his own actions. But to assume this is to assume without argument that the concept of BP causal production is a meaningful one for Bush. In summary, this reply on behalf of Beach and Pedersen ‘solves’ the problem of making BP causal production meaningful for the qualitative researcher by simply assuming without argument that BP causal production is meaningful for the actors that the qualitative researcher is interviewing. In my view, this solution does not succeed in illuminating the meaning of BP causal production. 11.8

The Ontology of Sufficient Causes

One promising alternative to the latter two ontologies is to see causal claims in qualitative political science as expressing claims of causal sufficiency (Ragin, 1987; Baumgartner, 2009). This idea is very closely related to the idea that causal claims express what Mackie ([1974] 1980) calls ‘INUS conditions’, which some methodologists think are what qualitative political scientists study (Mahoney, 2015).13 I will now explore the prospects of crosschecking triangulation between quantitative methods (which study counterfactual dependence) and qualitative methods (which study causal sufficiency we are now imagining). Since this is my goal, I’m going to offer a somewhat idiosyncratic interpretation of causal sufficiency – one in which causal sufficiency is cashed out in terms of counterfactual conditionals. I call this concept counterfactual sufficiency. To illustrate the idea of counterfactual sufficiency, consider Chile in 2019 – a year which saw massive political protests at the deepening inequality between the rich and poor. These protests hit the Chilean economy hard. In 2020, the COVID-19 pandemic also hit the economy hard as well. Following the protests and the pandemic, Chile experienced mass unemployment in 2020. To analyse this case, let’s use the ⇒ arrow as a shorthand for counterfactual conditionals, and let’s imagine that the following four counterfactual conditionals are true of Chile in 2020: 1. 2. 3. 4.

[Pandemic = yes and Protest = yes] ⇒ [Mass Unemployment = yes] [Pandemic = yes and Protest = no] ⇒ [Mass Unemployment = yes] [Pandemic = no and Protest = yes] ⇒ [Mass Unemployment = yes] [Pandemic = no and Protest = no] ⇒ [Mass Unemployment = no]

228

Christopher Clarke

Since there actually were protests in Chile, it follows that: [Pandemic = no] ⇒ [Mass Unemployment = yes]. That is to say, if the pandemic hadn’t occurred, Chile would have still suffered mass unemployment. So this mass unemployment was not counterfactually dependent upon the pandemic. So the ontology of causation as simple counterfactual conditionals says that the pandemic was not a cause of the mass unemployment. Nevertheless, there is an intuitive sense in which the pandemic was sufficient ‘in the circumstances’ for mass unemployment. To make this intuitive idea more precise, let’s note three things about counterfactuals 1–4. First, in considering these four counterfactuals, we are considering counterfactual scenarios in which the Pandemic variable varies (the variable that denotes whether there is a pandemic) and also in which the Protest variable varies (the variable that denotes whether there are protests). Indeed, these four counterfactual scenarios exhaust the possible variation in the Pandemic variable and the Protest variable. Second, in all of the possibilities in which pandemic occurs (namely counterfactuals 1 and 2), there is mass unemployment. Third, for some of these four possibilities, there is not mass unemployment. So our second observation – that in all the possibilities in which the pandemic occurs, there is mass unemployment – is indeed something distinctive about the possibilities in which the pandemic occurs. It is not something that is true of all four possibilities (see counterfactual 4). Let’s summarize these three observations by saying that the pandemic is non-trivially sufficient for mass unemployment across counterfactual variation in the Pandemic and Protest variables. More generally, to say ‘condition X1 = x is non-trivially sufficient for an outcome Y = y across counterfactual variation in variable X1 and another variable or variables X 2 ’ is to say that the following two conditions hold: Sufficiency. For any given value x2 of variable X 2 , it’s the case that [ X1 = x and X 2 = x2 ] Þ [Y = y ]. Non-Triviality. For some value x1 of variable X1 and some value x2 of variable X 2 it’s the case that [ X1 = x1 and X 2 = x2 ] Þ [Y ¹ y ]. When X1 = x is non-trivially sufficient for an outcome Y = y across counterfactual variation in variable X1 and another variable or variables X 2 , we can just leave this condition X 2 implicit and simply say that X1 = x is nontrivially sufficient for outcome Y = y . For example, we can simply say that the Pandemic is non-trivially counterfactually sufficient for mass unemployment (leaving it implicit that the variation we have in mind is variation in the Pandemic and Protest variables). Similarly, when X1 = x is non-trivially sufficient for an outcome Y = y across counterfactual variation in variable X1, we can

Mixed Methods and Causal Ontology

229

just leave things implicit and simply say that X1 = x is non-trivially counterfactually sufficient for outcome Y = y . In sum: Non-Trivial Counterfactual Sufficiency. To say that X1 = x is non-trivially counterfactually sufficient for outcome Y = y is to say that either: a) there is a variable or variables X 2 for which X1 = x is non-trivially counterfactually sufficient for an outcome Y = y across variation in variable X1 and X 2 ; or b) X1 = x is non-trivially counterfactually sufficient for an outcome Y = y across variation in variable X1 alone. For example, the pandemic is non-trivially counterfactually sufficient for mass unemployment if and only if (a) there is a variable X 2 (the Protest variable for example) for which the pandemic is non-trivially counterfactually sufficient for mass unemployment across variation in the Pandemic variable and variable X 2 ; or (b) the pandemic is non-trivially counterfactually sufficient for mass unemployment across variation in the Pandemic variable alone. This puts me in a position to state the sufficiency ontology of causation: a condition X1 = x is a cause of an outcome Y = y if and only if X1 = x is (non-trivially) counterfactually sufficient for Y = y . For ease of expression, when I talk about counterfactual sufficiency from now on, I will leave it implicit that I’m talking about non-trivial counterfactual sufficiency. How charitable is it to interpret qualitative researchers’ causal claims as claims of counterfactual sufficiency? Very much so, I think. This ontology is able to make sense of qualitative scholars who say: (1) the First World War did not counterfactually depend on the assassination, but (2) the assassination was a cause of the war (namely a counterfactually sufficient cause). After all, as we’ve just seen, one event can be counterfactually sufficient for a second event, without the second event counterfactually depending upon the first. How might counterfactual sufficiency apply when we are talking about quantitative variables? To answer this question, let Y denote the number of people unemployed in Chile (measured in millions), and let X1 denote the number of COVID infections (measured in millions), and let X 2 denote the number of people protesting (measured in millions). And let’s imagine that 1 unemployment Y in Chile is equal to Y = ( X1 + 5) ( X 2 + 1) + 1. That is to say, 28 no matter the number of COVID infections X1, and no matter the number of people protesting X 2 , this equation for unemployment holds true. For short, 1 the equation Y = ( X1 + 5) ( X 2 + 1) + 1 holds across counterfactual variation 28 in infections X1 and protestors X 2 . Since unemployment Y is an increasing

230

Christopher Clarke

function of infections X1, and since X 2 = 0 is the minimum number of protestors, it follows that for all values of X1 greater than 2  million infections, 1 unemployment will be at least Y = ( 2 + 5) ( 0 + 1) + 1 = 1.25. In other words, 28 infections being X1 > 2 is counterfactually sufficient for unemployment Y > 1.25 across counterfactual variation in both infections X1 and protestors X 2. How does this ontology of causation compare with other ontologies? As far as the simple counterfactual ontology is concerned, note that whenever Y = y counterfactually depends upon X1 = x , then X1 = x is non-trivially counterfactually sufficient for Y = y across variation in X1 = x , and so X1 = x is counterfactually sufficient for Y = y . So whenever X1 = x causes Y = y according to the simple counterfactual ontology then X1 = x causes Y = y according to the counterfactual sufficiency ontology. However, our Chile example shows how the reverse is not always true: mass unemployment doesn’t counterfactually depend upon the pandemic, but the pandemic is counterfactually sufficient for mass unemployment. In this respect, the concept of counterfactual sufficiency is a broader concept than the concept of counterfactual dependence. In summary, I  offered the ontology of causation as counterfactual sufficiency as a charitable way of making sense of some of the causal talk in qualitative political science. 11.9

Triangulation and Counterfactual Sufficiency

Let’s now return to my main exemplar of triangulation, namely Hummel et al. (2021), which contained both a quantitative study and a qualitative study of the relationship between political finance subsidies and corruption. I will now show how to make sense of this study when one interprets the qualitative study as making claims about counterfactual sufficiency. Remember that the causal conclusion suggested by the qualitative study is that the political finance subsidies that Paraguay introduced in 1996 reduced corruption (by the year 2015 for example). We called this proposition Single to mark that it refers to a single case, namely Paraguay. Interpreted as a claim about counterfactual sufficiency, Single says that the political finance subsidies that Paraguay introduced in 1996 were counterfactually sufficient for corruption to be relatively low. To translate this into a quantitative claim, let’s stipulate that ‘relatively low’ means no greater than .80 units.14 What’s more, to translate this into a quantitative claim that can be compared with the claims made by the quantitative study, we need to translate it into a claim that relates log subsidy stock to corruption five years later. Well, the political finance subsidies introduced in 1996 meant that, in 2010, the log subsidy stock was 4 units, according to Hummel’s dataset. Therefore, Single should be interpreted as: Single Suffice. Paraguay’s log subsidy stock in 2010 being 4 units was counterfactually sufficient for corruption to be no greater than .80 units in 2015.

Mixed Methods and Causal Ontology

231

In contrast, the overall causal conclusion of the quantitative study is the following: Ci [t + 5] = -.02Sit + .16Dit - .33Dit2 - .04Git + a i + g t + ‹ it Here Sit denotes the log subsidy stock for a given country i in a given year t . Dit is a measure of democracy; and Git is the log of GDP growth. Ci [t + 5] denotes corruption for that country five years later. Each a i denotes a number that varies from country to country, but not from year to year. This number is meant to account for ‘country fixed effects’ – the impact on corruption of factors that differ between countries but that are stable within countries over time. Similarly, each g t denotes a number that varies from year to year, but not from country to country. This number is meant to account for ‘yearly fixed effects’ – the impact on corruption of factors that are the same for each country but that differ from year to year. (Each of a i and g t was automatically calculated by the statistics package that Hummel et al. used to analyse their data.) ‹ it is the error term. This is introduced to account for the fact that subsidies, democracy, and growth together are not perfect predictors of corruption (for any given country and any given year). It will be useful to rewrite this equation in a mathematically equivalent form as Ci [t + 5] = -.02Sit - .33[Dit - .24] + .019 - .04Git + a i + g t + ‹ it 2

Let’s ask: when log subsidy stock S is equal to a given value s for a given country i and a given year t , what is the maximum value of corruption C five years later? To find the maximum value of C , we need to consider the minimum possible value of log growth G. Since the biggest recessions in recorded history have involved a log growth of just under -1, let’s consider G = -1. To find the maximum value of C , we also need to consider Dit = .24, since this 2 makes the -.33[Dit - .24] expression as high as possible. This yields: Cmax = -.02s - .33[.24 - .24] + .019 - .04 ( -1) + a i + g t + ‹ it 2

= -.02s + .059 + a i + g t + ‹ it So to calculate the maximum value Cmax of corruption when log subsidy stock S = s, we need to answer the question: what is the maximum value ‹ max of the error term ‹ it ? This is an awkward question to answer. Econometric models almost always assume that the probability distribution governing ‹ it is a bellcurve.15 This entails that there is a non-zero probability that ‹ it could take a large value, large enough to make C greater than it’s theoretical maximum of 1. Similarly, this entails that there is a non-zero probability that ‹ it could take a small value, small enough to make C less than it’s theoretical minimum value of 0. This shows that the econometric model that Hummel et al. use contradicts what they say about the theoretical maximum and minimum

232

Christopher Clarke

values of C . Therefore, Hummel’s econometric model is false – strictly speaking – although it may still be a useful approximation to the truth. This awkward point notwithstanding, one natural approach to estimating ‹max would be to observe the value of ‹ it for each year – country pair, and then use these observed values as a guide to what the maximum possible value of ‹ it might be.16 One very crude way of doing this would be to take the maximum observed value of ‹ it to be the maximum possible value. Let’s imagine that this gives us ‹max = .08 and so we have Cmax = -.02s + .139 + a i + g t. Thus the quantitative study entails the following conclusion: General Suffice. A  country i ’s log subsidy stock in year t being s units is counterfactually sufficient for corruption to be no greater than -.02s + .139 + a i + g t units five years later (where a i for a given country and the g t for a given year are as calculated by the statistics package that Hummel et al. used). Thus, although the quantitative study entails claims about the counterfactual dependence of corruption C upon political finance subsidies S, it also entails General Suffice, which is instead a claim about counterfactual sufficiency. Let’s assume for sake of illustration that a = .501 for Paraguay and that g = .18 for the year 2010. Considering s = 4, this yields the conclusion that: Paraguay’s log subsidy stock in 2010 being 4 units was counterfactually sufficient for corruption to be no greater than -.02 ( 4 ) + .139 + .501+ .18 = .74 units in 2015. This shows that General Suffice entails Single Suffice. In light of this, it is useful to consider General Suffice to be the conjunction of two sub-claims – the first sub-claim is that General Suffice applies to Paraguay in 2010, and the second sub-claim is that General Suffice applies to cases other than Paraguay in 2010. Let’s call this second sub-claim General* Suffice. And we’ve already seen that the first sub-claim is more or less just Single Suffice. So we can treat General Suffice as the conjunction of Single Suffice and General* Suffice. In light of this, it is easy to see the logic behind Hummel et al.’s study is that depicted in Figure  11.2. Namely, the qualitative evidence about Paraguay supports Single Suffice, and the quantitative evidence about all countries supports Single Suffice and supports General* Suffice. Since General Suffice is more or less the conjunction of Single Suffice and GeneralX Suffice, it follows that the qualitative evidence supports General Suffice (at least to some small degree) and the quantitative evidence supports General Suffice. In summary, each of Single Suffice and General Suffice is supported by both the qualitative evidence and the quantitative evidence. So here we have a genuine case of cross-checking triangulation.

Mixed Methods and Causal Ontology

FIGURE 11.2

233

Relationship of Evidential Support

Note, however, that in order to make this argument, we again needed to make an assumption of causal homogeneity (as we discussed in Section 11.3). And we also needed to make a large number of translation assumptions – some of which were rather awkward. First, we assumed that ‘relatively low’ corruption means no greater than .80 units. Second, we assumed that the qualitative study was making a claim that relates log subsidy stock to corruption five years later. Third, we made an assumption about the minimum possible value of log GDP growth. Fourth, we needed to estimate the minimum possible value of the error term. And we did so using an extremely crude method. This shows that standard quantitative methods can only establish causal sufficiency claims with considerable extra work – work that quantitative scholars never undertake, as far as I’m aware.17 For this reason, it is unclear whether Hummel et al.’s cross-checking triangulation succeeds – if one interprets the causal claim in their qualitative study in terms of counterfactual sufficiency. This poses a problem for interpreting Hummel et al.’s qualitative study in terms of counterfactual sufficiency. 11.10

Conclusion

This chapter has examined how qualitative evidence and modes of inference might be genuinely integrated with quantitative evidence and modes of inference. One such way is cross-checking triangulation. But the methodological status of triangulation very much depends upon one’s causal ontology, I’ve shown. In particular, it depends upon one’s answer to the following question: when qualitative political scientists make causal claims, what exactly are they claiming? If qualitative causal claims are claims of counterfactual dependence, then it is questionable whether qualitative research is able to establish such causal claims, I’ve noted. And so the possibility of triangulation is called into question. If instead qualitative causal claims are claims of causal production (in Beach and Pedersen’s radical sense) then it is unclear why evidence

234

Christopher Clarke

for Single (about causal production) would be evidence about General (about counterfactual dependence) and vice versa. And so again the possibility of triangulation is called into question. If instead qualitative claims are claims of counterfactual sufficiency, then triangulation is possible. But it will often require the political scientist to make and defend some awkward translation assumptions – a defence that political scientists never undertake. I conclude the logic of triangulation very much depends on an ontological question: what is the proper interpretation of causal claims in qualitative political science? Indeed, I suspect that the same is true of other types of mixed-methods integration, not just for cross-checking triangulation. Notes 1 One exception is Rohlfing and Zuber (2019), who briefly discuss mixed methods. Another exception is perhaps the ontological question of whether the social world is socially constructed. One prominent question is: how can one make sense of mixed-methods research, given that social constructivism (which is associated with the qualitative tradition) is incompatible with various realist alternatives (more associated with the quantitative tradition)? 2 See Lieberman (2005), Weller and Barnes (2014), and Gerring and Cojocaru (2016) for examples and discussion. Typically this approach involves using quantitative methods to draw a regression line. One then identifies a case that is very close to this regression line and is thus an interesting exemplar of the general trend. Or one instead identifies a case that is an outlier, and which is thereby worth studying with an in-depth qualitative case study to explain why this case is an outlier. 3 Take for instance Blattman and Miguel (2010) who preface their qualitative approach to the causes of civil war by rehearsing several theories of the causes of civil war, and citing several qualitative studies that prima facie support these theories, and thus make it interesting to test these theories. 4 Imagine that the treatment variables X do not share any ‘common causes’ with the outcome variable Y . That is to say, there is no third variable that is both (i) a cause of the treatment variables X and (ii) a cause of Y via some ‘causal path’ that doesn’t go through the treatment variables X . But the ‹ variable represents ii: it denotes the component of Y that arises from causes other than the treatment variables X . It follows that there is no variable that is a common cause of (i) X and (ii) the error term ‹. But by definition the error term is neither caused by X nor is a cause of X . And so, by Reichenbach’s Principle of the Common Cause, there will be no genuine correlation between ‹ and X . That’s because Reichenbach’s Principle says that the only way for two variables to be genuinely correlated is for one to cause the other, or for them both to share a common cause. 5 See Ragin (1987, pp.  76–80), Achen (2002), Freedman (2010), and Achen and Bartels (2018) for some real-life examples of Qualitative Research Justifies Qualitative Assumptions. In particular, see Lyall (2015) and Dunning (2015) for some suggestions for how process tracing can be used to support an econometric model. 6 See Ragin (1987, pp.  71–76), Tarrow (1995), Gerring (2006), McAdam et al. (2008), Mahoney (2010), Schimmelfennig (2015), Lyall (2015), and Humphreys and Jacobs (2015). For philosophical arguments that cross-checking triangulation is vital to establish causal claims in the social sciences, see Moneta and Russo (2014) and Shan and Williamson (2021).

Mixed Methods and Causal Ontology

235

7 This stock variable is calculated by taking the level of political finance subsidy for the current year and adding 99% of the past value of the stock variable. Thus, this stock is assumed to have a 1% depreciation per year. 8 The authors make this claim in Table C3 in the appendix. It seems to me that this figure must be reporting the maximum value that this stock variable can take after 75 years. By my calculations, after 75 years of political finance subsidies being at their maximum (five units) each year, this stock variable would be 267.05. 9 Actually, this is the conclusion with respect to models 2–4 of their five econometric models (see table 3). 10 According to their accompanying dataset. 11 I simplify for ease of illustration. In truth, the first sub-claim is more specific in that it says corruption will be exactly 4×.02=.08 units less, while Single Depend says that it is at least .05 units less. 12 I assume here that the qualitative evidence about Paraguay is either neutral with respect to General Depend or supportive of it. 13 Rohlfing and Zuber (2019) perhaps disagree. 14 This might seem like a very high number, given that maximum corruption is 1. But it’s worth noting that, according to Hummel’s dataset, corruption in Paraguay has never gone below .72. 15 They need to make this assumption or something very similar, in order to calculate the so-called standard errors and confidence intervals for their estimates. 16 I’m talking loosely here for ease of illustration. Strictly speaking, what one is really doing here is looking at the so-called ‘residuals’ calculated by one’s statistics package for each year-country pair. One then assumes that the residuals are a more or less reliable description of the value of the error term for each year-country pair. 17 Indeed, Mahoney (2004, 2010) argues that standard quantitative methods are unsuited for studying causal sufficiency claims.

References Achen, Christopher H. (2002). Toward a new political methodology: Microfoundations and art. Annual Review of Political Science, 5(1), 423–450. https://doi.org/10.1146/ annurev.polisci.5.112801.080943. Achen, Christopher H. (2005). Let’s put garbage-can regressions and garbage-can probits where they belong. Conflict Management and Peace Science, 22(4), 327–339. https://doi.org/10.1080/07388940500339167. Achen, Christopher H.,  & Bartels, Larry M. (2018). Statistics as if politics mattered: A reply to fowler and hall. The Journal of Politics, 80(4), 1438–1453. https://doi. org/10.1086/699245. Anscombe, Gertrude Elizabeth Margaret. (1971). Causality and Determination: An Inaugural Lecture. Cambridge: Cambridge University Press Bates, Robert H. (Ed.). (1998). Analytic Narratives. Princeton NJ: Princeton University Press. Baumgartner, Michael. (2009). Uncovering deterministic causal structures: A Boolean approach. Synthese, 170(1), 71–96. https://doi.org/10.1007/s11229-008-9348-0. Beach, Derek. (2017). Process-tracing methods in social science. https://doi.org/ 10.1093/acrefore/9780190228637.013.176. Beach, Derek, & Pedersen, Rasmus. (2016). Causal Case Study Methods: Foundations and Guidelines for Comparing, Matching, and Tracing. Ann Arbor MI: University of Michigan Press. https://doi.org/10.3998/mpub.6576809.

236

Christopher Clarke

Beck, Nathaniel. (2006). Is causal-process observation an oxymoron? Political Analysis, 14, 347–352. https://doi.org/10.1093/pan/mpj015. Bennett, Andrew,  & Checkel, Jeffrey T. (2015). Process tracing: From philosophical roots to best practices. In Andrew Bennett & Jeffrey T. Checkel (Eds.), Process Tracing: From Metaphor to Analytic Tool (pp. 3–38). Cambridge: Cambridge University Press. Bennett, Andrew, & Elman, Colin. (2006). Qualitative research: Recent developments in case study methods. Annual Review of Political Science, 9, 455–476. https://doi. org/10.1146/annurev.polisci.8.082103.104918. Berk, Richard A. (2004). Regression Analysis: A Constructive Critique (vol. 11). Thousand Oaks, CA: Sage Publications. Bevir, Mark,  & Blakely, Jason. (2018). Interpretive Social Science: An AntiNaturalist Approach (1st ed.). Oxford University Press. https://doi.org/10.1093/ oso/9780198832942.001.0001. Blattman, Christopher, & Miguel, Edward. (2010). Civil war. Journal of Economic Literature, 48(1), 3–57. https://doi.org/10.1257/jel.48.1.3. Brady, Henry E. (2010). Data-set observations versus causal-process observations: The 2000 US presidential election. In Henry E. Brady & David Collier (Eds.), Rethinking Social Inquiry: Diverse Tools, Shared Standards (2nd ed., pp. 237–242). Lanham, MD: Rowman and Littlefield.--Brady, Henry E.,  & Collier, David. (2010). Rethinking Social Inquiry: Diverse Tools, Shared Standards. Lanham: Rowman and Littlefield Publishers. Brodkin, Evelyn Z. (2017). The ethnographic turn in political science: Reflections on the state of the art. Ps: Political Science and Politics, 50(1), 131–134. https://doi. org/10.1017/S1049096516002298. Bryman, Alan. (2012). Social Research Methods (4th ed.). Oxford and New York: Oxford University Press. Cartwright, Nancy. (1999). The Dappled World: A Study of the Boundaries of Science. Cambridge: Cambridge University Press. https://doi.org/10.1017/cbo9781139 167093. Chatsiou, Kakia,  & Mikhaylov, Slava Jankin. (2020). Deep learning for political science. arXiv. https://doi.org/10.48550/arXiv.2005.06540. Clarke, Christopher. (2023a). Process tracing: Defining the undefinable? In Jereon van Bouwel & Harold Kincaid (Eds.), The Oxford Handbook of Philosophy of Political Science. Oxford: Oxford University Press. Clarke, Christopher. (2023b). Why your causal intuitions are corrupt: Intermediate and enabling variables. Erkenntnis. https://doi.org/10.1007/s10670-022-00570-6. Crasnow, Sharon. (2012). The role of case study research in political science: Evidence for causal claims. Philosophy of Science, 79, 655–666. https://doi.org/ 10.1086/667869. Cyr, Jennifer. (2019). Focus Groups for the Social Science Researcher. Cambridge: Cambridge University Press. https://doi.org/10.1017/9781316987124. Druckman, James N., Greene, Donald P., Kuklinski, James H., & Lupia, Arthur (Eds.). (2011). Cambridge Handbook of Experimental Political Science. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511921452. Dunning, Thad. (2015). Improving process tracing: The case of multi-method research. In Andrew Bennett & Jeffrey T. Checkel (Eds.), Process Tracing: From Metaphor to Analytic Tool. Vol. Strategies for Social Inquiry. Cambridge, UK: Cambridge University Press.

Mixed Methods and Causal Ontology

237

Freedman, David A. (1991). Statistical models and shoe leather. Sociological Methodology, 21, 291–313. https://doi.org/10.2307/270939. Freedman, David A. (2010). On types of scientific inquiry: The role of qualitative reasoning. In Henry E. Brady & David Collier (Eds.), Rethinking Social Inquiry: Diverse Tools, Shared Standards (2nd ed.). Lanham, MD: Rowman; Littlefield Publishers. https://doi.org/10.1093/oxfordhb/9780199286546.003.0012. Gerring, John. (2006). Case Study Research: Principles and Practices. Cambridge: Cambridge University Press. https://doi.org/10.1017/9781316848593. Gerring, John, & Cojocaru, Lee. (2016). Selecting cases for intensive analysis: A diversity of goals and methods. Sociological Methods and Research, 45(3), 392–423. https://doi.org/10.1177/0049124116631692. Greene, Jennifer C. (2007). Mixed Methods in Social Inquiry (1st ed.). San Francisco, CA: Jossey-Bass. Grimmer, Justin, Roberts, Margaret E., & Stewart, Brandon M. (2021). Machine learning for social science: An agnostic approach. Annual Review of Political Science, 24(1), 395–419. https://doi.org/10.1146/annurev-polisci-053119-015921. Hall, Ned. (2004). Two concepts of causation. In John Collins, Ned Hall, & L. A. Paul (Eds.), Causation and Counterfactuals (pp. 225–276). Cambridge MA: MIT Press. Hammersley, Martyn. (2008). Troubles with triangulation. In Manfred Max Bergman (Ed.), Advances in Mixed Methods Research (pp. 22–36). London: Sage. Hummel, Calla, Gerring, John, & Burt, Thomas. (2021). Do political finance reforms reduce corruption? British Journal of Political Science, 51(2), 869–889. https://doi. org/10.1017/S0007123419000358. Humphreys, Macartan,  & Jacobs, Alan M. (2015). Mixing methods: A  Bayesian approach. American Political Science Review, 109, 653–673. https://doi. org/10.1017/s0003055415000453. Johnson, R. Burke, Russo, Federica,  & Schoonenboom, Judith. (2019). Causation in mixed methods research: The meeting of philosophy, science, and practice. Journal of Mixed Methods Research, 13(2), 143–162. https://doi.org/10.1177/155868 9817719610. King, Gary, Keohane, Robert O., & Verba, Sidney. (1994). Designing Social Inquiry: Scientific Inference in Qualitative Research. Princeton NJ: Princeton University Press. Lieberman, Evan S. (2005). Nested analysis as a mixed-method strategy for comparative research. American Political Science Review, 99(3), 435–452. https://doi. org/10.1017/S0003055405051762. Lyall, Jason. (2015). Process tracing, causal inference, and civil war. In Andrew Bennett & Jeffrey T. Checkel (Eds.), Process Tracing: From Metaphor to Analytic Tool. Vol. Strategies for Social Inquiry. Cambridge: Cambridge University Press. Machamer, Peter, Darden, Lindley, & Craver, Carl F. (2000). Thinking about mechanisms. Philosophy of Science, 67(1), 1–25. https://doi.org/10.1086/392759. Mackie, J. L. (1974 [1980]). The Cement of the Universe: A Study of Causation. 2nd ed. Oxford: Clarendon. Mahoney, James. (2004). Comparative – historical methodology. Annual Review of Sociology, 30, 81–101. Mahoney, James. (2010). After KKV: The new methodology of qualitative research. World Politics, 62, 120–147. https://doi.org/10.1017/s0043887109990220. Mahoney, James. (2015). Process tracing and historical explanation. Security Studies, 24, 200–218. https://doi.org/10.1080/09636412.2015.1036610.

238

Christopher Clarke

Manski, Charles F. (2007). Identification for Prediction and Decision. Cambridge, MA: Harvard University Press. Marx, Axel, Rihoux, Benot, & Ragin, Charles. (2013). The origins, development and applications of qualitative comparative analysis (Qca): The first 25 years. European Political Science Review. 6: 115–142 McAdam, Doug, Tarrow, Sidney,  & Tilly, Charles. (2008). Methods for measuring mechanisms of contention. Qualitative Sociology, 31, 307. https://doi.org/10.1007/ s11133-008-9100-6. McKim, Vaughan, & Turner, Stephen (Eds.). (1996). Causality in Crisis? Statistical Methods and the Search for Causal Knowledge in the Social Sciences. Notre Dame, IN: University of Notre Dame Press. Moneta, Alessio, & Russo, Federica. (2014). Causal models and evidential pluralism in econometrics. Journal of Economic Methodology, 21(1), 54–76. https://doi.org/ 10.1080/1350178X.2014.886473. Moran-Ellis, Jo, Alexander, Victoria D., Cronin, Ann, Dickinson, Mary, Fielding, Jane, Sleney, Judith, & Thomas, Hilary. (2006). Triangulation and integration: Processes, claims and implications. Qualitative Research, 6(1), 45–59. https://doi. org/10.1177/1468794106058870. Morgan, Stephen L.,  & Winship, Christopher. (2007). Counterfactuals and Causal Inference: Methods and Principles for Social Research. New York: Cambridge University Press. Morse, Janice. (2010). Procedures and practice of mixed method design: Maintaining control, rigor, and complexity. In Sage Handbook of Mixed Methods in Social and Behavioral Research Teller Road (pp. 339–352, 2455). Thousand Oaks, CA: SAGE Publications, Inc. https://doi.org/10.4135/9781506335193.n14. Morton, Rebecca B.,  & Williams, Kenneth C. (2010). Experimental Political Science and the Study of Causality: From Nature to the Lab. Cambridge: Cambridge University Press. https://doi.org/10.1017/CBO9780511762888. Paul, L. A., & Hall, Ned. (2013). Causation: A User’s Guide. Oxford: Oxford University Press. Ragin, Charles C. (1987). The Comparative Method: Moving Beyond Qualitative and Quantitative Strategies. 1 paperback print, Nachdr. Berkeley, CA: University of California Press. Rohlfing, Ingo. (2012). Case Studies and Causal Inference. London: Palgrave Macmillan. https://doi.org/10.1057/9781137271327. Rohlfing, Ingo, & Isabel Zuber, Christina. (2019). Check your truth conditions! Clarifying the relationship between theories of causation and social science methods for causal inference. Sociological Methods and Research. https://doi.org/ 10.1177/0049124119826156. Romer, Paul. (forthcoming). The trouble with macroeconomics. The American Economist. Runhardt, Rosa W. (2015). Evidence for causal mechanisms in social science: Recommendations from Woodward’s manipulability theory of causation. Philosophy of Science, 82, 1296–1307. https://doi.org/10.1086/683679. Schimmelfennig, Frank. (2015). Efficient process tracing: Analyzing the causal mechanisms of European integration. In Andrew Bennett & Jeffrey T. Checkel (Eds.), Process Tracing: From Metaphor to Analytic Tool. Vol. Strategies for Social Inquiry. Cambridge, UK: Cambridge University Press.

Mixed Methods and Causal Ontology

239

Shan, Yafeng, & Williamson, Jon. (2021). Applying evidential pluralism to the social sciences. European Journal for Philosophy of Science, 11(4), 96. https://doi. org/10.1007/s13194-021-00415-z. Tarrow, Sidney. (1995). Bridging the quantitative-qualitative divide in political science. American Political Science Review, 89, 471–474. https://doi.org/10.2307/2082444. Tashakkori, Abbas, Brown, Lisa M., & Borghese, Peter. (2010). Integrated methods for studying a systemic conceptualization of stress and coping. In Toward a Broader Understanding of Stress and Coping: Mixed Methods Approaches (pp.  31–57). Charlotte, NC: IAP Information Age Publishing. Tashakkori, Abbas, & Teddlie, Charles. (2010). Sage Handbook of Mixed Methods in Social and Behavioral Research. Thousand Oaks, CA: SAGE. Teddlie, Charles, & Tashakkori, Abbas. (2006). A general typology of research designs featuring mixed methods. Research in the Schools, 13, 12–28. Waldner, David. (2012). Process tracing and causal mechanisms. In Harold Kincaid (Ed.), The Oxford Handbook of Philosophy of Social Science (pp. 65–84). Oxford: Oxford University Press. Waldner, David. (2015). What makes process tracing good? Causal mechanisms, causal inference, and the completeness standard in comparative politics. In Andrew Bennett  & Jeffrey T. Checkel (Eds.), Process Tracing: From Metaphor to Analytic Tool. Vol. Strategies for Social Inquiry. Cambridge, UK: Cambridge University Press. Weller, Nicholas, & Barnes, Jeb. (2014). Finding Pathways: Mixed-Method Research for Studying Causal Mechanisms. Cambridge: Cambridge University Press. Wolf, Frieder. (2010). Enlightened eclecticism or hazardous hotchpotch? Mixed methods and triangulation strategies in comparative public policy research. Journal of Mixed Methods Research, 4(2), 144–167. https://doi.org/10.1177/1558689810364987. Wood, Elisabeth Jean. (2003). Insurgent Collective Action and Civil War in El Salvador. Cambridge: Cambridge University Press. Wooldridge, Jeffrey M. (2016). Introductory Econometrics: A Modern Approach (6th ed.). Boston MA: Cengage Learning.

12 EVIDENTIAL PARTNERSHIPS AND MULTI-METHOD RESEARCH IN POLITICAL SCIENCE Methodological, Evidential, and Causal Pluralisms Sharon Crasnow

12.1

Introduction

Although quantitative, formal, and experimental methods still dominant political science research, there has been a burgeoning interest in multi-method research (MMR). Recent methodological discussions often consider how these mostly quantitative methods might be enhanced through qualitative work. In this chapter, I argue that various forms of multi-method research should be seen as evidential partnerships (Crasnow et  al., forthcoming). The value of such partnerships is best understood by thinking about them in relation to three kinds of pluralism – methodological, evidential, and causal. Although I focus on political science since it is the social science with which I am most familiar, the analysis that I offer and the claims that I make about these various pluralisms and how they bear on MMR should be applicable to the social sciences more generally. Much of the literature on MMR focuses specifically on the way different methods might provide a variety of evidence in support of causal claims in the social sciences. The value of evidential diversity is sometimes thought to result merely from the availability of more evidence providing better support for causal claims. Another argument is that the evidence from one method can compensate for weaknesses of another method. Recently Shan and Williamson (2021) have attempted a more explicit account of the strength of MMR, by extending the claims Russo and Williamson make in their 2007 article on integrating evidence of associations and evidence of mechanisms in the biomedical sciences. The Shan and Williamson account promotes an evidential pluralism in which quantitative methods are described as typically

DOI: 10.4324/9781003273288-14

Evidential Partnerships and Multi-Method Research

241

producing evidence of correlation or association, while qualitative methods might be employed to produce evidence of mechanisms. As they state it: Evidential Pluralism. In order to establish a causal claim one normally needs to establish the existence of an appropriate conditional correlation and the existence of an appropriate mechanism complex, so when assessing a causal claim one ought to consider relevant association studies and mechanistic studies, where available. (Shan & Williamson, 2021, p. 4) While I agree that there are uses of MMR in the social sciences that the Shan–Williamson Evidential Pluralism (EPsw) correctly describes, in this chapter I argue that there are other roles that MMR plays. Shan and Williamson do not deny this and so we are in agreement here. But I take these roles to be important and worth exploring and offer an account of what some other roles might be. While there is some dispute about terminology, MMR is usually understood as research that uses both quantitative and qualitative methods within a single research project.1 I  contrast this with an alternative description – ‘evidential partnerships’ – a term used by Crasnow et al. (forthcoming). ‘Evidential partnerships’ is intended to describe research that makes use of any combination of methodological approaches that enhance the search for and the production of evidence for causal claims. I contrast this with Shan and Williamson’s extension of Evidential Pluralism to the social sciences, which I understand as describing one very specific type of evidential partnership. EPsw is focused on the production of different types of evidence in support of the same causal claim. In addition to this role, evidential partnerships include the use of multiple methods to support many other aspects of research that aid in the production of such evidence.2 These partnerships result in interactions between and among various methodological approaches that contribute more to knowledge production than is captured in many accounts of MMR research. A survey of the political science literature reveals some examples of evidential partnerships (although this list is not exhaustive): case studies and regression discontinuity design; natural experiments and case studies; field experiments and ethnography; game theory and statistical analysis; game theory and experiments. In summary, evidential partnerships may involve MMR research – the use of both quantitative and qualitative methods – but the combining of methodological approaches involves more than simply combining methods. In order to make my case, I analyse two examples of evidential partnerships in Sections  12.2 and 12.3. I  then use insights from these analyses to

242

Sharon Crasnow

introduce methodological pluralism as I understand it in Section 12.4. Section 12.5 considers evidential pluralism. I discuss the Shan and Williamson version for social science (EPSW) and offer my own broader characterisation of evidential pluralism (EPB) informed by the methodological pluralism advocated in the previous section. I conclude in Section 12.7 with an exploration of the implications of these views for debates about causal pluralism versus causal monism. 12.2

Process-Tracing: Wood and the Civil War in El Salvador

Process-tracing is a research design used to produce evidence for (or against) a particular causal mechanism operating in that case. The key idea behind process-tracing is that through seeking the key elements of the hypothesised causal mechanism, it should be possible to identify whether the mechanism is or is not operating. Process tracing is generally thought of as qualitative research; however, this is somewhat misleading, since the traces that are sought may include evidence produced through both quantitative and qualitative methods. Case studies themselves – the loci of process-tracing – also involve the accumulation of evidence produced through both qualitative and quantitative methods. The point is that process-tracing can incorporate MMR while at the same time appearing in a partnership with some other methodological approach. For example, process-tracing is often used in conjunction with specifically quantitative methods such as regression analysis. Such evidential partnerships do use MMR in a variety of ways but there are other benefits that are not well captured if we focus on them solely as MMR. To see why, let’s look more closely at process-tracing. While a number of political methodologists have explored process-tracing (George & Bennett, 2005; Bennett & Checkel, 2014), in what follows I rely primarily on Beach and Pedersen (2019). Process-tracing is a research method for tracing causal mechanisms using detailed, within-case empirical analysis of how a causal mechanism operated in real-world cases. (Beach & Pedersen, 2019, p. 1) Beach and Pedersen describe four variants of process-tracing, identified through their differences in research goals. They are theory-testing, theorybuilding, theoretical-revision, and explaining outcome (see Figure 12.1 and Table 12.1). In practice, process-tracing within a case study may engage in all of these within the same research project and so while they may be conceptually distinct they are often intertwined.

Evidential Partnerships and Multi-Method Research

243

TABLE 12.1 Four Variants of Process-Tracing

Research purpose

Analytical focus

Theory-Testing Process-Tracing

Theory-Building Process-Tracing

TheoreticalRevision Process-Tracing

ExplainingOutcome Process-Tracing

Is hypothesized causal mechanism present and does it function as theorized? Theory-focused

What is the causal mechanism between the cause and outcome? Theory-focused

Why did the mechanism break down in the case?

What mechanistic explanation accounts for the historical outcome? Case-focused

Theory-focused

Source: From Beach & Pedersen (2019, p. 9)

I describe theory-testing process-tracing first. Researchers start with a hypothesised mechanism that guides the search for entities, activities, or events that one ought to observe if that mechanism is operating in the case under investigation. These observations are only to be understood as evidence relative to that hypothesis and the contextual background knowledge. In practice, good process-tracing arguments compare competing hypotheses, showing one to be a better fit for the hypothesised mechanism than the other. The research problem may arise as a result of a case or cases that appear to provide counterexamples to previously proposed theoretical explanations and thus call for theory revision. Revising theory may also lead to theorybuilding or theory-building may result from observations made within the case. Explaining the outcome is often the research goal. I use Elizabeth Wood’s 2003 book-length analysis of the civil war in El Salvador as an example.3 Wood’s research puzzle is the following: Why was there broad participation in the insurgency in El Salvador even though the costs of participation were so high and the rewards so minimal? Most available explanations are based on a background belief that insurgents are motivated by expected gains (or avoiding higher costs). Wood argues that these accounts do not adequately track the pattern of participation in El Salvador. ‘The relevant literature on revolutions, collective action, and social movements provide some guidance but not adequate answers to the puzzle of high-risk collective action in the Salvadoran context’ (Wood, 2003, p. 10). While participation was broad, it was not universal. Only about a third of the poor, rural class (campesinos) participated in the insurgency – a large enough rate of participation to have significant effect but standard hypotheses, such as Marxist accounts of class struggle, do not square with the large number of ‘free riders’. For Wood, it is this specific level of participation that is puzzling. The high level of risk and

244

Sharon Crasnow

the minimal reward would seem to predict a lower rate of participation than what actually occurred. Other theoretical accounts predict a higher rate of participation. The rate of participation is the quantitative evidence that does not comport with extant hypotheses and calls for theory-revision and ultimately theory-building. To answer her research question, Wood relies primarily on qualitative evidence, investigating differences between those who participated (approximately one-third of the rural poor) and those who did not. Her primary evidence is many extensive interviews (more than 200) over multiple years both during and after the civil war. These interviews yielded recurring themes: injustice of pre-war land distribution; desire for land; the contempt with which the respondents had been treated; brutality of the government responses to non-violent strikes and demonstrations; fear during the war; suffering of their families; post-war assertion of political and social equality; authorship of changes; pride in participation (Wood, 2003, p. 18). The correlation between participation and types of responses not only suggests that there is a mechanism operating but also, because of the nature of the responses, what those mechanisms are. Putting information from interviews together with details about the timing of specific events during the civil war – when they happened in relationship to other events, and how they are correlated with the varying levels of participation at different periods of the war – Wood proposes three psychological mechanisms responsible for collective action in El Salvador (theory building). She calls the first ‘participation’ a term that she defines more precisely than mere involvement in the insurgency. Participation in this sense is the desire to be involved in activities that reflect moral commitment. She identifies the influence of Liberation Theology on campesinos as fuelling this motivation. The second is ‘defiance’. The government response to strikes was believed by many campesinos to be an overreaction to legitimate means of protest (strikes and peaceful demonstrations) for unfair working conditions. This perception fuelled and justified defiance as a motivator of collective action. The third is what she calls ‘pleasure in agency’. Participants reported a pride and sense of authorship in having been involved in making history. Wood traces the operation of each of these hypothesised mechanisms through patterns of responses to interview questions, together with specific documented instances of collective action and government response that occurred through the decade of civil war. Her case study provides an account of the civil war through these and other mechanisms, incorporating particular events and shifts in strategy in response to such events. For example, she documents increased repression on the part of the regime and the subsequent shift from political mobilisation to armed insurgency. The process-tracing argument of the book is that the narrative of the case told through the psychological (emotional and moral) mechanisms that Wood identifies provides

Evidential Partnerships and Multi-Method Research

245

a better account than alternatives.4 The many different kinds of evidence – the traces – taken together fit well with her hypothesised mechanism but not with the alternatives. While I only sketch Wood’s argument here, it is worth noting several ways in which process-tracing is operating. First, there is the tracing of evidence that the psychological mechanisms that she postulates were operating. She notes these both as they are present in interviews with those who were participants in the insurgency and their absence in the interviews of those who were not (qualitative evidence of association). Additionally, Wood’s respondents appear to appeal to these mechanisms as motivations for their actions, revealing causal connections (qualitative ethnographic evidence for psychological mechanisms). While Wood’s primary goal is to explain the case through the operation of these mechanisms (to explain a singular case), she also uses the case to develop more general hypotheses (theory building). We can see this particularly well in her development of the notion of pleasure in agency which results from interpretation of a number of themes in interviews. She is also engaging in theory revision since she does not entirely reject material (Marxist) explanations in the literature on insurgency and collective action but does find them inadequate. A final point that is worth noting is that she makes use of background knowledge against which she both identifies the puzzle that she wants to address and through which she recognises as evidence particular events that occur in her case – the ‘traces’ of process-tracing. For example, the escalation of insurgent activity to violent government repression following strikes and peaceful protests in the early 1980s is evidence for her hypothesised mechanisms (together with the interview information), since escalation of resistance is one of the known responses to repression. She also argues for the relevance of the interview evidence through appealing to psychological research on memory. Pulling all of this together, Wood develops a coherent narrative of the civil war in El Salvador that solves the puzzle that she originally saw in the case. Wood’s process is MMR. She employs both quantitative and qualitative methods to produce evidence. While the process-tracing here does provide evidence of mechanisms, she is also using evidence of correlation to support her hypothesis – the correlations she notes follow from theory and her specific hypothesised mechanisms – and they appear in the case. This aspect of her analysis looks like the sort of MMR that Shan and Williamson (2021) aim at providing justification for. However, a description of what is going on through their account alone misses a number of ways that MMR contributes to knowledge production in this evidential partnership. First, the role of existing theory and background knowledge is crucial for identifying the research problem. There is a need for explanation in this case because the particular nature of the events did not appear to be consistent

246

Sharon Crasnow

with prevailing theory. In addition, what counts as evidence for the hypotheses under consideration depends on other things that are known. The acceleration of resistance that occurred after repressive responses by government is an expected outcome given background knowledge about insurgency. Identifying defiance as a psychological mechanism linked to greater participation in the insurgency after such events through interviews allows the interview evidence to be seen as relevant to answering the research question. The epistemic context in which the questions are framed and the research carried out are crucial features of knowledge production, although they do not themselves constitute evidence and consequently are neither evidence of correlation nor evidence of mechanisms. But background knowledge is necessary for indicating that these events count as evidence that the hypothesised psychological mechanism is operating. Second, Wood’s account is presented in the form of a narrative. EPSW focuses on content – it is a claim about what we need evidence of to establish causal claims. Narrative is typically thought of as a conventional way or convenient way of presenting case evidence but is not thought of as doing inferential work itself. But Noël Carroll argues that the causal nature of narrative connection is part of what distinguishes narrative from a temporal list of events. For Carroll, the construction of a narrative involves ‘piling up more and more causally necessary conditions’ (Carroll, 2001, p. 130). Although the outcome is underdetermined by these conditions, what subsequent events are possible is narrowed as the narrative is developed. Thus causal connections move the narrative forward to its conclusion and eliminates alternatives. The narrative explanation that results is convincing because it tracks causal networks. The coherence of the narrative – the way it pulls the parts of the story into a whole through the use of the causal mechanism Wood proposes – serves as a kind of evidence for the plausibility of those mechanisms, as I have argued elsewhere (Crasnow, 2017), It is the success of the narrative as a coherent story that provides evidence that the story gets causality right. Concerns about ‘just so’ stories can be addressed through a variety of theoretical and empirical constraints, such as sustaining or testing other causal paths or within-case counterfactuals. A good narrative shows not only how and why what happened did happen but also why something else didn’t happen (Beatty, 2016). Third, the iterative character of the research illustrated in the Wood example shows interaction between the methods at different phases of research. The research problem comes out of a puzzle, which in turn depends on prior theoretical accounts of collective action and insurgency. Wood’s interviews suggest alternatives, but those alternatives require going back to the case and examining it against the plausibility of these new hypotheses. Consideration of these hypotheses calls for reexamination of the interviews – and in doing so, features that might not have been recognised previously as evidence become evidence.

Evidential Partnerships and Multi-Method Research

247

All of these suggest that while MMR often does produce evidence of association and evidence of mechanism, in the context of an evidential partnership, it does more than this. It also aids in formulating the research question, provides means of incorporating background knowledge, aids in producing indirect evidence by revealing information that indicates what counts as evidence for the hypothesis under consideration. In summary, partnerships shape ways of thinking about research questions, inform research design, and aid in theory development. 12.3

Natural Experiments and Case-Study Research

The second evidential partnership that I examine occurs in a natural experiment research design. Natural experiments in the social sciences are not ‘true’ experiments but observational studies (case studies) of a particular sort. They involve a reconstruction of events or circumstances that allows them to be treated as a control group and an experimental group for comparison. Differences in outcomes between the two groups can then be understood as providing associational evidence for causal claims. Unlike quasi-experiments in which the experimenter assigns individuals to the control group or experimental group, a natural experiment is ‘found’, in the sense that the researcher recognises a circumstance in which such a division has been made by nature or in some other way that is not in the researcher’s control. Four kinds of circumstances that offer the possibility of such reconstructions are massive interventions that make issues of control irrelevant; situations of total isolation; unusual events that occur within a stable environment; or situations in which both the environment and the intervention are carefully controlled through natural or social circumstances (Morgan, 2013, pp. 345–346). Natural experiments are only good insofar as they resemble actual experiments and so any natural experiment research design will have to include arguments supporting the legitimacy of the reconstruction. In other words, the evaluation of the natural experiment – a determination of how good an epistemic tool it is – is a function of how strong the argument for the analogy to a true experiment is. In his analysis of natural experiments, Thad Dunning describes the evaluation of the natural experiment across three dimensions: (a) the claim that the assignment to groups is random or as-if random, (b) the credibility of the statistical and causal models, and (c) the relevance of the intervention to the causal claim (Dunning, 2012). The inferential strength of the natural experiment thus rests on the strength of the arguments that the natural experiment does well across these dimensions. Consequently, the evidence for the causal claim supported by the natural experiment includes evidence that it is indeed a natural experiment. The following example illustrates this point. In her 2007 World Politics article, ‘The Observer Effect in International Politics: Evidence from a Natural Experiment’, Susan D. Hyde offers a natural

248

Sharon Crasnow

experiment as evidence that international election observers reduced voting fraud in Armenia in the 2003 election.5 She chooses post-Soviet Armenia, because it is a case that she knows well having previously studied it. Her case knowledge allows her to make the argument that she can reconstruct the circumstances as a natural experiment. Hyde notes that the 2003 elections were widely considered a turning point in Armenian democracy – a point at which voter fraud was first seriously addressed. The Armenian Ministry of Foreign Affairs invited the Organization for Security and Co-operation in Europe/Office for Democratic Institutions and Human Rights (OSCE/ODIHR) to send international observers for the first time. Interpreting the introduction of observers as ‘an unusual event occurring against a stable background’ – one in which election fraud has been the norm – also supports treating this case as a natural experiment. Polling stations with observers contrasted with polling stations without observers suggesting the opportunity for a natural experiment where stations with observers serve as the experimental group and those without observers as the control group. To establish the circumstances as a natural experiment requires (among other things) providing evidence that the observers were randomly assigned (or as-if randomly assigned) to polling stations. Using evidence produced through both qualitative and quantitative methods, Hyde argues that the assignment was as-if random. She conducts interviews with members of the OSCE/ODIHR through which she determines the criteria used for assignment of observers and other relevant factors (qualitative). She also conducts randomisation checks, first examining the population characteristics of the polling stations and arguing that the distribution of observers was widespread throughout the population, relatively evenly distributed in urban and non-urban areas and relative to other potential confounders, giving no indication of being correlated with bias. To do so, she used data produced through quantitative methods (to establish the proportional distribution) and qualitative methods (to determine the whereabouts of the observers). In these arguments, both qualitative and quantitative methods are used to provide evidence of correlation but no evidence of mechanism is offered. The natural experiment design identifies the independent variable – the difference-maker (the international observers) – but not the mechanism through which the difference is made and consequently the evidence Hyde offers for the causal claim she ultimately makes – that international observers reduced election fraud – does not include evidence of mechanism. Thus we have an example of an MMR that does not conform to the pattern that Shan and Williamson identify.6 An important feature of Hyde’s argument is what she takes as a measure of voter fraud. She argues that given that only the executive office was organised enough and had the requisite power to perpetrate election fraud, a significant

Evidential Partnerships and Multi-Method Research

249

difference in votes for the incumbent in the control group (unobserved polling stations) versus the experimental group (observed polling stations) is an indicator of fraud. Where there is less fraud the percentage of votes for the president will be lower than it will be in polling places where there is substantial fraud. Here she is making use of a mechanism that she takes to already be established. It is a mechanism that is specific to this situation however. She uses it to construct a measure of fraud. That measure is based on both her disciplinary knowledge of how voter fraud occurs and her case knowledge of the Armenian context. There are two lessons that I want to take away from this example of MMR. First, it is an example where the causal claim is supported solely by evidence of correlation. Hyde acknowledges this perhaps suggesting that evidence to support a mechanism claim would be desirable. In the case of Armenia, election-day fraud deterrence meant that the international observers slowed down the rate of fraud in the polling stations that they visited. The precise mechanisms by which this took place are unknown, but a plausible explanation is that the presence of international observers caused polling-station officials to reduce the rate of intended election-day fraud, either because they were instructed to stop fraudulent activities in front of international observers or because many were worried about being caught. (Hyde, 2007, p. 61) Second, just as in the first example, the various ways in which methods are mixed in this research seem not to be captured by EPSW. Hyde uses both theory and background knowledge in her construction of the measure of fraud, for example. A  high percentage of votes for the incumbent is evidence of fraud. She uses case knowledge – Hyde’s knowledge of the mechanisms of power and elections in Armenia. This measure is evidence for her conclusion – observers are correlated with a lower rate of fraud – but not direct evidence. However, the research design that produces the evidence for the claim depends on this measure and so it might be thought of as indirect evidence or part of the evidential base of her argument. There are also elements of the research in which knowledge is produced that is used elsewhere in the research design. Qualitative interviews support that observers were assigned as-if randomly – something that she needs to establish in order to use quantitative means to detect differences in voting. This also might be thought of as indirect evidence for the causal claim that she ultimately makes. This analysis highlights the many crucial features of knowledge production that do not produce evidence directly related to the causal conclusions supported by the research. Again, this is not an argument against the usefulness

250

Sharon Crasnow

of EPSW as a justification for some uses of MMR, but the motivations and justification for evidential partnerships that include MMR go far beyond those offered through that thesis – which brings me to methodological pluralism. 12.4

Methodological Pluralism

Although ‘method’ and ‘methodology’ are often used interchangeably, I will distinguish them. Methods are the techniques used to produce evidence, research designs, and means of interpreting results of research, whereas methodology is a normative stance on how knowledge ought to be produced. On this characterisation, methodology includes ideas about what techniques (methods) are appropriate when gathering, analysing, or interpreting evidence, but methodology need not dictate method. Methodology includes views about how best to conduct research, what sorts of research questions are appropriate, when research should be considered complete, what counts as a result, and how such results should be reported.7 Although ethnography, for instance, is often described as a method, I take it to be a methodology on this understanding. The knowledge target of ethnography (and thus the appropriate research questions) is understanding different social, cultural, and political relations among humans from their perspective. A wide variety of methods might be used to do so; however, methods have to be able to serve this goal. Open-ended interviews, focus groups, and participant observation are examples of methods that are suited to ethnographic research questions. There are clearly other sorts of questions that cannot be answered very well (or at all) using them – questions about the distribution of particular characteristics in a large population for example – just as ethnographic questions cannot be answered very well through observational statistical methods. Methodology does not dictate method but method must fit the goals of the methodology. Methodological pluralism certainly allows for MMR but is not equivalent to it. Whereas MMR is the use of quantitative and qualitative methods in the same research project, methodological pluralism calls for a diversity of approaches within a discipline. In this sense, it involves rejecting the idea that there is one right method for achieving the goals of research in a discipline or even that there is one best method – even if that is MMR. This is in part because methodological pluralism acknowledges a variety of goals. MMR is certainly compatible with methodological pluralism, given the commitment to multiple approaches, but methodological pluralism would also support single method research projects, since it is a commitment to a multiplicity of approaches within and across disciplines and leaves open the possibility that given a particular goal, evidence of correlation alone or evidence of mechanism alone might be adequate to establish a belief in a causal claim. In this sense, it may challenge the normative claim of EPSW in two ways: first,

Evidential Partnerships and Multi-Method Research

251

in allowing that evidence of association and evidence of mechanism do not exhaust how we should think about evidence; and second, in allowing that research in the social sciences need not always be MMR. For the first, even if all direct evidence is evidence of association or mechanism, indirect evidence, information that establishes the relevance of particular entities, events, and activities, is a part of the evidential context and often produced through MMR. But there is reason to think that it is not the case that all direct evidence is evidence of association or mechanism. For example, much of what Hyde knows about the Armenian case serves as evidence that her research design – a natural experiment – is indeed a natural experiment. Case study evidence may also serve as evidence that one case is like another and so can serve to support a generalisation about cases of this type by supporting that they are of some type. Good research may not always be MMR either. Political science currently is methodologically pluralistic even though identifying and supporting causal claims dominates as a goal. This emphasis has led to a preference for methods that are focused on evidence of correlation – observational statistical and, more recently, experimental methods – all classed as association studies by Shan and Williamson. The recent interest in MMR in political science has been at least partially motivated by those who have alternative aims (understanding, for example), many of whom favour qualitative methods. But the move to MMR has also resulted from a recognition of the benefits of contextual knowledge for research design and other ways in which interactions among methodological approaches aid knowledge production. Consequently advocating MMR only goes part way towards motivating methodological pluralism. My motivations for advocating methodological pluralism bear some similarity to those that Hasok Chang cites when arguing for scientific pluralism: toleration and interaction (Chang, 2012). Toleration of many approaches allows for the possibility of a broader range of research questions to be addressed, which is more likely to produce a wider variety of evidence than if only one approach is taken, although it has other benefits as well. Interaction allows for the possibility of cross-fertilisation of the sort that at least some advocates of MMR have argued for. It is evidential partnerships that foster this cross fertilisation beyond the combining of evidence for causal claims that it is the focus of MMR. Encouraging interaction among methodologies in the ways that occur in evidential partnerships supports a variety of the benefits. One is the value of seeking for what might be called ‘indirect’ evidence. Indirect evidence includes background knowledge, alternative theoretical frameworks that can aid in recognising hypotheses that potentially need to be eliminated, uncovering confounds, and alternative pathways that might produce the same outcome. These are some of the ways that evidential partnerships appear to be

252

Sharon Crasnow

operating in the examples from Sections 12.2 and 12.3. Of equal importance is the recognition that there are different worthwhile goals to pursue within the discipline and that knowledge produced in the pursuit of one goal may aid indirectly in achieving other goals. R. Burke Johnson advocates a view that he calls ‘dialectical pluralism’ that has similar motivations (Johnson, 2017). He offers a variety of potentially mutually beneficial combinations, and argues normatively for what he describes as equal partnerships. While I agree that there are benefits to the equality that he advocates, methodological pluralism as I  understand it is also compatible with MMR in which one method dominates or with single method research projects. Methodological pluralism advocates for multiple methodologies within a discipline. Again, this differs from promoting MMR. As I read Shan and Williamson’s Evidential Pluralism, it is focused on individual research projects rather than pluralism within a discipline and it is focused on one goal of that pluralism – the establishment (and assessment) of causal claims.8 I think of this goal as informed by a ‘detachability ideal’ – an ideal that is prevalent throughout the literature on causal inference in political science (and elsewhere). This ideal takes as an aim producing evidence that allows for the causal claim to be detached from that evidence (and the context) in much the way a rule of detachment in logic allows the detachment of the conclusion from the premises in a sound argument. Although no one makes the explicit claim that the evidential relation constitutes a deductive argument, I  contend that the ideal nonetheless haunts thinking about how evidence for causality works. Under the detachability ideal, the goal of research is to come up with a set of causal claims that once warranted can be separated from the context in which they received that warrant. This is the thesis behind the notion of randomised controlled trials (RCTs) as a gold standard. A  claimed virtue of RCTs – what makes them the gold standard – is the idea that they allow researchers to identify and isolate causes. Thinking of causal inference in this way emphasises a ‘moment of inference’ – a point at which the evidence licenses the causal conclusion and renders it detachable – and thus suppresses the relevance of background knowledge (beliefs and assumptions that reveal the relevance of particular information as evidence) and aims (including practical implications) – the elements of context. The ideal is intertwined with an understanding of knowledge as universal, general, and perhaps even lawlike. The transportability of the causal claim – where else it might be applicable – is thought to depend on these characteristics. An alternative view calls for continued attention to context and argues for transportability to depend on detailed comparisons of contextual features. The broader evidential pluralism (EPB) that I advocate promotes attention to such details.

Evidential Partnerships and Multi-Method Research

12.5

253

Evidential Pluralism

As I  have noted, the view that I  identified as evidential pluralism (EPB) is broader than EPsw. It is more akin to what Nancy Cartwright calls ‘evidential diversity’ (Cartwright, 2021) or what Shan and Williamson have dubbed ‘evidential contextualism’ (2022).9 Although both Russo and Williamson (2007) and the extension offered by Shan and Williamson allow that researchers do a number of other things besides coming up with evidence for causal claims, the examples I  have offered highlight how much of what makes evidence relevant – that is, reveals it to be evidence for either association or mechanism – depends on evidential partnerships. The goals of research are various and go beyond the aim of providing direct evidence for causal claims. Consequently, the categories of evidence are not exhausted by evidence of mechanisms and evidence of associations. There are several reasons that we need a broader conception of evidence and evidential pluralism than that offered by Shan and Williamson. The first has to do with what I referred to in the last section as indirect evidence for both drawing and making use of causal conclusions. It is quite usual for political science researcher to use MMR for a variety of reasons other than as a means to directly produce evidence of either association or mechanism. For example, there is a considerable body of research that goes into making decisions on how to classify or measure abstract (latent) concepts that are crucial for social science. Anna Alexandrova (2017) looks at how well-being is understood and measured. I  have examined various ways of measuring democracy (Crasnow, 2021). How such concepts are understood shapes what sort of empirical phenomena are relevant when researchers argue for causal claims. Concept development and amelioration often come about through the interaction of methodological approaches: for instance, when it is clear that data doesn’t reflect experienced reality. Process-tracing within case studies is sometimes cited as a way of producing evidence of mechanisms, but case-study research has other uses as in the example of Hyde’s work where she uses case knowledge to argue that her research design qualifies as a natural experiment. Such research fills in details of context which support that the analysis can be used as evidence of association to support a causal claim and without which inferences to faulty causal conclusions are likely to occur (Faletti & Lynch, 2009).10 The second reason is that sometimes we might want a finer parsing of types of evidence for causal claims. The evidence for different types of mechanisms may be significantly different in ways that require different methodological approaches. The types of mechanisms appealed to in the social sciences vary widely in ways that are linked to different ontologies. Mechanisms are constituted by entities, events, and activities, but what those entities, events, activities are may vary depending on the methodological approach taken. For

254

Sharon Crasnow

example, in political science, they could be beliefs, desires, and actions of individuals in the political arena or they might instead be institutions, political roles, and policies. Different approaches to explanation might appeal to different sorts of mechanisms in ways that depend on theoretical background. Shan and Williamson’s Evidential Pluralism only specifies evidence of mechanisms (of some sort) in addition to evidence of correlation, but notice that causal claims that are established about one sort of mechanism will not be easily detached and transported to address concerns about another. These are ontological debates and so beyond the scope of EPSW of course but this issue raises questions about how to address such mismatches when they occur. I explore this issue further in the next section when I discuss causal pluralism. Another concern about mechanisms has to do with what Gary Goertz calls ‘constraint causal mechanisms’ (Goertz, 2017). Constraint mechanisms can play a role in explaining why something does not occur – for example, why Britain and France did not go to war over control of the Upper Nile Valley region of Northern Africa in 1898 (the Fashoda Incident). It isn’t clear that EPSW captures these differences, since it parses all evidence as either evidence of mechanism or evidence of correlation. Third, EPSW does not consider whether there are differences in the evidence needed to support and assess causal claims relative to the aims of research. Arguably under particular circumstances, evidence of association is sufficient to act – consider for example, circumstances under which we do not have evidence of a mechanism but we know that a particular treatment works. This point might make a difference to whether the goal is policy guidance or basic research. Shan and Williamson argue that their version of Evidential Pluralism applies for both, but in the case of policy research an argument might be made that how much and what kind of evidence depends very much on what the issue is – I am thinking of Heather Douglas’s arguments about inductive risk (Douglas, 2009). Depending on the risk of a false negative versus a false positive, decisions about how much and what kind of evidence is needed for a causal claim may vary. In the case of basic research, the sort of variety of interactions that can take place among evidence, theory, background knowledge, and other research described earlier need to be considered. While Shan and Williamson may have responses to these concerns, each brings up other issues that are important. If the focus is solely on how one establishes a causal claim, these concerns may not be given their due. Exploring evidential partnerships more generally helps to see what a broader conception of evidential pluralism might offer but evidential partnerships are not equivalent to EPB. EPB focuses on varieties of evidence that might be used either indirectly or directly to support causal claims. Evidential partnerships certainly include such pluralism but may have other benefits as well. Doing a case study may suggest hypotheses that can then be tested

Evidential Partnerships and Multi-Method Research

255

through observational statistical or experimental means. Case studies may suggest more fine-grained partitioning of reference classes for statistical work or for experimental design. Evidential pluralism is one benefit of evidential partnerships but not the only one. 12.6

Causal Pluralism

There is another issue that is closely related to evidential pluralism in whatever form (whether EPSW or the EPB that I am advocating). If there are varieties of evidence for causal claims might there also be varieties of causal claims? If we are to be pluralists about evidence should we also be pluralists about causality? Russo and Williamson (2007) explicitly deny metaphysical commitments to any particular philosophical analysis of causality or to causal monism or pluralism. Their proposal addresses the epistemic question of what evidence is needed to establish and assess a causal claim but says nothing about the nature of the cause that claim is about. Shan and Williamson’s extension to the social sciences is consistent with this view. To address this question, I start by asking another. Why is it that both sorts of evidence might be required to establish and assess a causal claim? What is it that each sort of evidence is contributing? One way to answer this question is that the need for two objects of evidence is related to what we do with causal claims. We use them to explain what has happened, predict what will happen, and to gain some measure of control over what will happen through the manipulation of causal factors. Perhaps, the best way to think about evidence for such claims is to note that, if a causal claim is to serve all these functions, it needs to be supported by evidence relevant to each of them. Evidence of mechanism supports causal claims being used for explanations. Evidence of association does not aid in answering why or how something happened, but only that when it happens it is (always or usually) proceeded by something else (the cause). Evidence of association does, however, support prediction. Both sorts of evidence appear to be implicated in our desire to control the world around us. Illari (2011) and Williamson (2013) both suggest something like this might be how we should think about it. However, this analysis takes as given that a causal claim should do all of these – that an established causal claim should be adequate to each of these tasks. Because of this, although EPSW is intended to be neutral with respect to the metaphysical question it seems again to be committed to some version of causal monism on this interpretation. The claim is one claim about one sort of cause with a variety of characteristics. However, once we consider the possibility that we can think about evidence as related to the role that causal claims play in our systems of belief, then there seems no reason to prefer causal monism to causal pluralism. The view that there are different sorts of causes – claims about one supported by

256

Sharon Crasnow

mechanistic evidence and claims about the other supported by associational evidence – seems no more strongly supported than the claim that there is one sort of thing that causation is and that claims about it are supported by both mechanistic and associational evidence. The view that causes are one sort of thing with multiple characteristics evidenced in different ways and supporting their various functions and the view that there are various sorts of causes each requiring different sorts of evidence and suited to different functions are two competing and internally consistent views. As I see it, the question of which we should adopt depends first on which way of thinking about causation and causal claims is more useful and fruitful, and second on which seems best to reflect the practices of the scientists working in that field. For the first, I offer the following argument, which is, admittedly, not particularly strong. The methodological pluralism that I have advocated supports pursuing a variety of different research projects in an open-ended way. One approach interacting with research pursued through another approach does sometimes produce both evidence of mechanism and evidence of association that can be integrated to support one causal claim, but it is not clear that this is always does – or that it should always. The requirements for such integration are minimally that the concepts used in the research are commensurate – that is to say, that the concepts through which the objects of study are understood are the same or similar enough that evidence from both approaches is relevant to claims about the causal relationships of the objects of study bear to each other.11 Derek Beach has recently argued that, within the social sciences, there may be methodological approaches that are not compatible with others. He offers the example of Critical Realism (Beach, 2021) in which reasoning attributed to agents requires understanding them in ways that may not be commensurate with the approaches other methods take. In addition, how ‘cause’ is understood in the different approaches also has to be compatible. If causal monism is correct, this isn’t a problem. However, Beach suggests that the Critical Realist understanding of causation takes it always to be situational and hence local and so not generalisable. If he is right about this and if Critical Realist approaches do have something to offer then this could be an argument for causal pluralism. Another worry is raised by Goertz and Mahoney (2012). They are concerned that the causal understanding of relations between variables at the population level and the way causality is thought about in single cases may not be compatible. I have suggested that these different approaches are not necessarily incompatible if there is an account of causality that clarifies the relationship between single case causation (actual causation) and general causation (Crasnow, 2012).12 Determining whether these requirements have been met calls for a case-bycase examination. Causal monism would seem to require the assumption that integration is always possible. Causal pluralism leaves open the possibility

Evidential Partnerships and Multi-Method Research

257

that integration may not be possible. However, even if such integration is possible it may not always be necessary or desirable to carry it out, since there are different purposes for which causal knowledge is sought. This is a pragmatic point arising from the observation that there are different functions that causal claims can serve and that it might not be necessary for any one causal claim to serve all (prediction, explanation, and control) in the same context. Indeed, it might be desirable or even necessary to employ different understandings of cause in different contexts. A possible monist response might appeal to economy of concepts. A causal monist might argue that we have no reason to multiply causes and that a simpler, single sense of cause should be preferred. In response, I would note that simplicity should be preferred if in fact what we are trying to capture is a simple phenomenon. I have argued that this is what is up for grabs in the case of causality – what we don’t know. Furthermore, given that we do seek a variety of evidence relevant to causal claims and that we use them for a variety of purposes, there is some reason to consider a pluralist approach. A final worry about monism is one that several causal pluralists have noted. All current philosophical accounts of causality have counterexamples (Cartwright, 2007; Reiss, 2009). As to the question of how scientists working in the field view evidence for causal claims, I  think that there may be important differences between the biomedical sciences for which Russo and Williamson first proposed evidential pluralism and the social sciences to which Shan and Williamson propose their extension. Arguably any individual causal claim in the biomedical sciences is a claim intended to be used for prediction, explanation, and control. While it is the case that sometimes we may have medical treatments for which there is associational evidence but not mechanistic evidence, for example, there is typically a push to establish what the mechanism behind the association is. However, the goals in the social sciences seem more varied. For example, when a political scientist seeks to explain a particular event – something like the Cuban Missile Crisis, – understanding the mechanism through which that event occurred may be adequate. In such a case, explaining is the goal – prediction and control may not be relevant. And in the case of individual events like this, which are rare, it is not clear what should count as associational evidence. This is not to say that there are never cases when prediction and control are sought. Just as in the biomedical sciences, when causal knowledge is used to inform policy, evidence of association may be adequate or it may be the case that evidence of mechanism is sought as well. Neither methodological nor evidential pluralism establishes causal pluralism, but the motivations that I have offered for these views are generally more compatible with causal pluralism than monism. A  causal pluralist stance leaves open the possibility that there may be research programmes where

258

Sharon Crasnow

evidence produced from one cannot be integrated into evidence produced from another either because the concepts are incommensurable or the different understandings of cause cannot be reconciled. This would, of course, have implications for MMR as well. Under such circumstances we should also conclude that there are times when the evidence produced from various methods cannot be pulled together in support on a causal claim – sometimes evidential partnerships may not be feasible. Perhaps, there are very few or even no cases in which this is so, but, particularly in the social sciences, this remains to be seen. Causal pluralism leaves the question open. 12.7

Conclusion

In closing, I  want to note another difference between the original site of this discussion in the biomedical sciences and the extension to the social sciences. Russo-Williamson proposed their account of evidence for causal claims in part as a response to the evidence-based medicine movement – a movement that favours associational evidence over all other evidence. There are many reasons to think this is problematic and the Russo-Williamson thesis calls for a reassessment of that approach. In political science, and perhaps this is true for other social sciences, the situation is not quite the same. While quantitative and experimental research have dominated throughout the social sciences, mechanistic, ethnographic, materialist, realist, and other methodological approaches have not vanished. The more recent turn to MMR and the emergence of a variety of evidential partnerships indicates disciplinary differences, at least as far as basic research is concerned. There may be some reason to be fearful about the eclipse of methodological pluralism with the move towards evidence-based policy, but this is more likely to be prevalent in fields other than political science. Insofar as Shan and Williamson bring attention to the dominance of one form of methodological approach in the field over another, I  am in agreement with them. And much MMR research functions in much the way they describe. However, I think it is important not to lose sight of the richness of interactions among methodological approaches and I hope that I have offered a taste of what more evidential partnerships offer to knowledge production. Notes 1 The disputes I  have in mind is whether multimethod research and mixed methods research should be understood as two names for the same sort of approach or whether there might be reasons to distinguish them. Some discussion of such disputes appears in the Introduction to The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry (Hesse-Biber, 2015). Political methodologists do not seem to distinguish them and more often use ‘multi-method’. I have followed that practice in this chapter.

Evidential Partnerships and Multi-Method Research

259

2 Evidential partnerships may involve more than two methodological approaches, unlike EPsw which focuses on two methodological approaches. Gary Goertz has proposed a research triad which involves causal mechanisms, cross-case analysis, and case studies (Goertz, 2017, p. 1). 3 A version of this analysis appears in Crasnow (2022). 4 Wood does not deny that other mechanisms were operating but only that without making reference to the emotional and moral features that emerge out of her interviews the accounts are limited. Most importantly, they do not make sense of the differences between those who participated and those did not, whereas her account fits well with the two-thirds non-participation. 5 A full analysis appears in Crasnow (2015a). 6 Shan and Williamson could respond that there is bad social science research that fails to conform to their criteria, however Hyde’s argument is thought to be exemplary of good research design and in fact she received an award for it. I am inclined to think that what is deemed good research within the discipline should be considered good research unless there are good reasons for rejecting that claim. Since EPSW is not received practice but still in dispute that research does not conform to it cannot be considered a compelling reason of that sort. 7 I owe the distinction between method and methodology to Harding (1987). 8 Although assessment of causal claims might be thought of as a separate and additional goal, their directives for assessment require examination of the association and mechanistic studies through which such claims purport to be established and so this goal appears to dependent on the establishment goal. 9 Helen Longino’s contextual empiricism is an earlier account of the contextdependence of evidence (Longino, 1990). Shan and Williamson note that such an account raises worries about relativism. Longino addresses these through an account of objectivity. Cartwright et al. also tackle worries about relativism and discuss objectivity (Cartwright et al., 2023). 10 Faletti & Lynch (2009) are specifically interested in the way in which causal claims supported by evidence of mechanisms always require context. 11 Helen Longino (2013) argues along similar lines that view must parse the causal space in the same way. I have also argued this in Crasnow (2015). 12 I suggest that Woodward (2003) offers a means of doing so but I do not explore how successful his account is.

References Alexandrova, A. (2017). A Philosophy for the Science of Well-Being. Oxford: Oxford University Press. Beach, D., (2021). Evidential pluralism and evidence of mechanisms in the social sciences. Synthese, 199, 8899–8919. Beach, D.,  & Pedersen, R. B. (2019). Process-Tracing Methods: Foundations and Guidelines. Ann Arbor: University of Michigan Press. Beatty, J. (2016). What are narratives good for? Studies in the History and Philosophy of Biological and Biomedical Sciences, 58, 33–40. Bennett, A., & Checkel, J. (2014). Process Tracing: From Metaphor to Analytic Tool. Cambridge: Cambridge University Press. Carroll, N. (2001). Narrative connection. In N. Carroll (Ed.), Beyond Aesthetics: Philosophical Essays (pp. 118–133). Cambridge: Cambridge University Press. Cartwright, N. (2007). Hunting Causes and Using Them: Approaches in Philosophy and Economics. Cambridge: Cambridge University Press.

260

Sharon Crasnow

Cartwright, N. (2021). What counts as evidence for a mechanism? and why? The Reasoner, 15, 53–54. Cartwright, N., Hardie, J., Montuschi, E., Soleiman, M., & Thresher, A. C. (2023). The Tangle of Science: Reliability Beyond Method, Rigour, and Objectivity. Oxford: Oxford University Press. Chang, H. (2012). Is Water H2O? Evidence, Realism, and Pluralism. Dordrecht: Springer Publishing. Crasnow, S. (2015a). Natural experiments and pluralism in political science. Philosophy of Social Science, 45(4–5), 424–441. Crasnow, S. (2015b). Feminism, causation, and mixed methods research. In S. HesseBiber & B. Johnson (Eds.), Oxford Handbook of Multi-and Mixed-Methods Research Inquiry (pp. 637–651). Oxford: Oxford University Press. Crasnow, S. (2017). Process tracing in political science: What’s the story? Studies In History and Philosophy of Science Part A, 62, 6–13. Crasnow, S. (2021). Coherence objectivity and measurement: The example of democracy. Synthese, 199, 1207–1229. Crasnow, S. (2022). Process tracing and narrative science. In M. Morgan, K. Hayek, & D. Berry (Eds.), Narrative Science: Reasoning, Representing and Knowing Since 1800. Cambridge: Cambridge University Press. Crasnow, S., Goertz, G., & Haggard, S. (forthcoming). Pluralism and partnerships: The evidential foundations of multimethod research in political science. In Janet BoxSteffensmeier, Dino P. Christenson, & Valeria Sinclair-Chapman (Eds.), The Oxford Handbook of Methodological Pluralism. Oxford: Oxford University Press. Douglas, H. E. (2009). Science, Policy, and the Value-Free Ideal. Pittsburgh: Pittsburgh University Press. Dunning, T. (2012). Natural Experiments in the Social Sciences: A  Design Based Approach. New York: Cambridge University Press. Faletti, T.,  & Lynch, J. (2009). Context and causal mechanisms in political analysis. Comparative Political Studies, 42(9), 1143–1166. George, A. L.,  & Bennett, A. (2005). Case Studies and Theory Development in the Social Sciences. Cambridge: MIT Press. Goertz, G. (2017). Multimethod Research, Causal Mechanisms, and Case Studies: An Integrated Approach. Princeton: Princeton University Press. Goertz, G., & Mahoney, J. (2012). A Tale of Two Cultures: Qualitative and Quantitative Research in the Social Sciences. Princeton, NJ: Princeton University Press. Harding, S. (1987). Introduction: Is there a feminist method? In S. Harding (Ed.), Feminism and Methodology (pp. 1–14). Bloomington: Indiana University Press. Hesse-Biber, S. (2015). Introduction: Navigating a turbulent research landscape: Working the boundaries, tensions, diversity, and contradictions of multimethod and mixed methods inquiry. In S. Hesse-Biber  & B. Johnson (Eds.), The Oxford Handbook of Multi-method and Mixed Methods Research Inquiry (pp. xxxiii–liii). Oxford: Oxford University Press. Hyde, S. D. (2007). The observer effect in international politics: Evidence from a natural experiment. World Politics, 60(1), 37–63. Illari, P. M. (2011). Disambiguating the Russo-Williamson thesis. International Studies in the Philosophy of Science, 25(2), 139–157. Johnson, R. B. (2017). Dialectical pluralism: A metaparadigm whose time has come. Journal of Mixed Methods Research, 11, 156–173.

Evidential Partnerships and Multi-Method Research

261

Longino, H. (1990). Science as Social Knowledge: Valurs and Subjectivity in Scientific Inquiry. Princeton, NJ: Princeton University Press. Longino, H. (2013). Studying Human Behavior: How Scientist Investigate Aggression and Sexuality. Chicago: University of Chicago Press. Morgan, M. S. (2013). Nature’s experiments and natural experiments in the social sciences. Philosophy of Social Science, 43(3), 342–357. Reiss, J. (2009). Causation in the social sciences: Evidence, inference, and purpose. Philosophy of the Social Sciences, 39, 20–40. Russo, F., & Williamson, J. (2007). Interpreting causality in the health sciences. International Studies in the Philosophy of Science, 21(2), 157–170. Shan, Y., & Williamson, J. (2021). Applying Evidential Pluralism to the social sciences. European Journal for Philosophy of Science, 11(96), 1–27. Shan, Y., & Williamson, J. (2022). Evidential monism, Evidential Pluralism, or evidential contextualism? An introduction to evidential diversity in the social sciences. Synthese, 200, 321. https://doi.org/10.1007/s11229-022-03801-z. Williamson, J. (2013). How can causal explanations explain? Erkenntnis, 78, 257–275. Wood, E. J. (2003). Insurgent Collective Action and Civil War in El Salvador. Cambridge: Cambridge University Press. Woodward, James. (2003). Making Things Happen: A Theory of Causal Explanation. New York: Oxford University Press.

INDEX

axiological pluralism 114 – 116 causal monism 8n3, 242, 255 – 256 causal pluralism 5, 7, 119, 240, 242, 254 – 258 dialectical pluralism 3 – 4, 8, 19, 25, 45, 100 – 102, 104 – 111, 113 – 120, 119 – 120, 143, 148 dialectical position 2 – 3, 6, 8, 19, 65, 83 – 87, 90, 92 – 93, 97, 100 – 101, 127, 136, 149 epistemological pluralism 111 – 112 evidential partnership 5, 240 – 242, 245, 247, 249 – 250, 253 – 255, 257 – 259 evidential pluralism 5, 8n3, 195, 202, 204, 240 – 242, 252 – 255, 257

indigenous position 2 – 3, 6, 8n2, 37 – 38, 45 – 46, 54 – 67, 69 – 79 methodological pluralism 5, 118, 240, 250 – 252, 256, 258 pragmatism 4 – 5, 13 – 27, 37, 88 – 90, 100, 104, 110, 112, 115, 171 – 174, 180, 187, 189 – 190 realism 4 – 5, 7, 17, 19 – 21, 64, 104, 109, 112, 115, 152 – 155, 160, 163, 163n1, 181, 187, 194, 202, 204 – 206, 206n1, 207n12, 256 scientific pluralism 251 transformative position 2 – 3, 6, 16, 19 – 20, 30 – 49, 54 – 55, 58, 60 – 62, 64 – 65, 66, 78, 110, 143, 148 – 149