Understanding Research in Early Childhood Education: Quantitative and Qualitative Methods [2 ed.] 1032407271, 9781032407272

Citation preview

UNDERSTANDING RESEARCH IN EARLY CHILDHOOD EDUCATION

This second edition invites readers to be informed consumers of both quantitative and qualitative methods in early childhood research. It offers side-by-side coverage and compar­ ison about the assumptions, questions, purposes, and methods for each, presenting unique perspectives for understanding young children and early care and education programs. The new edition includes updated examples and references as well as a new chapter on equity issues in research. By using this book, students will be able to read, evaluate, and use empirical literature more knowledgeably. These skills are becoming more important as early childhood educators are increasingly expected to use evidence-based research in practice and to participate in collecting and analyzing data to inform their teaching. Jennifer J. Mueller is Dean of the College of Education and Professor of Teacher Educa­ tion at DePaul University, USA. Nancy File is Professor Emerita in the Department of Teaching and Learning at the Uni­ versity of Wisconsin-Milwaukee, USA. Andrew J. Stremmel is Professor Emeritus in the School of Education, Counseling, and Human Development at South Dakota State University, USA. Iheoma U. Iruka is Research Professor in the Department of Public Policy at the University of North Carolina-Chapel Hill, USA. Kristin L. Whyte is Associate Professor of Education at Mt. Mary University, USA.

UNDERSTANDING RESEARCH

IN EARLY CHILDHOOD

EDUCATION

Quantitative and Qualitative Methods Second edition

Jennifer J. Mueller and Nancy File with Andrew J. Stremmel, Iheoma U. Iruka, and Kristin L. Whyte

Designed cover image: shutterstock Second edition published 2024 by Routledge 605 Third Avenue, New York, NY 10158 and by Routledge 4 Park Square, Milton Park, Abingdon, Oxon OX14 4RN Routledge is an imprint of the Taylor & Francis Group, an informa business © 2024 Jennifer J. Mueller and Nancy File The right of Jennifer J. Mueller and Nancy File to be identified as authors of this work has been asserted in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988. All rights reserved. No part of this book may be reprinted or reproduced or utilised in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers. Trademark notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. First edition published by Routledge 2017 Library of Congress Cataloging-in-Publication Data Names: Mueller, Jennifer J., author. | File, Nancy, author. | Stremmel, Andrew J., author. |

Iruka, Iheoma U, author. | Whyte, Kristin L., author.

Title: Understanding research in early childhood education : qualitative and quantitative

methods / Jennifer J. Mueller and Nancy File with Andrew J. Stremmel, Iheoma U Iruka,

and Kristin L. Whyte.

Description: Second edition. | New York, NY : Routledge, [2024] |

Includes bibliographical references and index. |

Identifiers: LCCN 2023039435 (print) | LCCN 2023039436 (ebook) |

ISBN 9781032407272 (hardback) | ISBN 9781032394909 (paperback) |

ISBN 9781003354499 (ebook)

Subjects: LCSH: Early childhood education--Research--Methodology.

Classification: LCC LB1139.225 .F55 2024 (print) | LCC LB1139.225 (ebook) |

DDC 372.21–dc23/eng/20230830

LC record available at https://lccn.loc.gov/2023039435

LC ebook record available at https://lccn.loc.gov/2023039436

ISBN: 978-1-032-40727-2 (hbk)

ISBN: 978-1-032-39490-9 (pbk)

ISBN: 978-1-003-35449-9 (ebk)

DOI: 10.4324/9781003354499

Typeset in Galliard

by Taylor & Francis Books

CONTENTS

Acknowledgements

vii

1 Introduction

1

PART I

Structures of Research

9

2 How Do We Know? A Primer on the Nature of Knowing in Early Childhood Research

11

KRISTIN L. WHYTE

3 Advancing Equity in Early Childhood Education Research

23

IHEOMA U. IRUKA

4 The Culture and Contexts of Research in Early Childhood Education

35

5 Methodologies

45

6 Qualitative Research: Aims and Methods

61

7 Understanding Analyses in Qualitative Research

73

8 Quantitative Research: Aims and Methods

87

9 Understanding Analyses in Quantitative Research

99

PART II

Asking Research Questions

117

10 Research Questions About Children

119

11 Research Questions About Adults in Children’s Lives

129

12 Research Questions About Curriculum and Classrooms

140

13 Research Questions About Institutions and Policy

151

v

CONTENTS

14 Teacher Research

162

ANDREW J. STREMMEL

PART III

Conclusion

171

15 Research and Practice: Potentials, Challenges, and Limitations

173

Glossary References Index

177

186

197

vi

ACKNOWLEDGEMENTS

This very beginning to the text is being written at the end. It is now we gather our thoughts about the work and what we have done together, with many people standing beside and/or behind us. Research has been a significant part of our careers. We each pursued doctoral degrees after years of being in the classroom with children. Our classroom experiences pushed us to find new ways to pursue understanding and continue to ask and find answers for our questions. And so for years we have identified as scholars in one of our job roles. We each, however, also identify strongly as a teacher. We no longer teach young children, but the role of teaching is important to who we are. This is where this text was conceived. Some of the students we have worked with in our professional lives were preparing for careers as researchers and moved into academia. Many of the master’s level students in our courses, however, were teachers and remained so for the long term. For them the undertaking was focused on gaining depth of knowledge in their work. The literature in the field adds to that depth of knowledge, but learning how to understand and use it requires effort and practice. We wrote this text because of our interest in helping early-career professionals learn about research. The text comes from our own teaching, the concepts we discussed with students, the big ideas and take-away messages we introduced to them. And so, while we have written this text from our roles as researchers, we have also written strongly from our roles as teachers. Unsurprisingly, then, we must acknowledge the many students we have taught in the course of our careers. Their questions, their frustrations and confusion, and their growing clarity of understanding have shaped our own understanding of how students can learn about research. It has been a privilege to walk alongside our students and support their learning. In addition, we must thank individuals who played a part in producing this text. Meghan Johnson, a graduate of the University of Wisconsin-Milwaukee’s School Psychology pro­ gram, took on the job of writing the glossary for the first edition. Amanda Hanrahan, an alumnus in the same program, took on the task of editing and alphabetizing a reference section for that edition. For the second edition, we continued to benefit from that work. In this edition of the text, we are gratified Andy Stremmel updated his chapter about teacher research. He is steeped in knowledge on this topic, and we are lucky to have him on the team. In addition, two new colleagues joined our work on this edition. We acknowledge the contributions of Iheoma Iruka and Kristin Whyte. Dr. Iruka is well-known for her work in identifying and dismantling racism in early childhood. We are extremely fortunate for her involvement in this book. We have known Kristin Whyte since she was an undergraduate in our program and then a master’s student learning in the courses we described above. Now a faculty member herself, we are so proud she is a chapter author. In addition, Katherine

vii

ACKNOWLEDGEMENTS

Delaney, another graduate of our master’s program and now a faculty member, participated with Dr. Whyte. They are incredible professionals in a new generation. Finally, we thank the eight individuals who responded generously when we invited them to submit essays about their work. They are (in order of appearance in the book): Jóhanna Einarsdóttir (with Sara M. Ólafsdóttir), Seung-Hee Claire Son, Shaddai Tembo, Diana Leyva, Valentina Pagani, Virginia Vitiello, Melissa Sherfinski, and Ersoy Erdemir. Readers will benefit from their first-hand perspectives. We have worked together and from a distance for many years. We have known each other as we’ve grown as scholars and teachers, and it has been a mutually supportive and beneficial web of professional and personal relationships. We could not have done this solo, and we appreciate that together we end up places neither could go alone. Individually, we have further acknowledgements. Jennifer: Of course I need to thank my family – those who live in my house and the extendeds. Lots of patience and doing life without me has been necessary and I appreciate all the love, texts, memes, and beverages. Here I will also thank one of my most challenging kiddos from my teaching days – Herman Williams. He was one of those children who pushed one to their limits and made me learn more about myself as a teacher than I thought possible. He also was the child who inspired me to go to graduate school so that I could make a ‘bigger’ impact. As always, big thanks to all of my students and colleagues in my dean life who continually brighten my day and make me hopeful for the future of early childhood education. Nancy: I must thank my family, in their nuclear form, and the addition of in-laws and grandbabies. I acknowledge the person who was my mentor during my introduction to research, the late Susan Kontos. Her voice is indelibly present in my mind, and it continues to be a pleasure to hear. I’ve learned from so many others as well, mentors, students, and colleagues. A rich set of friends within the profession has supported my ongoing growth for years, and I look forward to conversations with them about the important stuff and laughter about the rest.

viii

1

INTRODUCTION

“The research…” There are many ways this sentence is completed. Perhaps some of the following are familiar. “The research… suggests…” proves…” raises questions…” adds to our understanding…” implies …” leads to these conclusions…” contradicts earlier research…” supports earlier research…” fails to show…” These examples are drawn from ways researchers discuss their work and how research is pre­ sented in popular media. In our society, individuals are exposed to research through many channels. One can read original research reports written for professional audiences, read ver­ sions of research reports written for a lay audience, or hear or read about research as inter­ preted by others, such as journalists, professional development providers, and even marketers. It is common to hear a variety of claims about research. Anyone paying attention is bound to have questions. Does research prove something? Why does research result in contra­ dictory findings? What happens if research fails to show something? Why do researchers sound so cautious or tentative – can’t they just take a stand? Is research about finding answers or generating more questions? How does research lead to implications? One might be led to declare: “JUST TELL US!” (and, please, in plain language). The truth is that it is not that simple. Research does result in contradictory findings across dif­ ferent studies. When research findings are described, it can feel a bit underwhelming. Research is limited by how it was conducted. Limitations might relate to who participated (and who did not), what types of data were collected, and the degree to which findings are similar if the study was conducted with other people or in another place. Research is driven by questions, and it is likely questions will lead to more questions. Finally, and importantly, research is difficult to approach for those who are not research­ ers. The processes involved are intricate, and much of the professional terminology can sound like gobbledygook to the uninitiated. Our experience suggests that when discussing research with students, they admit that when reading a quantitative study, marked by dis­ plays of numerical results presented in tables, they “just go to the discussion section” at the end of the study, for an interpretation of results presented in words. And although they

DOI: 10.4324/9781003354499-1

1

INTRODUCTION

might feel more comfortable reading a qualitative study, described solely through narrative, the vocabulary and conceptual knowledge required to comprehend the work can get the better of them. Hence, our intent here is to provide guidance for understanding research. As profes­ sionals, teachers in early childhood education are expected to have some knowledge of the literature. Inquiry is a highly valued activity in teaching; good teachers continually ask questions about their work and the children they serve. The knowledge produced by researchers is a useful tool for teachers to consult in inquiry processes. Understanding the literature also provides teachers with a chance to participate in dialogue with others about where and how research links to practice. Thus, we believe being a skilled consumer of research literature is an important compe­ tency for educators. Not all research is created equally. It is important to be able to evaluate the methods used within a study and decide how much trust to place in the findings. Each study has a unique profile of strengths and limitations, and one must look below the surface to identify these. Additionally, readers of research must make the translation to their own situations. How might the results apply? What implications are warranted? Capable con­ sumers of research understand research more deeply. They can follow the processes involved, evaluate the decisions made in the study, and decide how the work impacts their under­ standing and practice.

Research in Society What is Research? Research defines a particular form of inquiry intended to extend our understanding. For some, this is meant to be a more thorough understanding of the topic of inquiry that can be generalized to other situations. For others, understanding is a way of making meaning that acknowledges the unique nature of situations and the choice of a lens for interpretation, which precludes generalizability to other situations. Research revolves around questions, and typically in the search for more understanding, more questions arise. For some, the research process begins with clearly delineated ques­ tions, formed from an understanding of the research conducted to date. For others, the questions are less specific at the outset, allowing the study and the questions to evolve as the research proceeds. Research procedures are meant to result in data, as research is an evidence-based process. In other words, evidence, in the form of data, is generated and analyzed to develop findings and implications. For some researchers, evidence is reflected via numbers, with numeric descriptions of phenomena and statistics used to analyze data. For others, evidence is expressed in words, and analysis supports identification of descriptions and themes. Research follows forms of logic. In presenting the results of a study, the researcher works to explain the decisions and theory that propelled the study, allowing others to agree (or not) with those processes. Additionally, the logic stems back to basic philosophical positions about the world. This begins with questions about reality and how we know what we know. For some, what we perceive is our reality. It can be measured and assessed, and known in objective ways. For others, reality is what we construct as we make meaning about the world. There are processes to be followed in research; rules of the road taught to successive generations of researchers. Although the processes exist within separate schools of thought or paradigms, and thus differ, they act similarly in any school of thought to enclose the

2

INTRODUCTION

project within a shared definition of what research means. Not unexpectedly, though, there are always some who push against boundaries to define new understandings of what proce­ dures mean and/or could mean. To summarize what research is, we offer the following. Research is a form of inquiry that relies upon evidence, or data. It is complex because there are many accepted ways to do research. And yet ‘accepted’ is a key word because research is governed as a particular form of inquiry with defined processes, differentiated from other ways of knowing about the world such as wisdom, reflection, or craft knowledge. Research is conducted through a variety of processes. Its scope might be finely-focused, for example, a case study of a single child. Or, it might be wide-ranging, such as the impact of a policy change on kindergarteners. Furthermore, research is traditionally conducted within a setting of accountability and scrutiny. The traditional standard for publication of a research article is the peer review process. Prior to publication, other researchers read the article without the authors being identified. Evaluation of the article (submitted back to the authors also without the review­ ers being identified) may raise questions and suggest concerns about how the research was conducted, oftentimes leading to a redraft of the article. Through this review process, the editors of journals decide which articles will be published and which will be rejected. In addition, foundations, think-tanks, or private entities can self-publish the findings of their own research. This has always been true, but in an internet-dominated world these reports are more accessible than ever. In many cases these reports reflect skilled research. However, these reports have not undergone the peer review process described above, leav­ ing more responsibility on the shoulders of the reader to determine the quality of the study. The Impact of Research Science and processes of research are revered in our society. Information gained through research is typically assumed to have credence among the public. After conducting a poll of both the public and scientists, the authors of a report wrote (Pew Research Center, 2015): Science holds an esteemed place in the public imagination and in the minds of pro­ fessionals. Americans are proud of the accomplishments of their scientists in key fields and, despite considerable dispute about the role of government in other realms, there is broad public support for government investment in scientific research. Yet, as mentioned, scientific research can be perplexing, particularly when viewed from the “outside.” Different studies focused on the same topic can vary in the process details, which can impact findings. Chains of logic utilized by researchers differ among studies, including what assumptions are accepted. New methods are developed that produce different results. To illustrate, consider a story reported in the New York Times detailing how the latest research on mammography for breast cancer detection raised questions about its effective­ ness. The newest research used a large group of women and was described as “meticulous.” The newest study reported the death rate from breast cancer was almost the same between the group of women who had mammograms and those who did not. Still, there were many unanswered questions. The study resulted in a set of findings, but did not answer questions about “why” mammograms did not offer the widely assumed health benefits. Was it because of newer, more effective treatments for cancer? Or because women are now more aware of breast cancer and more proactive in their health care? What more do we know about detecting and treating cancer over time? What questions do we not know to ask yet? Are

3

INTRODUCTION

there groups that were left out of the study that could provide important clues related to equity in health outcomes? Because of the mixed results and the unanswered questions, the reporter stated that it was unlikely that recommendations for regular mammogram screening would change (Kalota, 2014). In this case, a single study, described as well-done, was not enough to change practice. This story illustrates a central point about research. When examining complex subjects, the answers are often partial and accompanied by cautions. In regard to medical research, the human body is an intricate system. Some research findings provide us with averages, but rarely do these researchers describe the processes of one person’s genetic inheritance, life habits, environment, socio-structural disparities, and reaction to medications used in treatment. There are often many potential reasons why results might vary from one study to another. Similarly, intricacies abound in educational research. Classrooms are made up of many children from unique families, each with a distinctive relationship with the teacher. Although subgroups of individuals may share cultural norms and expectations, each child enters the classroom with an individual set of background experiences. Thus, there is a complex mix of similarities and differences among children. Similarly, each teacher is an individual. Just as with children, teachers have varied background experiences, and the differences extend across the personal and professional spheres. Considering classroom processes, during any particular learning activity some children may be deeply engaged while others are tired or disinterested. This could be related to elements of the activity, teacher–child relationship dynamics, or experiences from earlier in the day. Accordingly, results may vary across studies, and what we know through educational research is incomplete. In sum, our society has a history of holding scientific research in high esteem, and we agree on the impact of research on many aspects of our lives. Nonetheless, research often does not provide us with the clear and absolute answers we might desire and which those uninitiated to the complexities and limitations of research feel they can expect. Continued questioning, new methods, and different perspectives all contribute to the tentative nature of “what we know” via research. So, researchers cannot “just tell us,” as we mentioned earlier. What they can do, however, is tell us about their work. The rest is up to those who read the study – to understand the work, put it alongside other work, consider its implications, and make meaning of it within their own understandings of teaching young children.

Research in Early Childhood Education Any who have read an introductory textbook in the field might remember the propensity to look back across the centuries to attribute contributions to early childhood education from figures such as Locke, Rousseau, and Pestalozzi. Philosophical stances and ideologies regard­ ing childhood and learning have a long history. In contrast, the history of research as a way of knowing about children and education is much briefer, only slightly more than a century old. Even with a relatively brief history of research, there has been much change over time. In this book, we will help the reader better understand how the ideas and phenomena valued in research are greatly influenced by larger social and cultural contexts. What we believe to be important, what we value, and what we want to understand as a society or as a culture at any point in time is shaped by historical moment, dominant political ideologies, the technology of the time, and prominent paradigms of thought. It follows that early childhood education, as an institution of our society, has moved through different phases in what we have believed to be its purpose and goals, and so follows the purposes of and needs for educational research.

4

INTRODUCTION

To be a competent consumer of research in early childhood education, it is important to realize how larger social contexts come to bear in the research endeavor over time within the field. We therefore help the reader understand how the two main approaches to research – quantitative and qualitative – have, at different points, defined the field. They have deter­ mined what we value and study, and what it means to be a child in a classroom or an early childhood educator. We also discuss how social conditions and the needs of young children in educational settings have provided the impetus for different kinds of study and for mul­ tiple voices to be included in the research endeavor. In early childhood education, we have, for example, moved back and forth with various goals of research, from finding overall uni­ versal truths that allow us to explain and predict what children need to know and do in their learning settings, to examining more fine-grained understandings of localized phenomena. In the coming chapters we explore how those who operate in these differing paradigms, quantitative and qualitative, start from different places. They have different answers to questions about what reality is and how we come to know what we know. They pose dif­ ferent questions and work to answer them with different methods. Most importantly, they provide us with different perspectives about what it means to be a young child within a family and ever-enlarging contexts, including early childhood programs. Several reframings have been made in the second edition of our book, which we com­ ment on below. Since completing the first edition in 2016, we have witnessed and experienced monumental changes both personally and societally. We have experienced the #MeToo movement, the murder of George Floyd with the subsequent rise in our collec­ tive attentiveness to the deep and harmful impacts of racism and White supremacy on our society (and the subsequent and ongoing backlash), and of course a global pandemic. We each personally underwent large shifts in our professional lives that altered our connections to students and children in important ways. (One of us retired and became a grandparent, one of us moved into higher education administration and also became a childcare center director during the pandemic.) We write in later chapters about the importance of context in shaping one’s worldview, approaches, and understandings. Given the many transitions of the past seven years, we know we are changed as scholars, as teachers, as family members, as colleagues. We literally “know” things differently. It is not hyperbole to say we are different as humans – and perhaps one of our greatest recognitions is the need to center on the humanity of our work and our connections to others. While we did focus on this humanity element of research in the first edition, we construe that differently now – and much more centrally – in this different time. We both have always been committed to issues of equity, diversity, anti-racism, inclusion, and access in education writ large and in early childhood education. We acknowledge that our White privilege and a framing of White supremacy has allowed us to forefront and background that commitment freely over the course of our careers, picking and choosing when to speak out and when to stay silent. We also live within orbits of oppression (as women, one as a member of the LGBTQi+ community, one connected to the Jewish com­ munity). We have learned over and over that the choice of silence, which maybe self-pre­ serving at times, is, over the long-term, not a viable choice in our advocacy work for children, families, teachers, students, or early childhood education. It has been satisfying to work with the first edition and to find ways where we could enhance our explanations with this heightened framing. We know much more deeply the need for a multitude of voices to be included, to be fore-fronted, and to be centered. Understanding the ways that research has historically been used to oppress, dis-represent, diminish, exclude, and, in fact, com­ pletely dehumanize the “other” is an urgent and crucial task. Using that framing to question

5

INTRODUCTION

what research can and cannot do – what it can and cannot tell us – is incredibly important and IS the work, rather than being a “component” of the work. We live by the mantra that we owe it to “the babies” to do this well, and we keep learning and changing for them.

Outline of the Book Our intent is to prepare readers to be informed consumers of early childhood research. To that end, we include information across the major models for conducting research. We include both quantitative and qualitative methods so that readers may access all available literature. We coach readers in how to understand research literature and make evaluations about the trustworthiness and applicability of studies. It is possible to become a competent evaluator of research with a single text. The task of learning how to conduct research, however, is more complex and requires multiple texts and courses across both paradigms. To equip readers with the necessary background knowledge to understand the research, we draw our examples from the research literature on early childhood education and care. By drawing our examples from the field, we hope readers will also learn about the exciting and challenging work conducted in our field. We divided the book into two sections: Part 1 – Structures of Research; and Part 2 – Examining Research Questions. Dr. Kristin Whyte examines the basic foundations of research in Chapter 2 and provides definitions for research-specific terminology. Focal topics include the ethics of conducting research and how research and policy-makers interface. For Chapter 3 we invited Dr. Iheoma Iruka to share her insight and expertise related to the need to deconstruct boundaries, rethink equity and diversity in early childhood education, and center the voices and experiences of those who have been marginalized (dehumanized) in research processes. In Chapter 4, we discuss the context within which research is conducted and used, as well as the cultures of research. Beginning with Chapter 5, we introduce the major paradigms of qualitative and quantitative research. We compare the major tenets of the research process within each paradigm, includ­ ing the researcher’s role, standards of rigor, and assumptions about the implications of studies. In the two chapters that follow (Chapters 6 and 7) we describe how qualitative research is conceptualized and the processes for generating data, as well as how to understand presenta­ tions and interpretations of data. We follow a similar set of goals for Chapters 8 and 9, which examine the conceptualization of quantitative studies and processes for data collection, as well as explaining the presentation of findings in a quantitative study and what they mean. In this group of chapters we also present information on how to read and evaluate research studies. The second major section of the book is a set of five chapters intended to bring to life the logic and modes of inquiry used in early childhood research. Our structure is around the “unit of analysis,” or the focus of the study. Thus, there are chapters that center on: ques­ tions about children, questions about the adults in children’s lives, questions about class­ rooms and curriculum, and questions about institutions and policy. In each of these chapters we compare the perspectives and work of qualitative and quantitative researchers. We note the types of questions they ask and the kinds of data generated in each area. Each chapter is a conversation back-and-forth across the paradigms. The final chapter in this section exam­ ines teacher research. Our concluding chapter considers research through the lenses of potentials, challenges, and limitations. Research is a human enterprise and throughout the text we encourage a multi-faceted examination to support readers to decide for themselves what a study means, rather than uncritically accepting all statements that begin, “The research says… .”

6

INTRODUCTION

We approach this having been educated ourselves in different research paradigms. Nancy earned her degrees through programs in Child Development with a singular focus on quantitative research. Jennifer earned her degrees in Education programs and conducted qualitative research during her schooling while being educated about both paradigms. We have continued to grow and question ourselves as researchers, both singularly and together. Part of this book’s structure was “birthed” as we co-taught a graduate course in under­ standing research. We found our ability and willingness to simultaneously talk across para­ digms with students was invigorating and helpful to their understanding. In relation to any topic, we explored questions about what would interest researchers within the paradigms and how they would make meaning of phenomena. This experience solidified that we could not create a book where the two paradigms constituted two separate sections, never to meet. We include two other features to maximize the usefulness of this book. First, we include short essays from colleagues where they answer the prompt, “How I have used research to find out about… .” These essays appear in the second section of the book to illustrate the unique perspectives offered by varying types of research. Also, we include a glossary at the end of the text. Research is marked by terminology, no matter the paradigm. We hope readers consult the glossary freely until they have mastery over these many words and concepts. Finally, we encourage readers to think about research as a story. We do not intend to draw analogies between research and the elements of story. Rather, we suggest readers consider questions such as: What is the story here? Who is it about? What happened? What is left out? The beauty of stories is that they are accessible. Stories draw us in, even (perhaps especially!) if they are not as simple and straightforward as they first appear. It may help readers to regard research as a story to make a study more comprehensible, giving them an approach that aids in understanding. This disposition, along with the knowledge gained in the coming chapters, can reduce many of the barriers toward understanding research and research-to-practice links. It is also key to remember, just as with stories, research is about people. In every study we use as an example in this book, we learn something about chil­ dren, family members, and/or early childhood professionals. The goal of research, after all, is to enhance lives. We hope these research stories contribute to your strengths and capabilities in serving young children and their families!

7

Part I

STRUCTURES OF RESEARCH

2 HOW DO WE KNOW? A Primer on the Nature of Knowing in Early

Childhood Research

Kristin L. Whyte

An early childhood educator’s day is filled with innumerable decisions. Imagine a kindergarten teacher’s classroom during the first week of school: How should I approach teaching classroom routines? How do I know what literacy skills to teach first? What is the best way to start off the year with families? Picture an infant caregiver, in the moment, making decisions about a baby’s daily schedule: How do I know when the baby I’m caring for should be fed? How do I know how to put a baby down for a nap? What do I even do with a baby outside of feeding, diapering, and napping? Or, think about an early learning director meeting with her team to make decisions about their school’s goals for the upcoming year: What should we prioritize this year? How will we ensure we are providing high-quality education? What kinds of supports should we offer our teachers? Each of these questions is not only a question for educators, it is also a question of interest for researchers. Research plays a key role in early childhood education. Researchers often examine a given topic from various angles, looking across a variety of studies to develop understandings about bodies of literature or “what the research says.” Educators can use research to make decisions about how to navigate their work. Whether you are a consumer of research or learning how to conduct research, it is essential to develop understandings about bodies of knowledge researchers look to when deciding how to approach a topic of interest. Exploring beliefs about research and knowledge helps us know how to approach designing research, as well as how to interpret and apply research. This chapter offers an introduction to the initial philosophical underpinnings that are likely present for researchers.

A Beginner’s Guide to Understanding How We Know Researchers hold beliefs about the nature of knowledge. What does it mean to truly know some­ thing? How do we know what we know? These questions are key to understanding research. The way we respond to these philosophical questions should shape both our research designs and understandings of research findings (Creswell, 2009; Crotty, 1998; Scotland, 2012). One way to understand how these beliefs undergird different approaches to research is to develop working definitions of the common terms researchers use to describe the conceptual understandings guid­ ing their work, as well as how these terms relate to each other. We refer to them as working defi­ nitions because, as you will soon see, how these concepts are understood and applied are somewhat flexible. This is often because of a researcher’s beliefs about the nature of knowing and the intel­ lectual traditions in which they ground their work.

DOI: 10.4324/9781003354499-3

11

STRUCTURES OF RESEARCH

The term paradigm is often used to speak to the nature of research. The idea of a para­ digm, and of paradigm shifts, largely both come from Thomas Kuhn’s (1962/2012) work on the philosophy of science, in his book The Structure of Scientific Revolutions. A paradigm can be thought of as a group of beliefs about science and knowledge, and Kuhn’s work challenged beliefs about objectivity and science. His work speaks to the idea that science itself is not a monolith – there are different paradigms present within scientific inquiry and these paradigms “influence what should be studied, how it should be studied, and how the results of the study should be interpreted” (Kivunja & Kuyini, 2017, p. 26). In other words, understanding different paradigms not only helps a researcher make sensible design choices but also, knowing the paradigm that grounds a researcher’s work can help a reader make clearer sense of an individual study’s findings, which in turn can lead to clearer under­ standings of a body of research. Scholars focused on research design often use similar terms to think about the different components of research but tend to foreground particular elements. Lincoln et al. (2011), for one example, take up the idea of a paradigm and in doing so offer four constructs that ground research paradigms: axiology, ontology, epistemology, and methodology. Creswell (2009, 2012), for another example, is also concerned with these relationships but includes rhetoric. He also uses the term worldviews interchangeably with paradigms and discusses how research designs consist of philosophical worldviews, strategies of inquiry, and research methods. Crotty (1998), for yet another example, lists four elements of research: episte­ mology, theoretical perspective, methodology, and methods. These elements of research can be thought of as somewhat nested, with epistemology being the broadest category, as it is about the nature of knowledge, and methods being about a particular practice. Regardless of around which core structures an researcher organizes their work it is key is that these dif­ ferent components should be well-aligned. A researcher’s epistemological stance should make sense for the theoretical perspectives they draw upon, what methodology they choose, and the data collection methods they utilize. So what do all these different terms mean? There are widely-agreed upon definitions of the broader terms but they are often used in different ways and they are not static. We start with some definitions by looking at Crotty’s (1998) four elements of research: epistemology, theoretical perspective, methodology, and methods. Epistemology This speaks to one’s theory of knowledge. Epistemology is concerned with how we come to know something (Kivunja & Kuyini, 2017). People take different epistemological stances. Crotty speaks to three: Objectivism, Constructionism, and Subjectivism. �

�

Objectivism is the idea that things exist independently of human consciousness; that there is truth present within an object regardless of how humans experience the object. This makes truth objective. Because this reality exists outside of humans, with “scien­ tific” research, through a careful process in which reality is reduced to small instances, we can acquire these truths. Constructionism treats objects and truths much differently. With constructionism, “all knowledge, and therefore all meaningful reality as such, is contingent upon human practices, being constructed in and out of interaction between human beings and their world, and developed and transmitted within an essentially social context” (Crotty, 1998, p. 42). In other words, in constructionism, objects and subjects are inseparable

12

HOW DO WE KNOW?

�

and there is an acknowledgment of the collective generation of knowledge and mean­ ing, which calls for research that focuses on interpretations of how experiences, which are inherently social, create meaning. Connected to constructionism is the idea of con­ structivism. While these two concepts are well-connected to each other, it is important to note that constructivism emphasizes the uniqueness in how individuals make mean­ ing, rather than a collective generation, and it is also thought of as a research paradigm. Subjectivism treats truth differently from both objectivism and constructionism. Here, while truths exist, truth is about meaning being imposed onto an object, from a subject independent of that object. This makes truth(s) so impacted by context that no external or shared truths exist. This means that subjectivist researchers themselves are creating new truths as they interact with their participant(s) and share their research with the world. Theoretical Perspective

This is the philosophical stance on “our view of the human world and the social life within that world” (Crotty, 1998, p. 7). Our theoretical perspective is our attempt to describe, and thus make transparent, the assumptions that come along with these beliefs. While episte­ mology is about our beliefs about knowledge in general, our theoretical perspective is more about the relationships between our beliefs about the human world and the methodology we have chosen for our research. Theoretical perspectives are numerous and ever-evolving. Crotty (1998) spoke, broadly, to positivism, postpositivism, interpretivism, critical inquiry, postmodernism. �

�

�

Positivism and postpositivism describe the pursuit of truth via particular “scientific” methods that investigate relationships. Here, truth, or generalizations, are value-neutral, exist across contexts, and can be discovered. “Research located in this paradigm relies on deductive logic, formulation of hypotheses, testing those hypotheses, offering opera­ tional definitions and mathematical equations, calculations, extrapolations, and expres­ sions, to derive conclusions. It aims to provide explanations and to make predictions based on measurable outcomes” (Kivunja & Kuyini, 2017, p. 30). Both positivism and postpositivism are typically connected to objectivist stances. Also, they are often thought of as a research paradigm (e.g. Kivunja & Kuyini, 2017; Scotland, 2012). Interpretivism seeks “culturally derived and historical interpretations of the social lifeworld” (Crotty, 1998, p. 67). In other words, this approach is concerned with under­ standing the relationships between people and social reality, including thinking through the significance of culture (Crotty, 1998). Interpretivism is typically connected to con­ structionist epistemology and speaks against positivism, refuting the narratives around things such as an “objective, scientific observer.” Interpretivism embraces the role con­ text plays in knowing (Kivunja & Kuyini, 2017). Critical inquiry ascribes to some of the foundational beliefs of interpretivism, but the core focus is on how power and oppression operates, acknowledging that there are consequences that come with privileging particular realities (Kivunja & Kuyini, 2017). Denzin (2017) notes that critical theorists “are no longer called to just interpret the world, which was the mandate of traditional qualitative inquiry. Today, we are called to change the world and to change it in ways that resist injustice while celebrating freedom and full, inclusive, participatory democracy” (p. 9). Here we can see that critical theor­ ists embrace research with an agenda and that they place an emphasis on transforma­ tion – that they are called to transform reality through action, which speaks to the idea

13

STRUCTURES OF RESEARCH

�

that reality is not static, but in process. This theoretical perspective is connected to a transformative/emancipatory research paradigm. Postmodernism is a response to modernism, or rationality, and “engages in a radical decentering of the subject” (Crotty, 1998, p. 186). When thinking about the character­ istics of postmodernism, the following words come to mind: fragmentation, decentering, deconstructing, messiness, play, irony. Postmodernism calls reality as we know it into question and embraces uncertainty (Dahlberg, Moss, & Pence, 1999). Postmodernism is well connected to poststructuralism. These two concepts are sometimes thought of as the same; sometimes they are distinct (Lather, 1991). Postmodernism is also often thought of as a research paradigm. Methodology

Here is the core of research design. Broadly speaking, methodology is the general plan for addressing the problem we are attempting to examine and/or the body of literature to which we are trying to add. There are many methodologies from which to choose. Common ones are experimental/quasi-experimental research, ethnography, discourse analysis, groun­ ded theory, action research, case study, narrative inquiry, phenomenology, etc. Methods Methods are the ways we do research. These are the ways that data are gathered and ana­ lyzed, and the activities a researcher designs and implements. There are a wide-variety of commonly used methods and new methods are often created. To name just a few: sampling, questionnaires, focus groups, semi-structured interviews, participant observations, photoelicitation, artifact collection, deductive and/or inductive coding, memoing, fieldnotes, etc. Often these research elements are likely either to be considered quantitative or qualitative. These are key ideas we discuss both later in this chapter and throughout this book. While not a core element in Crotty’s work, it is also important to have a working knowl­ edge of ontology and axiology. These are both tied to broader beliefs about knowledge and the acquisition of new knowledge and are tied closely to our epistemological beliefs and the paradigms with which we align ourselves. Ontology is what we believe about the nature of reality itself – the idea of what is/what exists, and the assumptions we make in order to believe something exists (Crotty, 1998; Scotland, 2012). Axiology relates to ethics – what is an ethical way to know something (Kivunja & Kuyini, 2017; Mertens, 2012). A researcher’s response to questions like: Why do you study this phenomenon? How will you represent what you learned? etc., would likely be axiological in nature. To better understand these terms – epistemology, theoretical perspective, methodology, and methods – it is important to understand how they are applied. There are both consistent elements to these definitions and, simultaneously, they are dynamic in nature. Indeed, there can be a great deal of variation in how terms are nested or categorized. For example, what one scholar calls an epistemology, another calls paradigm, and another a theoretical stance. Or what one refers to as a methodology, another might call a method. For a more specific example, Morgan (2007) points out four different conceptualizations of the term paradigm (i.e. paradigms as worldviews, paradigms as epistemological stances, paradigms as shared beliefs in a research field, paradigms as model examples), arguing paradigms as epistemolo­ gical stances have been the most influential. He also suggests that Kuhn himself avoided using the paradigm in his later work as a response to the lack of clarity around the term. The

14

HOW DO WE KNOW?

paradigms that we have chosen to highlight in this book are: postpositivism, constructivism, transformative/emancipatory, postmodern, pragmatic. One could continue to explore how these stances are taken up. While Crotty (1998) calls positivism and postpositivism theories, others think of them as epistemological stances, and most refer to them as a paradigm. The term case study is also used in a variety of ways (Hyett, Kenny, & Dickson-Swift, 2014). For some, case study is a methodology and there are many different types of “cases”; for others it is more a set of methods. But, regardless of which stance is taken, all are likely concerned with what “bounds” a case (Hyett, Kenny, & Dickson-Swift, 2014; Merriam, 1988). Phenomenology might be thought of as a theoretical perspective and/or as a methodology. Interpretivism might be thought of as a theoretical perspective, methodology, or epistemology. Discourse analysis is a methodology that is typically associated with qualitative methods, but there are also those who use quantitative discourse analysis methods (e.g. Jacobs & Tschötschel, 2019). Some scholars combine terms to create emphasis and align themselves with particular schools of thought. For one example, there are researchers who speak about onto-epistemology (e.g. Pérez & Saavedra, 2017), which often aligns them with theories and/or methodologies that are more critical, feminist, post-qualitative, and/or post-humanist in nature (e.g. Nxumalo & Cedillo, 2017; Ulmer, 2017). While learning about how these ideas relate to each other, this flexibility within philosophically-defined parameters can be overwhelming. However, the more you learn about these bodies of knowledge, the more comfortable you become with this dynamicity.

Quantitative and Qualitative Research Generally, we can say that researchers conduct quantitative research, qualitative research, or both - mixed-methods research. Broadly, quantitative research can be thought of as “a means for testing objective theories by examining the relationship among variables. These variables, in turn, can be measured, typically on instruments, so that numbered data can be analyzed using statistical procedures” (Creswell, 2009, p. 4). Qualitative research aims to develop deep understandings of the meanings present in and between people and their worlds. These understandings can be constructed within a person or a group of people, often through analysis of observations, conversations, and/or artifact collection. Mixedmethods research combines both quantitative and qualitative approaches, by “collecting, analyzing, and interpreting quantitative and qualitative data in a single study or in a series of studies that investigate the same underlying phenomenon” (Leech & Onwuegbuzie, 2009, p. 267). Those who work in a pragmatist paradigm, which are researchers who focus more on solving the problem at hand than aligning themselves strictly with a single philosophy, are most likely to use a mixed-methods approach (Onwuegbuzie & Leech, 2005). This choice, to study and use one of these approaches – quantitative, qualitative, and/or mixed-methods – connects to one’s ontological and epistemological stances, which again are our beliefs about the nature of reality and how we come to know something. For example, quantitative research is often most closely aligned with positivist and/or postpositivist para­ digms and objectivist epistemology. Qualitative researchers, on the other hand, tend to believe that “reality is constructed by individuals interacting within their social worlds” – that meaning is inherently a socially-constructed process (Merriam, 1998, p. 6.; Merriam & Grenier, 2019). Thus, they will likely align themselves epistemologically with construction­ ism and subjectivism, as well as paradigms that embody these ideals such as constructivism, transformative/emancipatory, and postmodernism. Mixed-methods researchers tend to see value in both types of research, acknowledging that we gain different types of knowledge

15

STRUCTURES OF RESEARCH

from each. Not only are there connections between what we believe about knowing and which of these approaches we choose, these choices also impact what we can know from any given study. With this basic introduction to these approaches, it is important to know that these choices, whether to pursue qualitative, quantitative, or mixed-methods research, have, his­ torically, been hotly contested. One reason regards the epistemological stances present within the paradigms discussed. Think about the differences between objectivism, con­ structionism, and subjectivism, and that they contain core conflicts about how knowledge exists in our world(s). Now, imagine that you are a researcher who believes strongly in one of them. You have dedicated much of your career working within a quantitative or qualita­ tive approach, knowing that there are other education researchers who do not see your approach as valid. Further, these debates are connected to beliefs not just about the nature of knowing, but about which types of knowledge are seen as more valuable or worthy. Some would argue that quantitative research is seen as more rigorous, scientific, and/or valuable than qualitative work. For one example, a randomized control trial, which is a quantitative, experimental method, is, arguably, still seen as “the gold standard” of research. Some do not see this debate with such contention, likely believing that different approa­ ches can teach us different things. Not only would researchers who conduct mixed-methods research fall into this camp, many strictly quantitative and qualitative researchers do as well (Bryman, 2017; Howe, 1988). At the same time, many researchers do lean toward one of these approaches and some caution against mixed-methods because of epistemological con­ flicts (Denzin, 2017).

A Philosophical Fit: Creating Coherent Research We see that there are relationships between the paradigms and epistemologies with which we align, and whether we are more likely to engage in quantitative, qualitative, or mixedmethods research. This is true for each element of research. Each component, from epis­ temology to method, should make sense together. There are traditions here, with parti­ cular methods, methodologies, theories, and epistemologies that are often used together. For example, it could make sense to see a qualitative researcher who aligns themselves with interpretivist theories to conduct a case study that uses semi-structured interviews and participant observations to collect data, followed by a combination of memoing, as well as deductive and inductive coding to begin their data analysis. For another example, it would not make sense for a postmodernist to adopt an experimental design. A core reason for this being the unlikelihood that a postmodernist would believe that objective truth can exist in objects, or that truth can be discovered by a process which mathematically accounts for context. Some of these structures are more rigid than others. The methodology of grounded theory can offer an example here. Grounded theory requires a very particular set of meth­ ods. For instance, since the goal of grounded theory is to generate a new theory based on a researcher’s observations of and identification of meaningful patterns in an environment, open coding must be used in analysis, rather than deductive coding. Some theoretical per­ spectives would likely be called for as well. For example, using interpretivist rather than cri­ tical points of view would make sense because grounded theory calls for striving to enter a space with a blank slate and a critical theorist would be more likely to enter a project want­ ing to highlight a particular phenomenon (e.g. how race and racism shape a practice, how gender is performed, how class is reproduced, etc.).

16

HOW DO WE KNOW?

These theoretical, methodological, and method choices also must align with one’s epis­ temology, as well as paradigm. Researchers do get to make these decisions about their research designs. Yet, while this process is somewhat flexible, there are existing traditions and, regardless, we must justify our choices. This is especially true for scholars who deviate from norms, done often to create a new way to think about the problem or topic at hand. Table 2.1 at the end of this chapter, entitled “Research Paradigms in Early Childhood,” shows how some of these various research elements tend to align within different paradigms.

Unearthing Paradigms and Epistemologies: A Conversation with

Dr. Kate Delaney

When reading a piece of empirical research, you should easily see what methodologies and/ or methods a researcher employs. To an extent, this is also true about theory, but often in a more pointed way than described above. Unless a piece is more conceptual in nature, it is not likely you will read about a researcher’s epistemological stance or about the paradigm(s) to which they align their work. To better understand what these philosophical underpinnings mean, we are going to focus on one early childhood researcher’s work, Dr. Kate Delaney.1 Dr. Delaney is an early childhood researcher and teacher educator. She is trained as a mixed-methodologist. While the majority of her work is qualitative, she has more recently been incorporating quantitative methods into her work. I asked Dr. Delaney about how her beliefs about how we come to know have impacted her work. Here are the questions we used to guide our conversation. � � �

Currently, with what kinds of research paradigm(s) and epistemologies do you align your work? How do these beliefs shape the rest of your research design? The methodologies, and methods you use? The theories you draw from? How, if at all, has this changed for you over time? Kate starts by talking about what research paradigm where she places her work. “I most closely align my research with an interpretivist paradigm. I am really interested in how participants, in my research, make sense of the world around them, including the systems in which they participate.” Kate draws attention to the role power plays in her work and how this relates to how she locates herself within an interpretivist para­ digm. Here she actively foregrounds interpretivism by making a point about how she sees a slight distinction between interpretivist and constructivist approaches. “I think about how power is important to a lot of constructivists and it’s not that I’m not interested in power dynamics, but I guess I’m more interested in lived experience. In the studies I’ve done, I can see power systems and understand power systems, but that’s not necessarily what a person’s day-to-day lived experience is about and I’m more interested in how they make sense and mettle through the system they’re within.” As we talked, at first I was surprised because I think a lot of her work deals with how power operates. But as she continued, I saw how she was foregrounding her participants’ perceptions rather than power itself. We continued to talk through this idea, that people actively make sense of their worlds. At times we found ourselves laughing a bit, because we easily see where our training overlaps and how it impacts our beliefs about knowing in research. We could also see where our approaches diverged. Kate was moving into more mixed-methods

17

STRUCTURES OF RESEARCH

work. I asked her about this and what it meant for her beliefs about knowing. “The quantitative research that I read creates an opportunity for me to see the qualitative question. For me, as a qualitative researcher, I need to understand the quantitative research in order to pick up the qualitative thread and say, okay, this is what we’re seeing in a big data set, but if we look even more closely at a finer grain size of data, we see that this is really actually playing out in very different ways.” Here she made a point about what other knowledge shapes her beliefs, “The influence of developmental science is in the back of my brain here. I can better understand what’s happening if I can see the big data. It helps me know where I want to zoom in, where a question feels unanswered.” Her thinking reminds me of how pragmatists ground their work in what’s most useful. I’m struck by how Kate truly sees the value in the many different frames researchers take to their work. “I’m interested in how participants interpret their roles, but there are other really interesting elements. For instance, how do teachers construct knowledge and senses of themselves within different contexts? I just don’t work in that space. I feel like one of the wonderful things about qualitative researchers is that you can have five qualitative researchers in a room. You can all have the same research context, and everyone is going to examine unique aspects of that context, because it’s so complex, right? The lived experience of children, teachers, and families in sites of care and education are just so complicated.” As we wrap our conversation, she offers advice for all who are new to thinking in this way. Here she cautions against certainty. “Some have such a degree of assumed certainty about their data. If you’re so certain, it’s likely you haven’t spent enough time in the data.” She always seeks the “counterfactual,” the “disconfirming data.” She comfortably admits, “Maybe that is reflective of the paradigms in which we work, the degree to which we can know something.” Kate ends by suggesting, “The most impor­ tant thing is to explain why you did something. And, when making these decisions, always ask – what can be known.” 2

Applying Burgeoning Understandings of Research While Dr. Delaney and I spoke, she also told me about two articles she wrote that draw from the same data set, but where the data are examined through different lenses. The first article is entitled, ‘Looking Away: An Analysis of Early Childhood Teaching and Learning Experiences Framed through a Quality Metric’ (Delaney, 2018) and the second article is ‘Exploring Head Start Teacher and Leader Perceptions of the Pre-K Classroom Assess­ ment Scoring System as a Part of the Head Start Designation Renewal System’ (Delaney & Krepps, 2021). To end this chapter, we use these articles to discuss the idea of what can be known. Before getting to the articles, where they were published speaks to how each piece dealt with the nature of knowing. The first article was published in Contemporary Issues in Early Childhood (CIEC), which is a journal that primarily publishes critical qualitative work. A part of the description of the journal states: CIEC incorporates interdisciplinary, cutting edge work which may include the fol­ lowing areas: poststructuralist, postmodern and postcolonial approaches, queer theory, sociology of childhood, alternative viewpoints of child development, and deal with issues such as language and identity, the discourse of difference, new

18

HOW DO WE KNOW?

information technologies, stories and voices, curriculum, culture and pedagogy, or any combination of such ideas. The second article was published in Early Childhood Research Quarterly (ECRQ), which tends to publish quantitative work. “ECRQ publishes predominantly empirical research (quantitative or qualitative methods) on issues of interest to early childhood development, theory, and educational practice.” Here are this journal’s stated topics of interest: � � � � � �

Children’s social, emotional, cognitive, behavioral, language, and motor development applied to early childhood settings. Childcare, program quality, and children’s transition to school. The efficacy of early intervention and prevention programs. Public policy, early childhood education, and child development. Best classroom practices and effective early childhood curricula. Professional development and training for early childhood practitioners.

If we attend to the terms used in these descriptors, we can connect to the epistemologies and/ or paradigms discussed above. One could argue that it would be more likely for those working within a postpositivist paradigm to publish in ECRQ, and one working with a more critical lens/within a transformative paradigm to publish in CIEC. This likely feels clearer with how CIEC speaks about their work than ECRQ. That said, looking at ECRQ’s statement, consider the word efficacy. The connotation is that particular results are sought. The word feels like it is part of a “scientific” discourse and that there might be some kind of measurement needed to determine efficacy. “Efficacy, best practices, applied,” etc., is not likely to be the verbiage used in a CIEC article. Why? Because each of these journals supports different ways of knowing. Back to Dr. Delaney’s work. Broadly, we can say that both articles regard accountability in Head Start and the roles PreK CLASS3 play in defining a high-quality classroom. Both are qualitative pieces. and it is easy to see Dr. Delaney’s interpretive frame because both articles highlight how teachers have responded to PreK CLASS. At the same time, there are clear differences in the philosophical underpinnings of the two articles. In the CIEC piece, there is a more subjectivist lean, seen in her discussion and application of comic subjectivity theory (Zupancˇ icˇ , 2008). In the ECRQ piece, traditional ethnographic methods are used, which are often acknowledged as rigorous across research traditions, making a qualitative piece more relatable to a more postpositive audience. Each of these articles also ask different questions of the data. The ECRQ article focuses on learning more about teachers’ perceptions of PreK CLASS. Here the authors identify six main findings detailing what the teachers thought of PreK CLASS. The CIEC piece explores how PreK CLASS within a high-stakes accountability context changes what counts as quality and what learning goes unseen during PreK CLASS observations. Children’s voices are included in this article. Here the findings “explore” the ways in which PreK CLASS shapes what quality is, who counts as a teacher, and how power/compliance impact what is possible for teachers and children. In the discussion, Delaney (2018) stated, “this article aims to push back on the notion that quality can be simply known, measured, and used to hold individual programs accountable to one standard or one way of being” (p. 182). The ECRQ article (Delany & Krepps, 2021) contains similar suggestions about variance in definitions of qual­ ity, but since this piece was focused more on unearthing the teachers’ perceptions, the authors focus on “understanding how teachers and leaders experience high-stakes policy” and how these new understandings can help move the work forward.

19

Other Associated Labels



Purpose







Axiology – what is valued

Critical Freierian Neo-Marxist Feminist Action Research Queer Theory Indigenous Critical Disability Theory To liberate

Naturalistic Interpretivist Phenomenological Hermeneutic Qualitative Ethnographic Grounded Theory To understand

Experimental Quantitative Correlational Randomized control trials

Objectivity is critical and obtainable. Reality is reduced to smallest parts that can be observable.

Epistemology – how we know what we know

20

There is a relationship between the researcher and the participants that is made explicit. Meaning is constructed together with attention given to context and complexity.

There is a relationship between the researcher and participants. Action and change are part of the process of knowing.

Meaning is given to the object by the subject. Exploring contradictions.

Reality is subjective, incoherent, and discontinuous.

There is one reality that is knowable.



Ontology – what is the nature of reality Reality is socially conThere are multiple realities structed. There are multi- that are created based on power positioning. Some ple realities. realities are privileged over others and this leads to oppression.

To solve problems

Relationships are defined by what the researcher views as important to solving the problem at hand.

Some may reject any claim to truth. Truth is what is useful. There is one reality but everyone has their own inter­ pretation of reality. What is most important is “what works.”

Flexibly using research tools

To deconstruct Breaking down the grand narratives and questioning reality and systems of “reality.”

Post-paradigmatic Mixed Methods

Pragmatic

Post-structural Deconstructionist Post-Colonial

Postmodern

Context and a balance of Promotion of social justice. An attempt to be free of Positionality of researcher values; objectivity; distance multiple perspectives. and researched. from the researched. and the researcher

To predict

Transformative/ Emancipatory

Constructivism

Postpositivism

Table 2.1 Research Paradigms in Early Childhood

STRUCTURES OF RESEARCH



21

Survey/ques­ tionnaire Assessment instru­ ment Performance data Census/demo­ graphic data Statistical analysis

Quantitative Interventionist Decontextualized

Interview Observation Participant-observation Focus group Audio/visual artifacts Documents Narrative

Qualitative Contextualized Case study Ethnography

Constructivism

Many of the same tools as constructivist with attention to reflexivity and praxis.

Qualitative (perhaps quanti­ tative) including historical and political factors. Issues of power and trust are highlighted to address oppression. Examples: Participatory Action Research Critical Discourse Analysis Critical Ethnography

Transformative/ Emancipatory

Qualitative and quantitative applied to the appropriate questions.

Pragmatic

Reviewing and deconstructing Mix of quantita­ tive and dialogue, historical text, and media. qualitative

Foucauldian archeology Critical Discourse Analysis

Postmodern

Source: Adapted from Creswell (2014); Crotty (1998); Lincoln et al. (2011); Lather (1991, 2006); Mertens (2014). We acknowledge the contributions of Debora Basler Wis­ neski to this table.

Types of Methods – the tools and proce­ dures used to collect and analyze data

Methodology – an approach or system of inquiry

Postpositivism

HOW DO WE KNOW?

STRUCTURES OF RESEARCH

While the interpretive paradigm Dr. Delaney works within drove this project as a whole, including her methodology and methods, in these articles we see how she used different theories, wrote different kinds of research questions, and accessed different discourses while writing, creating philosophically cohesive articles for each of these journals. We see that engaging in these different traditions of knowing is one way that this researcher can intentionally work through a relevant problem of practice that teachers, children, and families face.

Conclusion We end where we started with the question of How do we know what we know? To speak to this question, we must educate ourselves on the philosophical traditions that guide educa­ tion research as a whole. We must think deeply about our own knowledge and the ways we have come to know something. We must not take these understandings about knowledge for granted. Instead, we are vigilant with our thinking, both about what others claim to know and what we seek to know. In early childhood education, we must also be sure not to lose sight of why we commit to developing strong philosophical understandings about the nature of knowledge. While we each create our own why, we recommend that you always keeps the well-being of teachers, children, and families in mind.

Notes 1 Honoring transparency, the author notes that Dr. Delaney and she worked together in graduate school and had similar qualitative methods training. Dr. Delaney however, also trained in quanti­ tative methods and then furthered her mixed-methods skills in postdoctoral work. The author, on the other hand, continued to work on qualitative training with a scholar who often conducted case studies using ethnographic methods. Both these commonalities and differences in training made for an interesting conversation about the philosophical underpinnings of these research traditions. 2 While this is a conversation between colleagues and not a research activity, in light of this book’s topic I captured our conversation via a “vignette,” which is one of many qualitative methods, one that is concerned with representation (Langer, 2016; Richardson & Lockridge, 1998). 3 The PreK Classroom Assessment Scoring System (CLASS) is a standardized observational tool that measures classroom quality by focusing on teachers’ interactions with children.

22

3

ADVANCING EQUITY IN EARLY

CHILDHOOD EDUCATION RESEARCH

Iheoma U. Iruka

The U.S. report from the National Academies of Sciences, Engineering, and Medicine (NASEM, 2023), Closing the Opportunity Gap for Young Children, reminds us that children’s school and life outcomes are often set before birth and during early childhood. That is, the access to and experi­ ences in high-quality, enriching, and health-promoting early learning environments are often tied to one’s sociodemographics, such as race, gender, ethnicity, language, disability, place, and the intersectionality of these identities. Due to these inequities in access and experiences, disparities are seen in health, socio-emotional, cognitive, and academic outcomes. While there have long been recognized disparities in children’s early outcomes based on race, gender, language, poverty, and place, there has been limited attention to their causes (i.e., inequities) and, more importantly, the role of racism, sexism, classism, and other isms that are baked into the scientific enterprise. In this chapter, we seek to unpack how the scientific enterprise can be retooled to advance equity and ensure that future science attends to root causes rather than focusing on “gap­ gazing” (Humphries & Iruka, 2017). We examine the core aspects of research and call attention to how current and future researchers can ensure their science moves from reifying privilege, oppression, and dehumanization and, instead, moves toward opportunity and humanity. While we primarily use U.S.-based examples in this chapter, it is important to note that much of the science conducted by Western countries, including the U.S., continues to reify white supremacy and what many portray as a WEIRD – western, educated, industrialized, rich, and democratic – perspective that is inconsistent with the world population (Henrich et al., 2010a, 2010b).

Defining Racism and Other Isms Racism is the use of institutional power to organize around white supremacy and the oppression, denigration, and dehumanizing of individuals based on their phenotype (i.e., skin color, hair texture, facial features) (Crenshaw et al., 1995; Iruka et al., 2022). This means that white people, people who look white, or whiteness based on language and cul­ ture, benefit from this system and arrangement of power, privilege, and resources. Whiteness is then regarded as beautiful, intelligent, worthy, and something to aspire to, thus main­ taining the racist system. It is important to note that there are different forms of racism, including cultural racism, structural racism, systemic racism, interpersonal racism, inter­ nalized racism, vicarious racism, and so much more, including colonialism. Using the idea of power, we can also define sexism as the organization of power to privilege and benefit one based on their gender, classism as the organization of power based on one’s socio-economic status, and so forth. While racism is a U.S. phenomenon, it is parallel to the global

DOI: 10.4324/9781003354499-4

23

STRUCTURES OF RESEARCH

phenomenon of colonialism, which is a hierarchical global system created by the European settlement to exert absolute domination over the rest of the world, including Africa, Asia, and the Americas, to benefit Europeans economically, socially, politically, and psychologi­ cally resulting in a systematic form of exploitation (Sartre, 2001). Examples of how racism impacts children’s development has been written about in many reports and papers, including various graphics that show how systemic racism impacts chil­ dren’s early development (Shonkoff et al., 2021; Slopen & Heard-Garris, 2021). For example, racial disparities in birth outcomes have been linked to structural, institutional, and interpersonal racism (Dayo et al., 2023; Hailu et al., 2022; Iruka et al., 2022). Black families are, for instance, likely to live in segregated communities with limited resources, especially health-promoting resources, and more likely to live in communities with high levels of environmental toxins, which have been linked to birth defects. Furthermore, the segregation of Black people to homes and housing projects with lead paint and lead water pipes has been linked to poor birthing outcomes. There is also considerable evidence of institutional racism with Black birth people less likely to have access to high-quality culturally responsive birthing clinicians and likely to have low quality rated hospitals and clinics in their community. Additionally, interpersonal racism is likely to impact birthing outcomes with Black birth people less likely to get timely services needed or get their pain adequately addressed when in distress. These cascading experiences of racism, discrimination, and bias have a com­ pounding impact on birth outcomes, with children born pre-term and low-birthweight likely to need cognitive and behavioral services (Hack et al., 1995; McCormick et al., 2006). Although we are now more aware of these racial disparities and some of the root causes, when looking at the data for birthing outcomes, early learning access, and experiences, racial disparities still exist. For example, Black children are 2.5 times more likely to be suspended in preschool compared to children and oftentimes for similar behaviors (Office for Civil Rights, 2014). Black and Latine1 children are likely to be in Head Start and Pre-kindergar­ ten (PreK) programs rated as lower quality (NASEM, 2023); this is even more surprising given the high and strict standards these federal and state-run programs are expected to uphold. Furthermore, data show that children living in poverty, children from immigrant families, and Latine children are less likely to access childcare subsidies (Adams & Pratt, 2021). While there have been some examinations as to the potential reasons for these dis­ parities in access, experiences, and outcomes, the research science regarding structural racism and other systemic inequities has rarely been studied. This lack of attention to the role of systems and inequities ensures that children’s opportunities to learn are based on socio­ demographic factors, maintaining the status quo.

Purpose of Research There is a long history of research being used to maintain racism and other systems of hierarchy and oppression. Interestingly, it was Black bodies and other vulnerable populations that served as the foundations of many scientific discoveries (Macklin, 2013). For example, there is the well-known Tuskegee Syphilis “Study,” which went on for over 40 years. In this study, Black males were coerced into what they thought was a medical treatment but it was instead a deceptive research study that sought to understand the impact of syphilis on the men without their knowledge and consent. Similar research studies have been conducted on other vulnerable communities, such as the Jewish Hospital Cancer Cells “Study” and the Willowbrook Hepatitis “Study.” Willowbrook was a state-run institution for children labeled with retardation who were given live hepatitis cells, often sometimes laced into the fecal

24

ADVANCING EQUITY IN EARLY CHILDHOOD EDUCATION RESEARCH

matter. These sorts of dehumanizing research studies accelerated the move toward the set­ ting up of a bioethics commission, called the Belmont Commission, and then the pro­ mulgating of regulations that we now live under that mandate institutional review boards, informed consent, and protection for the vulnerable. When we turn our attention to early childhood, we see that Black children and families overwhelmingly made up the sample for the HighScope Perry Preschool Program, the Car­ olina Abecedarian Study, and the Chicago Parent Centers, which are often touted as the seminal studies on early childhood intervention, showing that every $1 invested in the early years results in 10–20% return on investment (Heckman, 2011). Sadly, while Black children and families served as the bases for many of these seminal medical and child development discoveries, they did not benefit from them at the same level as their white counterparts, such as their continued inequitable access to high-quality care (Latham et al., 2021) and disproportionate suspension and expulsion rates in early childhood programs compared to their peers (Office for Civil Rights, 2014). Furthermore, even when examining how research is interpreted, it reifies pathology and deficit thinking about Black people, other people of color, and those from communities furthest from opportunities. One of the seminal federal reports that castigate the Black community based on the data used, analyzed, and interpreted – the Moynihan Report i.e., The Negro Family: The Case for National Action (Moynihan, 1965) – provides a classic case as to how one’s theoretical framework and understanding of context could skew one’s per­ spective, which can have lifelong consequences on children, families, and communities. The report begins by noting the role of racism and discrimination: First, the racist virus in the American blood stream still afflicts us: Negroes will encounter serious personal prejudice for at least another generation. Second, three centuries of sometimes unimaginable mistreatment have taken their toll on the Negro people. The harsh fact is that as a group, at the present time, in terms of ability to win out in the competitions of American life, they are not equal to most of those groups with which they will be competing. Individually, Negro Americans reach the highest peaks of achievement. However, the report concludes that “The fundamental problem [with the Black family], in which this is most clearly the case, is that of family structure. The evidence – not final, but powerfully persuasive – is that the Negro family in the urban ghettos is crumbling.” The data used to point to the disintegration and pathology of the Black family primarily rests on single, female-headed households who are receiving federal dependent aid in contrast to white families. Recognizing that this pathologizing view of Black families was inaccurate, especially when devoid of context and limited recognition of the systems of oppression that continue to impact the everyday life of Black families, Dr. Robert Hill of the National Urban League used national data to identify the strengths of Black families. In his report, Hill (1972) identified five strengths of Black families, which are the “adaptations necessary for survival and advancement in a hostile environment.” These include: (1) strong kinship bonds where Black families are likely to formally and informally adopt relatives; (2) strong work orienta­ tion with the majority of Black adults in households working, though often paid less than their white counterparts; (3) adaptability of family roles with a sharing of decisions and jobs in the home to ensure home stability; (4) high achievement orientation with many Black college attendees likely to come from homes where parents did not go to college and less

25

STRUCTURES OF RESEARCH

likely to drop out of college; and (5) religious orientation which is the use of spirituality as a source of resistance and survival mechanism. Similarly, Yosso (2005) emphasizes in her work that while many people may think of cultural wealth as residing in mostly white, middle-class spaces, communities of color have much cultural wealth that must be recognized. That is, it is critical to acknowledge the strengths of Black people and other people of color rather than a deficit frame that takes the position that minority children, families, and communities are at fault for poor outcomes from their health, wealth, and educational challenges. Yosso calls out six types of cultural wealth that communities of color activate to address the oppressive system and interactions they are subjected to. 1 2 3 4 5 6

Aspirational capital refers to maintaining hopes and dreams for the future, even in the face of real and perceived barriers. Linguistic capital includes the intellectual and social skills attained through commu­ nication experiences in more than one language and/or style. Familial capital refers to the cultural knowledge nurtured among familial (kin) that carries a sense of community history, memory, and cultural intuition. Social capital can be understood as networks of people and community resources. Navigational capital refers to skills of maneuvering through social institutions. Resistant capital refers to the knowledge and skills fostered through oppositional beha­ vior that challenges inequality.

In expanding this framework, Iruka et al. (2022) add a seventh, spiritual capital, which refers to beliefs, knowledge, values, and dispositions that drive societal, organizational, and interpersonal behavior based on the notion of a higher power. Thus, the purpose of research must be to uncover the strengths and challenges experi­ enced by diverse communities through an asset-based frame that recognizes the historical and contemporary context of oppression that may limit opportunities. Future research must recognize the oppressive foundation of scientific discovery and ascertain how this must be rectified in future studies and reports to advance the science that improves lives and well­ being, especially for those from minoritized backgrounds.

Attending to Equity in the Research Process Research often starts with an inquiry, seeking to answer various questions or address gaps in knowledge to advance practice or policy. Taking early childhood as an exemplar, the seminal studies sought to address the “cultural deprivation” that many children, primarily Black children, experienced and sought to remedy that through early childhood intervention. The underlying theory was that children living in poverty did not get the enriching experiences that prepared them for school and, thus, needed remediation to get them on par with higher-income, predominantly white, families. One of these early childhood seminal stu­ dies – the Carolina Abecedarian Study — occurred in the 1970s in the Chapel Hill, North Carolina area. The Carolina Abecedarian Project involved 111 low-income, predominantly Black children and families as part of a randomized control trial seeking to examine the impact of receiving an intensive early education program that provided a supportive learning environment to prevent school failure for children with mild retardation, beginning in infancy (Campbell & Ramey, 1994). The learning environment encompassed a series of game-like activities to promote cognitive, language, perceptual-motor, and social develop­ ment (Sparling & Lewis, 1979). Through careful monitoring of children, new learning

26

ADVANCING EQUITY IN EARLY CHILDHOOD EDUCATION RESEARCH

games were introduced to children over time. A follow-up study at the age of 30 showed that compared to the control group, the treatment group had more years of education (12.31 vs. 13.46), were likely to have a high school diploma (72% vs. 83%), were less likely to extensively use public assistance (4% vs. 20%), and worked full-time (53% vs. 75%). There were no differences in income-to-needs ratio, criminal activity, marital status, social adjust­ ment, mental and physical health, and substance use at age 30 (Campbell et al., 2012). While this seminal early childhood education study, as well as the HighScope Perry Pre­ school Program and Chicago Parent Centers, provide substantial evidence about the importance of high-quality early education intervention for children living in poverty with mild cognitive impairment, they continue to instantiate that Black children require sub­ stantial intervention due to their limited cognitive capacity and poor home environment. This idea of cultural deprivation and sub-personhood was promulgated through the eugenics movement, which viewed Black people in particular (and Indigenous and Chinese people) as less than human and thus not deserving of full human rights (Gillbon, 2010; Opara et al., 2022). As noted by Bruno and Iruka (2022), the Carolina Abecedarian research took a “colorblind” approach that rarely considered “the sociohistorical context of the race of the children in the sample” for the early childhood intervention or the interpretation of the findings, and “race as a contextual factor was rarely considered in the publications that reported the key study findings” (p. 168). While Bruno and Iruka acknowledge the impor­ tance of the study, they recognize the context of the study and implore current and future research and researchers to move towards anti-racist research that sees the beauty of racial diversity and makes racist systems and practices visible through science. In addition to calling for researchers of color to be provided with leadership roles, Bruno and Iruka call for using critical race theories and theorists to investigate traditional constructs, phenomena, and assumptions.

Research Questions The historical and contemporary influence and impact of racism, discrimination, and bias were not evident in these studies, even though these studies occurred at a time when Black people were demanding civil rights and equal opportunity. Considering the extensive impact of racism and other systemic inequities because of one’s social identities, such as race, eth­ nicity, class, and immigration, advancing research requires attending to macrosystemic fac­ tors, and then microsystemic factors. As noted by Rogers et al. (2021), “A consequence, then, of centering individuals and microsystems is that our science often obscures the per­ sistent and pervasive ways that social hierarchies and ideologies of white supremacy and antiBlackness shape every facet of human development” (p. 271). Decontextualized research negates the impact of these socio-historical forces on families and subsequently children, instead turning the gaze on children and seeking to “fix” them and prepare them to perform like their white and advantaged peers and blame them and their families when they underperform. Thus, the research process, such as the research questions being examined, must be care­ fully examined to ascertain the extent to which they maintain narratives of subjugation of particular groups or if they take a more holistic and humanizing approach. For example, many questions in early childhood center on the premise that certain groups of children are not measuring up to particular metrics by formal school entry. For example, analyses from the U.S. Department of Education, Birth and Kindergarten cohort data often show that children from low-income households, Black and Latine children, and dual language learners

27

STRUCTURES OF RESEARCH

show lower scores on various assessments that measure reading, math, and executive func­ tion compared to children from high-income households, white children, and monolingual English speakers (West, 2017). This then leads to the questions focused on how to improve these child-level outcomes, examining factors that contribute to these low outcomes (i.e., the home environment and parenting), and how to ameliorate these low child outcomes through early childhood interventions. While these questions are critical to support chil­ dren’s school and life success, for research to be a tool to ensure progress, especially for children from historically marginalized communities, there is also a need to query whether the outcomes are culturally valid, whether the lens of examining families and home envir­ onments are asset-based, and whether the interventions are culturally meaningful and valid and address the root causes. For instance, Gardner-Neblett et al. (2023) call out five pro­ blems with the Black–white achievement gap paradigm, which include setting white children as the ideal, elevating Eurocentric standards, ignoring Black children’s competencies, relying on flawed assessments, and numbing teachers to Black children’s educational underperformance. They call for attending to structural racism as the root cause of Black chil­ dren’s educational outcomes, from housing segregation to inequitable educational resources, attending to Black children’s competencies, such as their oral language skills to socio­ emotional skills, and engaging culturally relevant pedagogy that builds on Black children’s strengths and experiences. Even in studies that “blame” families for children’s poor outcomes, there is a need to examine whether what is considered “poor” parenting has the same meaning across racial and cultural groups. In their investigation of the extent to which maternal intrusiveness and warmth predicted mother–toddler relationships 10 months later, Ispa et al. (2004) found that while parental intrusiveness predicted increases in later child negativity in all groups (i.e., white, Black, more acculturated Mexican American, and less acculturated Mexican American low-income families), this relationship was moderated by maternal warmth only for Black families. Furthermore, parental intrusiveness predicted negative change in child engagement with mothers for only white families. This differential relationship between what is often termed “negative parenting” and child outcomes across racial groups has been found in other studies (Dotterer et al., 2012; Iruka et al., 2012; Pungello et al., 2009), indicating that the meaning and valence of some of the parenting constructs and variables operate within sociocultural contexts, resulting in different child outcomes.

Theoretical Frameworks Considering the sociocultural environments that children of color are likely to live in, there is a need for theoretical frameworks that address their unique experiences and contexts that creates disadvantages and exclusions and their impact on child development. Much of the early childhood research has leaned on Bronfenbrenner’s (1989) bioecological theory. His bioecological theory attends to how children’s development is influenced by the microsystem, which are proximal contexts such as the home and school environments; the meso­ system, which is the connection across microsystems, such as the relationships between families and schools; the exosystem, which is the social environment that indirectly influ­ ences a child’s development such as the geographic locale (e.g., rural, urban) and parental work environment; and the macrosystem, which is the value, norms, and policies exhibited within the various systems. While it could be extrapolated that the bioecological theory can attend to issues of racism and classism, it was not explicit. Thus, García Coll and colleagues (1996) developed the integrative model for the study of developmental competencies in

28

ADVANCING EQUITY IN EARLY CHILDHOOD EDUCATION RESEARCH

minority children to examine how one’s social position (e.g., race, social class, ethnicity, gender) influences one’s experiences of racism, discrimination, and oppression and level of residential, economic, social, and psychological segregation leading to environments that could be promotive or inhibitive. Promotive and inhibitive environments influence the adaptive cul­ tures that are then created in response to these experiences, such as family traditions, accul­ turation, structure and role, beliefs, and socialization practices. Promotive and inhibitive environments, and family adaptive cultures, beliefs, and practices, also interact with children’s characteristics (e.g., age, temperament, physical characteristics): “children’s developmental competencies emerge as a direct function of individual contributions of adaptive culture, family processes, and the child’s own characteristics operating through the interactions among these systems of influence” (García Coll et al., 1996, p. 1897). For children from racially and ethnically minoritized communities, the idea of competence must be expanded beyond tradi­ tional measures to include skills developed in contexts created by social stratification and oppression, such as being bicultural and bilingual and coping with racism. Extending this theoretical framework, in combination with critical race theory, structural deter­ minants of health, and the life course theory, Iruka and colleagues (2022) developed the Racism + Resilience + Resistance Integrative Study of Childhood Ecosystem (R3ISE integrative model). R3ISE is an integrative model that recognizes that to support children’s optimal devel­ opment and ensure they thrive, we must identify the interplay between racism and family and community cultural assets. This model emphasizes the multidimensionality of racism – taking into account children’s gender, age, sex, and health – starting with cultural racism, which is deeply rooted in the cultural fabric of US society (“the water we swim in”) and is perpetuated through societal structures and institutions (i.e., structural/institutional racism), racist interactions between individuals experienced either personally (i.e., interpersonal racism) or indirectly (i.e., vicarious racism), and adoption of racialized attitudes about one own’s racial group (i.e., internalized racism) (Iruka et al., 2022, pp. 118–119). The goal of a theoretical framework of this nature is to help researchers become aware and understand how multiple forms of racism and other systemic inequities, individually and collectively, impact children’s development over their life course, and the importance of researchers attending to historical and contemporary inequities. It allows researchers to explore how racism and other isms, understudied in early childhood, influence key mechanisms that affect children’s learning and development. While much of the research has attended to some form of interpersonal racism, such as bias in interaction, there has been limited attention to cultural, structural, and institutional racism. Furthermore, the R3ISE integrative model calls for attention to family and community cultural assets to capture the coping mechanisms that racially minoritized and culturally marginalized groups activate to deal with racism and other dehumanizing experiences, systems, and policies (Yosso, 2005). Thus, attending to the theoretical framework can help research attend to root causes or, at the minimum, recognize that findings may be due to invisible systems.

Analytical Approach Without considering the racialized nature of associations, researchers may assume universal associations and relationships. While there are universal associations to ensure programs, policies, and interventions are effective, it is critical to ensure that the problems, solutions, and effectiveness are meaningful across various sociodemographic identities from race, gender, class, and place. Most research tends to “control” for noise, such as race and class. It should, however, be recognized that race (and ethnicity)

29

STRUCTURES OF RESEARCH

is a social construct primarily based on skin color, hair texture, and other physical attributes. It is a societal social structure engineered to communicate who has power and who does not … there is no biological underpinning for race, yet it operates via societally created notions of difference and superiority. (Iruka et al., 2021, p. 4) Thus, there is a need to treat race and other sociodemographic characteristics that communicate social position as a critical part of the research study, especially the analytical process. Using critical race theory applied to quantitative analyses called QuantCrit, Gillborn et al. (2018) look at how numbers are used to disguise racism in education and protect the racist status quo, that is, a position of White supremacy where the assumptions, interests, fears, and fantasies of White people are placed at the heart of everyday politics and policymaking. (p. 160) Quantitative approaches, compared to qualitative approaches, are often viewed as preferred and more objective because they provide numbers that are factual and indisputable. Gillborn and colleagues (2018) note how numbers such as those used to rank countries on education achievement often don’t note the small sample size, selective curricular coverage of tests, who actually takes the assessments, and the different assessments used. And yet, these data are then used as accountability without inquiring whether the underlying numbers are valid. In another example, the authors show how the aggregation of all non-white groups versus white into one can make invisible the inequities that may be experienced by different racial and ethnic groups. Gillborn et al. (2018) outlined five principles for QuantCrit. The first, the centrality of racism, acknowledges that race as a stand-in for racism is insufficient because of its multi­ dimensional, fluid, and changing nature. Race is a “crude approximation [of racism]” that can be easily misinterpreted and misunderstood, especially when framed through a white supremacy lens that reifies the superiority of one racial group. The second principle, num­ bers are not neutral, emphasizes that numbers have often been used to benefit whiteness and do not consider the central role of racism, such as building into predictive models that Black people are likely to perform lower on achievement tests and normalizing underperformance in analyses. The third principle is categories are neither “natural” nor given: for “race” read ‘racism.’ Researchers must take caution in how they treat race categories as static; often viewing minoritized groups as inherently deficient without recognizing how racism may be operating. Researchers also need to examine why certain patterns are racialized and must couple race and racism together when conducting analyses to limit the tendency to treat a racial group as having an underlying problem. The fourth principle is voice and insight: data cannot ‘speak for themselves,’ which means that data can be interpreted in many different and conflicting ways. People view quantitative analyses as exact and able to inform us how specific factors lead to specific outcomes, ignoring the complexity of life and system, including the impact of racism over the life course. “[T]herefore, QuantCrit assigns parti­ cular importance to the experiential knowledge of people of color and other ‘outsider’ groups … and seeks to foreground their insights, knowledge, and understandings to inform research, analyses, and critique” (Gillborn et al., 2018, p. 173). The final and fifth principle is using numbers for social justice by working with numbers that tell the whole story and the impact of racism and against numbers that reify deficit narratives and maintain white supremacy.

30

ADVANCING EQUITY IN EARLY CHILDHOOD EDUCATION RESEARCH

In a similar vein, Garcia et al. (2018) point to the historical legacy of quantitative analyses to maintain oppression, but indicate QuantCrit “has the potential to be used for racial jus­ tice and other liberatory projects” (p. 154), but it has to come to terms with quantitative analysis’ “historical, social, political, and economical power relations” that are embedded in the eugenics movement. Therefore, researchers must avoid the “blind” use of quantitative methods without attending to their social position and axiological (the nature of ethics), ontological (the nature of reality), and epistemological (the nature of knowledge and the relationship between the knower and that which would be known) praxis when committed to social justice research (Mertens, 2012).

Interpretation and Dissemination The final cycle in the research process is often the interpretation of the findings and the dis­ semination. Oftentimes, this includes examining how the findings are consistent (or not) with the extant literature and the research’s theoretical framework. First, one must recognize the bias that exists in the science enterprise that produces the extant literature. The science enterprise includes primarily white funders, lead researchers, and journal editors and reviewers, potentially dismissing the role of racism and other forms of oppression in the lives of children, families, and communities. In their paper discussing the history of the scientific enterprise’s support of gender, identity, and racial inequality, Graves Jr. et al. (2022) discuss how historical forces … have led to a science system within the United States that remains restricted to an exclusionary subset of society. That fact influences literally everything else about science: who participates in science, who sets research agen­ das, who benefits from science, and the degree to which scientific outcomes are accepted by the public. (p. 3) They call for a new science agenda where scientists assess their roles and their responsibility to a just society where findings benefit all and not just the privileged and do not exacerbate harm. In addition to addressing equitable funding and the inclusion of minority scientists and institutions, Graves Jr. and colleagues (2022) call for accountability systems to address “insidious forms of gatekeeping, misconduct hidden within settlement agreements, and social group codes of silence” (p. 9). The interpretation of findings is also based on the theoretical framework used to sub­ stantiate the premise of the study. Thus, when there is no explicit attention to racism and other forms of oppression, then these factors won’t be discussed in the interpretation and implication of the findings. For example, when one talks about the underperformance of Black, Latine, and Indigenous children, dual language learners, children living in poverty, and children with a disability on various outcomes, there needs to be an explicit examination of systems that limit equitable access and experiences due to their race, language, class, and ability. Without recognition and interrogation of these systems and opportunities, the blame is placed on the shoulders of children. If not, the subsequent responses will entail fixing the children and their families without attending to the larger systems that view them from a deficit lens. Thus, it is critical to consider the sociopolitical and historical contexts of the study and incorporate the perspective of participants and communities to accurately interpret the findings. As shown in Table 3.1 below, based on the work of Nelson et al. (2020), there are problematic and positive approaches to reporting and dissemination.

31

STRUCTURES OF RESEARCH

Table 3.1 Problematic and Positive Practices in Reporting and Disseminating for Racial Equity Problematic Practice

Positive Practice

Creating a report that is inaccessible to a general audience (e.g., paywall, dense report)

Creating a range of communication products accessible for free from online and offline sources for diverse audiences free from jargon

Providing data that are not actionable or punitive

Reporting data in an actionable way to improve the lives of participants

Describing experiences in an aggregate form without analyzing by sociodemographic characteristics and their intersectionality

Providing access to disaggregate data and acknowledging structural racism and other forms of oppression embedded in the data

Providing data without context or discussion

Including stories to complement quantitative data and better explain, contextualize, and humanize the numbers

Not providing details about the analytic approach that can be reproduced

Providing clear documentation of the analyses and access to the data, when possible, to maintain security and anonymity of respondents

Disregarding how findings will impact communities, especially historically marginalized communities

Conducting a racial equity impact analysis at different junctures asking whether the project is exacerbating, improving, or masking racial disparities and other disparities

Note. Adapted from Nelson et al. (2020).

In sum, advancing equity in research must go beyond changing measures and inviting more communities of color into the study to one that holistic in approach that starts with the researcher examining their positionality in the study (Jacobson & Mustafa, 2019). Positionality is the ‘position’ from which we see and interpret the world (and how the world sees and interprets us). One’s position is affected by how, where, and when one is located in society. Understanding our social positions helps researchers to know their relations with and to others, especially the community members they may work with through their research activ­ ities. Appreciating their positionality in society allows researchers to also become aware of the power dynamics that operate in their relationships with participants and others who interact with their research. Beyond that self-awareness about positionality, it is critical for researchers to examine why they are engaging in a particular research or area of study, whether it is for the sole benefit of their scholarship or also to benefit the larger society and address inequities. Furthermore, there is a need to also critically examine the research questions, theoretical fra­ mework, and methodology, including the analytical approach, and the interpretation and dis­ semination of findings. A key part of engaging in anti-racist research requires engaging in R.I. C.H.E.R. actions (Iruka, 2022; Iruka et al., 2023). The R.I.C.H.E.R. approach seeks to limit the dehumanization of all people and eradicate racism and bias in all aspects of our world, including in programming, policy, and research. R.I.C.H.E.R. actions include the following: � �

Re-educating researchers about the history of the U.S. and global society where various groups have been systematically oppressed because of their race, gender, religion, sexual orientation, and ability level. Integrating rather than just desegregating our research teams, discipline, and perspectives to find solutions to global problems to advance the human condition.

32

ADVANCING EQUITY IN EARLY CHILDHOOD EDUCATION RESEARCH

� �

�

�

Critical consciousness as a necessary approach to engaging in equity-centered research that begins with one being aware of one’s positionality and the racialized context they live in. Humility of privilege is necessary when engaging with research participants considering the power dynamics inherent with researchers often viewed as the expert. Research must center on ensuring human dignity and not viewing communities through a lens of deficiency and pathology. Erasing racism and other -isms should be central to equity-centered research. Recog­ nizing the damaging, dehumanizing, and oppressive nature of racism and other isms, such as classism, sexism, and ableism, it is paramount that research makes these systems of power and privilege visible and uncovers ways to dismantle their detrimental impact. Re-visioning different ways and approaches is necessary to meet the adversities facing many children, families, and communities.

Meeting the vision of the R.I.C.H.E.R. framework will require a radical mindset recognizing that research can be a tool for human progress when operating from a critical consciousness lens. As noted by Brazilian educator Paulo Freire: A critically conscious person is aware of (a) the historical, political, and social impli­ cations of a situation (i.e., the context); (b) his or her own social location in the context; (c) the intersectionality of his or her multiple identities (e.g., race, socio­ economic class, gender, sexual orientation); and (d) the inherent tensions that exist between a vision of social justice and the current societal conditions for all people. (Landreman et al., 2007, p. 276) Freire believed that if people were to become critical, escape a naïve consciousness, and increase their capacity to reject the prescriptions of others, progress could be made toward dismantling systems of oppression. A critical conscious scientist can see how racial disparities can be traced back to systemic inequities and failures and seek to uncover the system operators. While there is not an explicit agreement about what critical consciousness is, there is an indication that it includes a cognitive component (i.e., awareness of injustices), attitudinal component (i.e., belief in the capacity to effect change), and a behavioral component (i.e., action) (Jemal, 2017). Jemal (2017) further lays out that specific strategies and tools are needed to go from con­ sciousness to action: (1) dialogue and critical reflection that discusses the status quo and privi­ lege; (2) reflective questioning that often asks about root causes and role of race, class, and gender; (3) psychosocial support that allows people to be socially supported as they observe and participate in actions that challenge social inequality; (4) co-learning with others to enhance one’s ability to bring their perspective and experiences while also learning from others in a nonoppressive environment; (5) group process where individuals engage in small group discussions and interactions with an open mind that has the capacity to change; and, finally, (6) action and identity development, a process where individuals begin to confront oppressive systems, result­ ing in new insights about how they see themselves and identify with diverse groups.

Conclusion NASEM’s Closing the Opportunity Gap report made the following recommendation: Early learning and K–12 education systems, health care systems, and employers should test and institute policies and protocols for identifying and addressing

33

STRUCTURES OF RESEARCH

manifestations of institutional racism to reduce inequities in access to resources and quality services in education, health care, and public health. (p. Summary-11) Meeting this recommendation will require the research enterprise in early childhood to attend intentionally and specifically to operationalizing, systematically measuring, and addressing the multiple forms of racism and their impact on the healthy development of children. In this chapter, we sought to provide ways this can be done, from attending to the positionality of researchers to addressing inequities inherent in the scientific process from the research ques­ tions, theoretical frameworks, and analytical approaches to the dissemination. Engaging in research that addresses racism and other isms requires an anti-racist mindset that attends to equity by addressing the historical manifestation of oppression and ensuring that the research is fair and humanizing for all, especially those from marginalized communities.

Note 1 Consistent with experts in the field, we use Latine to refer to individuals whose cultural background originated in Latin America. Rather than using Latinx, a term Spanish speakers find unpronounce­ able in Spanish, we have opted to use the gender-inclusive term Latine, commonly used throughout Spanish-speaking Latin America.

34

4

THE CULTURE AND CONTEXTS OF

RESEARCH IN EARLY CHILDHOOD

EDUCATION

In this chapter we consider the contexts of research, by which we mean the larger social, cultural, political, and historical influences that shape how we view research and judge what research is appropriate. We consider how ideas and values of the larger society influence what research is conducted, how researchers approach their work, and the groups utilized as participants. The goals and aims of education in the United States have shifted over time. Accordingly, what the field of education needs and expects from research has also shifted. Early childhood education has followed an interesting path within these contexts, and in this chapter we discuss how this has played out. Considering the contexts of research leads us necessarily to questions of ethics related to how, when, where, why, and with whom research is conducted. This is another focus of this chapter. There has been a significant shift over time in how researchers conduct their pro­ jects and in the stances they have taken regarding their relationship to the work. We note how the quest for ethical practices affects inquiry within the differing paradigms of educa­ tional research. Finally, we illustrate the “human-ness” of the research enterprise, considering the realities of passionate disagreement among researchers and the stakes of the situations within which policy-makers and researchers interface. We argue that we cannot ignore the “broad moral and political frameworks that undergird research” (Howe & Moses, 1999, p. 33), in both how researchers approach their work and in how research is ultimately used (or not) to shape policy and practice in educational settings.

Contexts Social sciences research is a social activity where we are interested in learning more about the relationships people have with each other, to events and social situations, and to the institutions that govern their lives. What we believe to be important, what we value, and what we need to understand as a society or as a culture at any point in time is shaped by historical moment, dominant political ideologies, the technologies of the time, and the prominent paradigms of thought. Thus, education as an institution has moved through different phases regarding what we deem as its purpose and goals and, so follows, the purposes of and needs for educational research. Research in early childhood education has thusly flowed along this stream. Following, we describe some important and influential moments across the last century that help us understand from where research has come and where it might help us to go.

DOI: 10.4324/9781003354499-5

35

STRUCTURES OF RESEARCH

In the United States in the early years of the twentieth century, social institutions were shifting in importance and influence, and the population was rapidly changing. At this time, the country struggled to define the aims and goals of education (Kliebard, 2004). The major belief was that we needed to study education scientifically, indicating research needed to be entirely empirical and analytical. Those studying social behaviors and phenomena attempted to describe “law-like theories of social behavior” (Bloch, 1991, p. 100), and researchers sought to find a universal truth that was objective and context-free. Researchers believed they could isolate variables for study without regard to context, resulting in research that could predict future behavior based upon the relationships found among variables. There was also a belief that research (theory) was distinctly separate from practice and, hier­ archically, that theory should serve to inform practice (Popkewitz, 1984). There was con­ viction in the ability to determine “exactly what should be taught in schools and how educational knowledge should be structured” (Scott, 2008, p. 6). Early childhood research at this time was heavily influenced by the doctrine of developmentalism. The focus of research was on children’s development, rather than being directed toward learning and/or curriculum (Saracho & Spodek, 2013). This doctrine held that we could determine exact and universal understandings about how children developed; the goal was to optimize children’s development via scientific inquiry (Beatty, 2005). Scientific inquiry would be “the model for truth, [would provide] definitions of valuable knowledge, a way to get factual information about ‘normal’ child development, and [offer] guidance for pedagogy” (Bloch, 1992, p. 9). The developmentalist movement, led by G. Stanley Hall, operated from the belief that through careful observation and documentation we could literally catalog what happened in the minds of children. Observable and quantifiable behavior was key, and the research was conducted via surveys, careful observation, and recording of children’s behaviors at various stages of their development. Hall’s work was highly influential in advocating for the impor­ tance of including children as a worthwhile area of study, and it opened the door to serious attention being paid to children by the fledgling psychological sciences (Brooks-Gunn & Johnson, 2006). Bloch (1991) noted that Hall’s notions “were used to begin the child development movement which related psychology to science, psychology to child develop­ ment, and psychology to the study of pedagogy and curriculum” (p. 100). Moving forward, a highly influential researcher of the mid-twentieth century was Arnold Gesell, who charted descriptions of children’s development, believed to be genetically programmed and to unfold within an appropriately supportive environment. Goffin (1996) pointed out that the ‘scientism’ of research in early childhood at this time supported the professionalization of the field. She noted that there was an “undervaluation of children as public responsibility,” and great faith was put into the ability of “scientifically derived solutions” in the form of “predictable and achievable child development outcomes” (p. 125) to lead advocacy for higher quality early childhood settings with better prepared and better compensated personnel. It was also during the early twentieth century that nursery schools, first institutionalized in Great Britain, were imported to the United States. Women, both philanthropists and pro­ fessionals, were key figures in establishing supports for research associated with the growth of nursery schools. For example, Cora Bussey Hillis, a citizen-advocate, led efforts to estab­ lish a research station in Iowa focused on children’s development. Key introductory early childhood texts often cite Lucy Sprague Mitchell and Caroline Pratt as key early practi­ tioners/leaders within the field who, along with Harriet Johnson, also made an early impact on research by founding the Bureau of Educational Experiments (Beatty, 2005). In the

36

CULTURES AND CONTEXTS OF RESEARCH IN EARLY CHILDHOOD EDUCATION

1920s, large influxes of funding were distributed via the Laura Spelman Rockefeller Mem­ orial to support research associated with nursery schools and the growing number of research institutes across the country. These early efforts helped to clearly establish early childhood research as a field worthy of attention, influenced greatly by developmental psychology, and marked by a mission to understand and nurture children’s development. However, there was little attention to context or to diversity amongst the children and families who participated in this research. It was common for generalizations about children’s development to be made after studying only White middle-class children. This increased interest in the field of early childhood research impacted perspectives regarding earlier efforts. Despite Hall’s early dominance, his methods were eventually criti­ cized as not being scientific enough. Hall’s methods led him (and many of his students) to begin to draw conclusions about biological determinism – the belief that genetic inheritance determines development and learning. By being able to exactly determine the developmental path of a child, curriculum could be developed with a probable destination (Kliebard, 2004) in mind. Yet, these kinds of determinations were roundly critiqued by social scientists in education who had begun to consider how social contexts and social structures could impact a child’s access to opportunities. On the heels of the critiques of biological determinism as a means to shape what occurred in schools, a social meliorist faction in educational research emerged in the middle decades of the twentieth century. This shift stemmed initially from the Great Depression of 1929– 1939, which had engendered an “undercurrent of discontent about the American economic and social system” (Kliebard, 2004, p. 154). This period spawned an increased awareness over time regarding the inequitable distribution of opportunity across social groups, and social unrest resulted. Research in early childhood began to change, with streams of inquiry focused on learning and programmatic interventions that could help to improve the lot of children and families who were deemed disadvantaged. Around the same time, psychologists began to study learning processes, influenced by ideas in the modern world that the early years could have an impact on children’s developmental course. For example, Piaget’s theory and Bloom’s assertions that the preschool years were particularly important to learning spurred interest in research (Saracho & Spodek, 2013). Additionally, some early childhood researchers began to define their work more strongly within the educational field, exploring questions about curriculum. As a result of these research efforts, programmatic interventions for children living in poverty sprang up, framed around the vision during these times that the children were “culturally deprived.” During the 1960s, research was conducted on intervention programs, including, but not limited to, Head Start. In addition to attempts to determine program effectiveness, this body of research included examination of various curriculum models (Consortium for Longitudinal Studies, 1983). These researchers utilized more complex designs, such as randomly assigning children to classrooms or classrooms to curriculum models (this is used to create groups that are roughly equivalent). In addition, several types of standardized, direct assessments of children’s learning and development were included, such as achievement and IQ tests. The importance of this research was that it marked a turning point in expectations for early childhood education. By the 1960s, more complex research methods from experi­ mental psychology were being used to examine the impact of educational experiences on children. Additionally, researchers conducted studies that targeted children in particular cir­ cumstances, such as living in poverty. This research firmly established the place of

37

STRUCTURES OF RESEARCH

quantitative methods in looking for effects deemed to be significant. Hypotheses were pro­ posed and tested via statistical analyses, and the results were assumed to be generalizable to other, similar programs or situations. The goal was to find evidence that early childhood education programs provided benefits for young children. As time went on, quantitative methods were used to answer questions regarding the impact of many programs and policies. For example, following the rise in usage of childcare, researchers explored a series of questions, commencing with determining whether being in child care was harmful or beneficial. From there, researchers moved toward questions that complicated the initial suppositions about childcare’s effects by including attention to dif­ ferences in quality of care and the family context (Phillips & McCartney, 2005). Currently, with universal prekindergarten programs increasing in number, research examining program effectiveness has emerged (for example, see Wong, Cook, Barnett, & Jung, 2008). Another area of high research activity in the current era has been early literacy learning, spurred in part by federally-funded reading initiatives (for example, see National Early Literacy Panel, 2008). Other, smaller-scale studies examine many aspects of early childhood education, ranging from topics such as multi-language development, teacher–child interactions, peer play, components of literacy and math learning, and children’s self-regulation. Therefore, as can be seen, throughout the history of early childhood research, researchers have pursued studies that draw upon quantitative methods; the dominant form of published research in many journals until more recently. These studies most closely align with what is familiar to the public regarding “science” – questions or hypotheses are posed, data are col­ lected in numerical form (i.e., scores on assessments, ratings on observation instruments, counts of behaviors, ratings on surveys or questionnaires), and then analyzed via statistics. This history parallels that of the early decades of general educational research. The importance of quantitative measurement was, in fact, evident in the early twentieth century in the training of educational researchers and amongst those who founded the American Educational Research Association (Lagemann, 1997). Bloch (1987, 1992, 2000) helps us understand, however, that the paths of early child­ hood education and general education research did also start to diverge during this time. Social reformers/activists, and those desiring to realize the potential of education to address inequalities, were more successful in shifting the gaze of education research and curriculum in general education research. Separate schools of education emerged at the university level during the first half of the twentieth century and alternative research paradigms proliferated. Meanwhile, in many cases, early childhood programs and departments remained within or heavily aligned with developmental psychology, focusing on individual development and family influences in order to remain a “hard” science. This was perpetuated by the need to push the professionalization of the field (Goffin, 1996), which was highly feminized, and so lower in regard. This means there are some important moments where early childhood research diverged from (if not lagged behind) a larger ‘interpretive turn’ taken in the social sciences and the general education research agenda in the mid-twentieth century (Rabinow & Sullivan, 1987). Social relationships, writ large and small, became the units of analysis. Goals were to understand human relationships and experiences, rather than outcomes related to proof or prediction. Rather than the almost singular focus on individuals and behavior, there was a shift to understand better how people engaged together and with the social settings within which they moved. The group dynamics of dominant culture and minoritized groups and how institutions and ideologies both shaped and were shaped by the agentic actions of people also came to the forefront. The postmodern thought movement began to take hold

38

CULTURES AND CONTEXTS OF RESEARCH IN EARLY CHILDHOOD EDUCATION

during the latter decades of the twentieth century and interpretive qualitative methodologies gained prominence with the larger goals of social meliorism and/or goals of social recon­ structionism. This required a focus on the local, the situated, and on different conceptions of knowledge, power, and voice. Marked as a period of “reconceptualization,” in education writ large, beginning in the 1970s, particularly in relationship to curriculum studies (Pinar, 2004), began a time where the basic assumptions of mainstream educational research were called into question and alternative approaches to research were proposed. Critical analyses of schooling and the power relations within highlighted the force of dominant culture and ideology to circumscribe educational opportunities across groups. Sociology of education emerged, and researchers began to document the actual experiences of children in schools, the impact that children and teachers had on curriculum, and inequality of access to quality schooling (e.g. Gracey, 1972; Lubeck, 1985; Rist, 1970). Anthropological studies started to appear which provided rich descriptions of the lives of children and families to highlight how a variety of cultural norms interfaced with the dominant cultural norms of schooling (Heath, 1983; Lubeck, 1985). Early childhood scholars working within this new paradigm were fewer in number than in general education, and gaining voice within the field took time, with work emerging in the 1980s and beyond. These researchers took the position that it was crucial to understand childhood, early childhood education, and classroom practice as inseparable from larger poli­ ticized and value-laden contexts. Research highlighted the important notion that neither early childhood practice, nor the study of it, was value-neutral. Cannella and Bloch (2006) pointed out that this work “crossed disciplinary and geographic boundaries [and] fostered hybrid ways” (p. 6) of understanding early childhood theory and practice. Researchers operating within qualitative research methods brought two things to early childhood research. First, they offered another way to conduct inquiry, asking different questions and generating different information. Second, they brought to bear the influence of postmodern/post-structural thought, with expanding ideas of what counted as knowledge and truth. It is important to note that while the history of qualitative research in relationship to the field of early childhood education is briefer than that of quantitative research, it is no less important. In upcoming chapters, we explore in detail how qualitative studies differ from those influ­ enced by the quantitative methods of developmental psychology. To briefly set the stage, the range of qualitative approaches used share some characteristics: understanding phenomena in light of complexity, concerns with issues of power, and understanding children and adults through their voices and firmly situated within their unique contexts (Walsh, Tobin, & Graue, 1993). Qualitative studies provide rich descriptions of children, classrooms, and practice, such as these early exemplars: narratives about the experiences of children who were either Black and enrolled in Head Start or growing up White and middle-class (Lubeck, 1985), analyses of how differing communities defined readiness for kindergarten (Graue, 1992), and examina­ tions of children’s peer relationships and culture (Corsaro, 1985). Qualitative researchers often push against the dominance of the quantitative approaches from developmental psychology. Having their work recognized, both professionally and in the political arena, has been met with both challenges and successes. Only a decade after his first book on the use of qualitative methods in early childhood, Hatch (2007) described how changes in federal policy with the No Child Left Behind legislation passed in the early twenty-first century (re)privileged quantitative approaches. He warned that this perspective impacted the credibility of qualitative researchers. Thus, questions such as “Is this research valuable?” “Should it have weight in the world?” “Should it be supported?” are far from being answered with common agreement among researchers and policy-makers.

39

STRUCTURES OF RESEARCH

In sum, we provide a brief overview of how changing historical, political, social, and cultural contexts of education shape how we consider, approach, and study the education of young children. Shifts in what early childhood education researchers and teachers want to understand and how we go about research have pushed the field to move beyond the methodological boundaries of developmental psychology. The ties to quantitative research are no less important in the field today, yet the field has also opened to a wider variety of research methods that help us grapple with the complexity and the heightened importance placed on the field today. There is little quibble about the social and economic benefits of quality early childhood education (for example, Barnett, 2008; Heckman, 2011). As programming expands and states increase interest in supporting comprehensive early childhood education, we fully support the idea that a wide range of research and research methods only help us better understand what young children and their families want and need from the field.

Ethics Smith (1990) suggests: ethics refers to that complex of ideals showing how individuals should relate to one another in particular situations, to principles of conduct guiding these relationships, and to the kind of reasoning one engages in when thinking about such ideals and principles. (p. 141) Certainly, in relationship to the enterprise of educational research, ethics should arise as an important consideration of the researcher’s work. It is true that when we consider the dif­ ferent paradigms of research, ethics are approached somewhat differently, and the contexts of research shape how researchers consider issues of ethics related to their work. The history of early childhood research presented here should provide enough basis to engender questions of what it means to be a researcher, particularly in relationship to the people or groups that one might study. We have discussed how research positions range from a belief in objective observation of phenomena for some to a full embrace of the impossibility of neutral objectivity for others, begging authentic questions of how researchers might behave as human beings in this endeavor. Questions of ethics are not always high­ lighted in presentations of research unless there is a particularly vexing ethical dilemma involved in the study. It is not common for a journal-length research report to include a discussion of the ethics involved in the study. It is becoming more common, however, for researchers to provide a statement attesting that they have followed ethical guidelines, typi­ cally under the guidance of a local oversight committee. So, if we are not required to delve into ethics while reading research, why might we choose to do so here? Perhaps the interpretive turn in social sciences research has provided the contexts in which the ethics of research have become more prominent and even more necessary to consider. In the interpretive paradigm, positionality, relationship, and trust between researchers and participants have become central, and issues of ethics have taken on a more intimate flavor. In the field of early childhood, we take the stance that since we are always engaging with questions about young children or about phenomena related to their care and education, we have a particularly keen responsibility to consider the ethical responsibilities and implications of our work – regardless of the research paradigm used to conduct the research. Research in early childhood education always has the potential to

40

CULTURES AND CONTEXTS OF RESEARCH IN EARLY CHILDHOOD EDUCATION

impact educational opportunities, and thus the lives of children and their families. In the United States, children are considered a ‘vulnerable’ population warranting particular care so that the best interests of the children involved (whether as the primary target of the research or as those for whom the research has implications) are at the forefront at every step of any research project. In the shifting paradigms of research, it is important to consider how the different para­ digms frame ethical responsibilities. Researchers working from a positivist frame tradition­ ally claimed scientific neutrality. The writ-large goals of research in this paradigm are the “traditional utilitarian goals of advancing knowledge … and benefitting society” (Howe & Moses, 1999, p. 24). Objectivity itself is a goal of quantitative work, so the goal of any project is to design as much objectivity as possible. Ethical responsibilities are fulfilled via following the standards set by professional organizations, and via the process of pre-study research design. However, as times change, so does the potential stance toward ethics. For instance, the Society for Research in Child Development’s (SRCD) most recent ethical standards declare that research should avoid, minimize, and remove harms to the participants, a longstanding principle. Research should also be conducted with consideration for sociopolitical contexts; the statement gives an example of protecting children whose families do not have documentation where they reside. The guidelines go on to assert that research efforts should promote fairness, justice, and equity, while respecting the dignity of people and communities and providing a “compassionate and safe” environment (Society for Research in Child Development, 2021). This recent iteration of the ethical standards indicates much more advocacy for society and individuals and a less “neutral” position toward the researchers’ work. Conscientious researchers must consider what to do when they uncover sensitive infor­ mation. For example, if an instrument measuring depression is used in the research, should there be a response from the researcher if a participant’s responses show the potential of higher levels of depression? What are the researcher’s responsibilities if they see evidence of potential child abuse or neglect in the course of collecting data? How do promises of con­ fidentiality matter in these situations? For example, in a multi-site study in which one of us (File) participated, the study protocols called for offering a local mental health hotline number to participants who answered questions on a depression scale in the affirmative. Also, concerns about potential child abuse must be reported, and this information is pro­ vided upfront in the process of informed consent. Overall, “for both quantitative and qualitative research studies, the integrity of the research is determined by the authenticity of data, proper data representation, and poli­ tical issues surrounding research findings” (Howe & Moses, 1999, p. 29). In line with the interpretive turn taken by social science research over the past century, questions of ethics have necessarily taken on more importance, urgency, and complexity, and bene­ fited from deeper consideration (Cannella & Lincoln, 2011). We have moved into paradigms of thought and research which insist that “research is not neutral because all research has embedded in it particular assumptions, and ways of viewing the world; particular positioning to knowledge, knowing and meaning making, and then the types of research relationships that are possible” (Coady, 2010, p. 82). While slower to react, this stance has gained ground even among researchers working within the traditions of developmental psychology. This means that, while issues of ethics can and should be an integral part of designing research projects, deep thoughtfulness about ethics and potential ethical dilemmas must be considered throughout the research process, includ­ ing later impact on participants.

41

STRUCTURES OF RESEARCH

The interpretive turn necessitated growth in qualitative research methods (Cannella & Lin­ coln, 2011; Christians, 2011; Rabinow & Sullivan, 1987). Qualitative research at its core is about understanding and interpreting the perspectives of participants and the whys of human behavior and phenomena. The importance of relationships with study participants in order to most accurately reflect their experiences necessitates that ethical considerations are broader and can be more complex. Since qualitative research is an umbrella paradigm with several move­ ments of thought under it, the considerations of ethics do vary somewhat across studies. In research in early childhood education, the considerations have a great deal to do with the unit or level of analyses occurring within the study – does the research focus on social structures (schools, care settings, policy structures), on cultural groups (children in groups, families, teachers), or on individuals (children, teachers, family members)? Within this, any number of ethical questions can arise that need to be handled. These include issues such as how the presence of the researcher may impact the research setting, how the participants feel about the research questions, how the research process impacts them, if the data that are being collected are enough/appropriate to capture their perspectives, how power dynamics are playing out in the research setting or with the researcher, and what the later impacts of the research findings may be on the group/individuals participating in the project. Added to this is that, in many cases in early childhood research, children are the main focus of the research and thus an additional element of care in relationship to ethics is imperative. And, in all cases in early childhood research, the findings of research will impact children and families in some way, making ethics crucial. In later chapters, we further discuss some of the methodological devices that both quan­ titative and qualitative researchers use to address ethical considerations in their work. As mentioned, the building of as much objectivity as possible into the research design, prestudy implementation, is a way that quantitative researchers incorporate ethics into their work. Carrying out the project according to the plans and protocols that acknowledge and protect against identified potential harm and ensure that informed choice to participate is offered are primary considerations for quantitative researchers. For qualitative researchers, ethical considerations are an ongoing part of study design. Because qualitative researchers pointedly grapple with objectivity (or do not believe it exists), as questions arise while a study is carried out, ethics must be reconsidered from point to point. And it is the case that ethical questions may cause a researcher to change the study design, even during the study, to reflect or address ethical dilemmas. Qualitative research focuses on relationships, per­ spectives, and interpretations, and how these phenomena occur can raise unexpected ques­ tions and concerns needing attention.

The Human-ness of the Research Enterprise We end this chapter with consideration of the “human-ness” of the research enterprise, considering the realities of passionate disagreement among researchers and the stakes of the situations within which policy-makers and researchers interface. One might believe that with the processes of science underlying research there would be relative agreement about research findings. In other words, if the methods utilized in research reflect accepted con­ ventions about how to do research, how much can others contest the findings? In fact, there can be intense disagreement and debate among researchers. There are many decision points during the research endeavor that can incite discussion. Researchers disagree about the decisions made. They disagree about the perspectives taken in asking a research question. They disagree about what was not included in a research question. Researchers come with

42

CULTURES AND CONTEXTS OF RESEARCH IN EARLY CHILDHOOD EDUCATION

their own ideas about how to investigate a phenomenon. They also have biases about how best to conduct research and what methods make the most sense. At times researchers may take opposing positions or make decisions that will be criticized. Because so many matters in early childhood can relate to public policy and the decisions made within a democratic society, the context often becomes even more fraught. For instance, when do research findings justify a large outlay of money toward an intervention or program? Or a change in public policy? A vivid example of this sort of turbulence exists within early childhood research related to infants in childcare. In the mid-1980s Belsky (1986) published a brief article in the United States contending research evidence showed infants who were enrolled in childcare on a more-than-part-time basis were at greater risk for insecure attachment with the mother. The stakes were high, as care for infants was increasing at rates higher than other age groups because mothers were in the workforce more often and sooner following the birth of a baby. A New York Times reporter noted, “His statement created a professional uproar, and the subject was hashed out on talk shows and in midnight discussions of countless working parents” (Eckholm, 1992). Intense discussion, and more research, followed for years. Two years after the initial piece, a special issue of the journal Early Childhood Research Quarterly was devoted to the topic. Belsky (1988) published another article in this issue clarifying that with more recent research he viewed the risk as such: infants who participated in more than 20 hours of nonmaternal care in the first year of life were more likely to show behaviors in a research setting that reflected avoidance of their mother. Belsky (1988) even expressed, “I know from experience that this is not a popular point of view within the developmental sciences today … it is charged with being politically and ideologically driven” (p. 266). The rest of this issue contained articles from other researchers who discussed their con­ cerns about Belsky’s conclusions. For example, the research procedure used at the time to assess attachment in infants and young children was questioned. Called the ‘Strange Situa­ tion,’ it involved a sequence of events during which the child was with the mother, separated from the mother while with a “stranger” (the research assistant), and reunited with the mother. It was devised decades earlier, before children in childcare regularly separated from their parents. Two researchers (Thompson, 1988; Clarke-Stewart, 1988) questioned whe­ ther this procedure had remained meaningful in the same way in recent research, given social change. Clarke-Stewart raised questions about whether the concepts regarding attachment in use at the time were too limited for examining this phenomenon. She also questioned what was known about the effect of employment on mothers, speculating that difficulties with employment could impact the infant, rather than only the experience of childcare. Another pair of researchers (Richters & Zahn-Waxler, 1988) raised technical questions about Belsky’s reading of the research. As researchers debated, many in the public were alarmed. Widespread employment trends could not be easily reversed. Were children being harmed? Should there be policy in this area? If so, what sort – paid leave for parents or more regulation for quality care settings? Because this issue was so potent, a large federally-funded study (the Study of Early Child Care and Youth Development – SECCYD) was launched by the National Institute of Child Health and Human Development (NICHD). Over 1300 babies and their families, from ten locations across the United States, were enrolled in the study. Data collection continued through the middle school years, and as the data were analyzed, the results were published. The research questions were complex, taking into account what was happening in childcare centers and what was happening within families. In an account of the study written for the

43

STRUCTURES OF RESEARCH

lay-public, results of childcare enrollment through children’s entry into school were summarized as follows: Center-based child care is associated with both positive and negative effects. This type of care is linked to better cognitive development through age 4½ and to more positive social behaviors through age 3. But, center-based and large-group settings are also associated with more problem behavior just before and just after school entry. (NICHD, 2006, p. 21) As publication of the study’s findings continued throughout the years, so did discussion about their meaning and implications. Questions about how much the quality of the child­ care settings mattered and about what was being emphasized in discussions – the large per­ centage of children functioning well or the much smaller percentage reported as having behavioral issues – were replayed in professional and lay outlets. Other questions about the design of the study and how it could and should inform public policy were debated. Because of the public nature of some of these debates, this particular long-term research project has provided a front-row view into a very human story.

Summary Perhaps a good word to summarize this chapter is “complexity.” While we hope upfront to ensure readers understand how incredibly complex the research enterprise is, our intention is not to create anxiety. Complexity does not render the topic beyond understanding. We hope that it leads readers to a perspective that includes the following big ideas. First, how we conduct research and what has interested researchers has varied over time, and it is true today that there are very different approaches. Not surprisingly, the value placed on varying approaches differs depending upon who is doing the valuing. Second, as is true of any human enterprise, research is a space where people bring their own biases and preferences. From the beginning, as a study proceeds, the space where a researcher works is somewhat unique, shared by some and foreign to others. Researchers are no less human than anyone else who might have passionate ideas in response to the question, “If we know this thing, what does it mean for where we might go from here?” Importantly, there are many approaches to answering this sort of question, which we take up in the chapters to follow.

44

5

METHODOLOGIES

Research is a human activity and one with multiple points of entry. In Chapter 2, Whyte introduced the foundations that underlie the research project itself: the researcher’s own positions on epistemology and ontology, those larger questions that frame ways of under­ standing the world, as well as the questions that drive an inquiry. And, of course, these questions occur within a context that may, as we heard from Iruka in Chapter 3, rest on an often unexamined history of inequities. In this chapter we introduce the major tenets governing how researchers work within qualitative and quantitative paradigms, and also discuss mixed methods.

Comparing Qualitative and Quantitative Research In the simplest terms, qualitative researchers base their data and analyses in text and narrative, while quantitative researchers use numerical representations of data and statistical analyses. As shown, researchers across these paradigms have different beliefs at the core of their work. In this section we outline the premises of research by drawing comparisons between qualitative and quantitative approaches. Additional information about generating, analyzing, and presenting data is discussed in the chapters that follow. Placing the Study Research never exists within a vacuum, but instead within a larger body of knowledge or literature. Each study is conducted within a space where there are studies that preceded a particular project and studies that will follow. How that body of knowledge factors in the researcher’s decisions and presentation of the study are considered below. Quantitative Research Quantitative researchers tend toward focus upon the “long story” of research and the body of literature. Their work typically begins with a consideration of the questions: What do we know now about this phenomenon? What has come before? Where are the gaps? In this way researchers attempt to make the next step in building a body of knowledge. In their pre­ sentation of a study, quantitative researchers situate their studies within this long story by beginning with a review of the literature (literature review), describing what previous research has found and questions that remain unanswered. Then, in the concluding discus­ sion section, the researchers typically indicate how their current study has contributed to the body of knowledge and next steps for future research. The picture that emerges is that each

DOI: 10.4324/9781003354499-6

45

STRUCTURES OF RESEARCH

piece of research is like a part of a brick wall, contributing to the whole but needing each of the other pieces to make sense. To illustrate, Piasta et al. (2012) conducted professional development with teachers in American Head Start and public pre-kindergartens serving low-income groups. The research group then examined how teachers responsively engaged children in conversation, as well as selected features of the children’s language production. They situated the study broadly within the focus on children’s school readiness and achievement gaps. Early on, in stating the purpose of their research, the researchers said their study was “drawing on a large body of research” (p. 387). In the literature review the researchers examined what was known from previous research about preschools as language-learning environments, establishing that observational instruments had documented relatively low levels of lan­ guage development support. Noting the potential of language development supports for children, they stated, “Findings such as these have led researchers to consider the potential benefits of PD [professional development] for enriching the preschool classroom languagelearning environment” (p. 388). They noted few studies had been conducted to examine PD in this area. In this way, the researchers made the case that their study was the next logical step in the development of the knowledge base. They utilized a particular PD program for which previous research had shown “promising prior reports” (p. 389) when implemented in Canada. Consequently, their purpose in this study was to examine whe­ ther a briefer version of the PD, implemented with teachers serving low-income children in the United States, was effective. In the discussion section of their paper, Piasta et al. (2012) compared their findings to other studies. For example, they noted, “the findings of the present study converge with prior reports …” (p. 397) and “our findings are unique in that …” (p. 397), and “the value of the present work, in extending [previous] findings, is ….” (p. 398). Note how these statements help the readers to understand the place of this study within the larger body of research. In summary, quantitative researchers tend to regard their work as building upon the information and gaps from other work, illustrated by the metaphor of the brick wall. Showing the structure of that wall to the reader is important. Thinking back to Iruka’s chapter, one can see that without a great deal of consideration of newer worldviews (e.g., critical race theory), how easy it can be for studies based upon deficit perspectives to stand without those viewpoints being disrupted. Qualitative Research Qualitative researchers also spend time situating their studies within the larger body of literature and research related to their question of study. But we will immediately change the metaphor of the brick wall from the previous section. Qualitative researchers seek less to add bricks to the wall, and more to remove the bricks and use them to build new walls (or bridges or fences or roadways). Rather than establishing what it is that we already “know” about at topic or phenomenon, qualitative researchers situate their work within the larger body of research to help us ascertain what is understood thus far. This understanding points to where deeper, more sophisticated, or more critically derived understandings might be beneficial – a nuanced but important distinction. The existing body of knowledge is not taken for granted as the canon. It is, rather, questioned and contested to explicitly point out what has been left out, the voices that have not been considered, and the power relations that shape a phenomenon that have not been previously accounted for.

46

METHODOLOGIES

This questioning occurs as a result of most qualitative researchers’ epistemological beliefs about the nature of knowledge – that it is local, situated, and framed socio-culturally, poli­ tically, and historically. Explicating the theoretical framing of a study is a key feature of qualitative work, and this is where the grounding in existing literature is often focused. There are a variety of paradigms of thought and theory from which that qualitative researchers draw, and no one school of thought is considered at a higher status than another. In fact, the same corpus of data could be examined from a variety of lenses, each garnering differing values, interpretations, and different kinds of findings. “Results” are not really a feature of qualitative work. Researchers engage in an accounting of findings and a discussion of their relevance that is consistent with the organizing theoretical framework. This makes the explicit situating of a study within a particular theoretical framework a critical component of this work. For example, Souto-Manning (2014) examined and described engagement with a “taboo” topic on the part of a teacher and the children in her preschool classroom. Her goal in this article was to question and critically examine the notion of the “ideal” early childhood class­ room that is presented in more traditional early childhood research. She gave an overview of research related to this notion of ‘ideal,’ but not with the objective of building upon that as a settled and assumed concept. She took an established concept, pointed out the ways in which it has been culturally derived and is fraught with power relations to dominant cultural ideas, thus leaving out what could be productive avenues to supporting children in classrooms. In order to do this, Souto-Manning (2014) introduced the lens of “conflict theory” as a mechanism to critically examine what the research currently has established as the “ideal” classroom. She reviewed previous research to establish how this concept/phenomenon has been discussed. She specifically sought to disrupt the idea using this “new” theoretical lens, carefully pointing out the ways that conceptualizations thus far have not been complete, or have indeed excluded the values and norms of a diversity of cultural perspectives. SoutoManning specifically stated, “I suggest that the normative construct of the ‘ideal’ early childhood classroom needs to be challenged, deconstructed, and reconceptualized” (p. 610). She situated the study within the framework of conflict theory to highlight the short­ comings of previous work, and to set the stage for her critical examination of her data. Souto-Manning’s (2014) presentation of findings and her discussion were utilized to establish and provide evidence for her interpretations. She used the extant literature base to continue to compare and contrast her lens and interpretations with what had come before, rather than to build upon it per se. As we said at the beginning of this section, qualitative researchers seek less to add bricks to the wall, and more to remove the bricks and use them to build something new. Reasoning and Logic Reasoning and logic are the processes by which researchers move from one point to another, drawing connections and making inferences. While quantitative and qualitative researchers might draw upon both inductive and deductive reasoning models, there are patterns within each paradigm. Quantitative Research Quantitative researchers most often use deductive reasoning in their work. To begin, they rely on theories about how things work. Theories provide an explanation and can range

47

STRUCTURES OF RESEARCH

from more general (e.g., Vygotsky’s theory of learning and development) to more spe­ cific (e.g., a theory of how coaching affects teachers’ use of strategies learned during professional development). Researchers then devise questions and methods to test a specific example of the theoretical explanation. Traditionally researchers presented their test of the explanation in the form of hypotheses, relationships that they expected to find (e.g., x leads to y). It is more common now that they put forward research questions or statements of purpose; these questions or statements do not utilize the same formal wording as the hypothesis statement. Returning to our example of the study by Piasta et al. (2012), we find the general theore­ tical explanation early in the literature review: “Both theory… and recent research … suggest that children’s language development is contingent upon the quality and quantity of language and communicative acts to which they are exposed” (p. 388). From there, getting more spe­ cific, they examined a particular form of teachers’ language, conversational responsiveness, which facilitates several turns for each speaker within a conversation. They also centered on the problem of using PD to increase the occurrence of these types of conversations, using two groups of teachers, one who received PD in this area and one who did not. Their aims were summarized at the end of the literature review (incidentally where one hopes to find them): The first aim was to investigate the extent to which preschool teachers’ participation in PD influenced their immediate and sustained use of … strategies over the aca­ demic year. The second aim was to investigate whether the conversational pro­ ductivity and complexity of children in PD classrooms was increased compared with children in comparison classrooms. (p. 390) Qualitative Research Qualitative researchers engage in close study of phenomena in our lived worlds. Inductive reasoning, where a variety of premises are considered and are valid, guides the work of qualitative researchers. Researchers draw from a theoretical framework to study the parti­ culars of an occurrence in a social setting that then lead to new and more sophisticated understandings and different interpretations about what is going on. The goal is to expand existing theories – sometimes to disrupt, sometimes to add nuance, sometimes to apply a new theory to interpret the phenomena, and sometimes to create entirely new theoretical explanations. Qualitative researchers do not study a situation or phenomena to substantiate an existing theory, but rather use a theoretical framework to guide their interpretations. Indeed, there is a strand of qualitative research called “grounded theory” where the goal is to study a phenomenon fully and from it derive new and unique theories about the workings of the world related to that phenomenon. Logic in a qualitative study is steeped in the notion of consistency within the conceptual and theoretical groundings of the study and the conduct of the study. This logic helps draw a thread through a study where the theoretical framing guides the types of questions asked, the types of data collected or the kinds of situations examined, the manner of the inter­ pretations, and nature of the conclusions that might be drawn or discussions that may ensue. Conclusions that are drawn are done so as a result of providing evidence of the researcher’s interpretation of the phenomena under study, which relates back to the theoretical framing of the study. It is a weaving together of a story that helps us to better understand or inter­ pret differently the phenomena under study.

48

METHODOLOGIES

Returning to the Souto-Manning (2014) example, she stated up front that she engaged in an ethnographic research project. While ethnography is a methodology, it entails a particular set of data collection methods that provide insight into the ways that this researcher con­ sidered phenomena and how she theorized the workings of the world. She spent long periods of time in a setting, collecting careful notes, and observing what happened in that setting. She interviewed carefully the “actors” of the setting and collected artifacts that represented the working of the culture in that setting. Her goal was to be able to share some interpretations and draw some conclusions based on this kind of “rich description” (Geertz, 1973). In the example, Souto-Manning (2014) took a slice from what she observed. She interpreted the occurrences in a unique manner (i.e., using a fresh theory to examine the data) to disrupt more traditional notions of the “ideal” early childhood classroom. She used conflict theory to reinterpret classroom occurrences, inviting readers to “reconceptualize conflicts as powerful and useful learning spaces potentially leading to responsive and authentic educational experiences” (p. 613). In this study, Souto-Manning (2014) sought to (re)interpret cultural phenomena in cri­ tical ways. To do so she demonstrated the belief that one must carefully and closely study these phenomena to understand them (as best as is possible) from the perspective of the “actors” in the setting. Then, she applied a theory (conflict theory) to a unique setting to re­ interpret a classic notion (the ideal early childhood classroom) in a new and productive way. The logic of her choices drew a thread through her study and her interpretations, deriving from her original theoretical framing. The Researcher’s Role – Objectivity/Subjectivity For the general public, science is typically considered to be an objective activity. It is up to the scientist to conduct a study that provides unbiased information, data that can be con­ sidered dispassionately for what they reveal. In the modern world, in which positivism represented the approach to research, it was assumed that value-free neutrality would steer researchers toward objectivity. They could leave aside their beliefs, hopes, and passions and follow the conventions of research. Yet, within both the natural and social sciences, there is common agreement that this vision of objectivity is not possible. Research is conducted by people, who enter the situation with experiences and perspectives. How those experiences and perspectives are handled differs between qualitative and quantitative researchers. Quantitative Research Quantitative approaches are more closely tied than qualitative approaches to the positivist roots of modern-era research. Most researchers, however, do take the issue of human falli­ bility seriously, thereby disallowing a belief in the ability to achieve complete objectivity. The stance on objectivity has been rewritten to make it a goal toward which to strive, described by Gersten (2013) as the researcher being an “objective-as-possible designer and imple­ menter” (p. 141). Consequently, quantitative researchers consider objectivity as an important consideration of their work. How can they reduce partiality, lessen the impact of predispositions, and achieve a more accurate measure of the phenomenon at hand? In the words of researchers in describing a postpositivist position: “It is still possible for a field of research that is externally influenced by values to operate internally in a relatively objective manner (indeed it is crucial for the scientific enterprise that it does so) ….” (Phillips & Burbules, 2000, p. 53).

49

STRUCTURES OF RESEARCH

Quantitative researchers work toward being objective through many strategies, which can be understood as: 1) removing the partiality assumed of people; and 2) reducing the perso­ nal stakes in order to be unbiased. We will explain forms of these strategies below. Partiality is countered through the careful development and use of research instruments. It is common for research instruments such as surveys, child assessments, or observation tools to undergo lengthy development to ensure they “do what they are intended to do” without systematic bias. We discuss more about this in relation to reliability and validity below. For now it is important to understand that through the use of research instruments, quantitative researchers strive to operate more objectively – the instrument reduces the influence of a subjective individual researcher when it is used carefully and as it was designed to be used. Data collection flows through an instrument designed to operate in similar and objective ways whenever used as intended. To illustrate from the Piasta et al. (2012) study, the researchers adapted and used two teacher surveys that had been developed for studies conducted in 1985 and 1993. It is not uncommon for instruments to have a long period of use and also to be adapted along the way; often earlier research has established that the instruments operate in objective ways. Additionally, in this study classroom videotapes submitted by teachers were coded for the presence of clearly-defined teaching strategies by trained research staff who watched 30 seconds of tape and then recorded whether or not the strategies were used. This systematic coding system and the training to use it allowed the research team to increased objectivity. Another way to strive toward objectivity is to reduce the stakes for those involved in research. For example, many research decisions are made before data collection begins, and the analyses to be performed follow the questions that frame the study. The idea is that researchers will not examine the data looking for what strikes them, or conduct analyses until something “turns up.” Decisions made before data are collected are regarded as more objective in nature. As another example, data collectors may not know which children experienced an intervention and which ones did not, so that they are not inclined to act differently when assessing them. This is referred to as being “blinded.” Quantitative researchers may not reveal much about the purposes of their study when recruiting and enrolling participants, so that the participants do not tailor their responses or behavior toward what they believe the researcher wants to see. In all of these ways, researchers are in effect detaching themselves, their team members, or participants from becoming invested in a particular end point for the study. In an example from the Piasta et al. (2012) study, transcripts of classroom videos submitted by teachers were “checked for accuracy by an independent … transcriber after initial transcription” (p. 393). By having a check on the initial transcription that was performed by independent research staff, the researchers hoped to ensure the staff did not let their knowledge of the study impact the transcription of oftencomplex and messy classroom conversations that may be miscontrued. Qualitative Research In early anthropological studies the notion of the objective researcher sent to study and describe the cultural practices of “exotic” populations reigned supreme. It was largely believed that the social scientist, trained in traditional methods of ethnography (observation of phenomena with detailed field notes, interviews of participants/informants, and collection and cataloging of cultural artifacts being key methods), could objectively catalog and accu­ rately describe the cultural practices of the mythical “other.” However, over time in this discipline, the post-modern interpretive turn of research (Howe & Moses, 1999) has firmly

50

METHODOLOGIES

turned the possibility of objectivity on its proverbial head. Trends in qualitative research in education have mirrored this same journey. Hence now, rather than attempting to “prove” or “increase” objectivity, there has been a movement in qualitative research to rigorously consider, examine, and explain the perspec­ tives, worldviews and points of identity from which a researcher’s work derives. This pushes qualitative researchers to address, navigate and attend to the ways that these frame and influence the research endeavor. Qualitative researchers try to engage with and acknowledge exactly the ways in which who they are influences what they do. In many reports of research, the researcher will not only explain perspectives and frames of reference, but will also engage in discussion of the limitations of these perspectives. Qualitative researchers do not develop tools of data collection in order to increase objectivity, rather the researcher him/herself is the tool, and objectivity is not necessarily a goal. Two approaches are used in qualitative research to address researcher subjectivity: “posi­ tionality” and acknowledgement of “insider/outsider” perspectives. Positionality refers to explaining social and cultural positions that one holds and embodies as a human being. Researchers often interrogate the social structures (institutions, ideologies, power) that have created the possibilities for their positions and consider them in relationship to the project at hand. The insider/outsider perspective refers to one’s position in relationship to the group under study. Might the researcher be considered to be a member of the group (an insider), or is the researcher sufficiently unfamiliar with the activities of the group to be considered an outsider? The researcher must consider how this relationship impacts the assumptions, per­ ceptions, and interpretations of the research agenda. It is the case that within a manuscript, researchers may have space constraints to report on the above processes. When qualitative folks have more space (i.e., in presenting a book-length treatise of their study) it is not uncommon for entire chapters to be devoted to discussions of researcher identity and relationship to the context of the study and to the study participants. A wonderful example of this is in Shirley Brice Heath’s book, Ways with Words: Language, Life, and Work in Communities and Classrooms (1983) where Heath provided in-depth discussions of her relationships with the communities she studied and the ways these shaped the project. In the Souto-Manning (2014) research example, Souto-Manning described her relationship to the teacher in the classroom where she conducted her data collection. She noted that she was the facilitator for “the inquiry-to-action culture circle meetings in which [the teacher] partici­ pated” (p. 615). Souto-Manning described the kinds of inquiry processes in which this group engaged and the kinds of insights and discussions that occurred. From this, the reader can make clear inferences about some of Souto-Manning’s social positions and her values and beliefs. For instance, she indicated that she based the culture circles on the work of Paolo Freire, which set her in a particular theoretical camp in relationship to her beliefs about teaching and learning. Forms of Data We have mentioned previously that qualitative data are textual while quantitative data are numerical. Upcoming chapters focus on data collection in detail. Below we make distinc­ tions about the forms of data generated for a study. Quantitative Research It is important to remember that while a quantitative study presents data in numerical form, the study begins with concepts that are defined and expressed with words. From there the

51

STRUCTURES OF RESEARCH

researcher devises ways to measure what is of interest, thus translating the concept into numbers. For instance, Piasta et al. (2012) were interested in several pieces of information about the teachers in their study. They recorded how many years of experience individual teachers had. This is a straightforward example of how a concept (experience) is measured in a numerical way (years). They also recorded the teachers’ highest level of education. This is a readily understandable example to most of us. Did a teacher have a high school diploma or an associate’s degree? A bachelor’s degree or perhaps a master’s degree? To measure edu­ cational level, the researchers would have assigned a number to each of these levels, such as a 1 for a high school diploma, a 2 for an associate’s degree, a 3 for a bachelor’s degree, and so on. In this particular study the researchers did not describe this coding scheme, but because the study is quantitative, readers can infer that these levels of educational experience have to be represented with numbers that make sense, in this case, higher numbers mean a higher level of education. In a similar vein, a characteristic such as gender is rendered into numer­ ical form by assigning a number to children’s gender. By these means, the information can then be used in statistical analyses. The researchers also used two attitude surveys (Piasta et al., 2012). One measured tea­ chers’ self-efficacy based upon the teachers’ responses to 20 questions. The researchers named four focal areas within which the questions were clustered: discipline, instruction, positive environment, and school/classroom decision-making. To measure the concept of self-efficacy the teachers rated each item on a 5-point Likert scale, where 1 corresponded to “no feelings of efficacy” and 5 corresponded to “very strong feelings of efficacy” and the numbers in between could be used for responses in the middle of this continuum. The number used in analyses was the average score across all 20 items. In sum, in order to understand quantitative research, one must first understand the con­ cept being measured, for instance, self-efficacy. Second, the reader also has to understand how that concept was used by the researchers. In the Piasta et al. study the researchers looked at teachers’ sense of effectiveness in relation to several aspects of teaching. Finally, one has to understand how the measurement works; as in, what does a higher number indicate? A lower number? This translation from concept to measurement to score is a cru­ cial part of quantitative research. Qualitative Research The goal of qualitative research is to understand the how and why of human and social phenomena and human decision making. Quantitative data takes the forms of careful observation and documentation of the phenomena within the social and cultural contexts in which they are occurring. Qualitative projects often involve a large corpus of data of varying kinds. Furthermore, copious amounts of time spent collecting and analyzing data are a key feature of this work. Given the caveat and desire of design consistency within the project, as described earlier, the kinds of data and the methods of collection are shaped by the theoretical approach to the study, the kinds of questions being asked, and the choice of an appropriate context in which to study a particular phenomenon or group of people. The varying paradigms of qualitative research employ and subscribe to different methods and methodologies and rely on different kinds of data to varying degrees. The goal in qualitative research is to collect enough data to garner an understanding of and be able to have an evidentiary interpretation of what is under study. It is the case that the types of data that qualitative researchers collect are fairly similar across types of projects. We later elaborate upon the uses of the various

52

METHODOLOGIES

types of data, but for the overview purposes here, consider the following list with an example (where applicable) from the Souto-Manning exemplar study: Direct observations and fieldnotes/reports. Souto-Manning (2014) described how she “visited Ms. Jill’s classroom for a minimum of 5 hours each week, taking notes, audio recording classroom and playground interactions, and collecting artifacts” (p. 614). The work extended over a period of nine months. Interviews. Souto-Manning (2014) described her interviews with the teacher in the classroom where she conducted her study. She justified these interviews as a form of evi­ dence for her project by noting the importance of developing a trusting relationship with this teacher and developing a ‘safe space’ for the teacher to try out new ideas and under­ standings, and that allowed the teacher, “to open up and to share her uncertainties” (p. 615). This kind of data collection is consistent with Souto-Manning’s purpose to explore conflict and power relationships in classrooms. Collection of artifacts. Souto-Manning (2014) reported that she collected artifacts from both the teacher and the children. One of the artifacts she gathered from the teachers was the writings that the teacher produced in the “inquiry to action culture circle” that SoutoManning facilitated. She also reported collecting “classroom artifacts” to round out the evi­ dence of her observations and interpretations of the happenings in the classroom. Defining Trustworthiness and Rigor Scientific research is associated with the word “rigor.” It is expected that researchers work carefully and follow common procedures to ensure their work is precise and thorough, leading to results (in quantitative research) and interpretations (in qualitative research) that can be trusted. Because of the differences in data sources, qualitative and quantitative researchers have different ways to achieve the goal of rigorous investigations. Quantitative Research Measurement (translation of a phenomenon into a numerical score) is central in quantitative research. This is key to understanding how rigor is approached within this paradigm. In this section we explain how instruments are perceived as valid and reliable. Instruments should measure what we design or want them to measure. This is at the heart of the concept of validity. The question here is how “accurate” an instrument might be. Researchers approach validity from several perspectives. As we describe each, we will refer to examples from Soukakou (2012). Soukakou’s paper addressed her development of an observation instrument (the Inclusive Classroom Profile – ICP) to assess what happens in inclusive classrooms in relation to the developmental needs of young children who have disabilities. Once Soukakou (2012) wrote items for the instrument, she asked several content experts to review them. They provided their evaluation of the substance of the items, as well as a rating on how important they thought each item was to the purposes of the instrument. This is referred to as content validity. Do experts in the field regard the instrument as including what it should and representing the important content well? Next Soukakou (2012) used the ICP in 45 inclusive classrooms, along with three other classroom observation instruments. The other instruments had been previously established as appropriate measures to assess classroom environments, curriculum, and teacher–child interac­ tions. All had been used previously in published research, although not in a particular

53

STRUCTURES OF RESEARCH

relationship to inclusive settings. Soukakou used the data from her observations to compare the ICP scores to the other instrument scores, an appraisal of construct validity. Think of a con­ struct as similar to the underlying concept being measured. Soukakou’s theory was that there should be a pattern of relationships among the ICP and the other instruments. If the constructs, or concepts, are more similar, the relationship should be stronger. If they are more different, there should be less of a relationship between the ICP and another instrument. The former – looking at relationships expected to be stronger – is referred to as convergent validity. The latter – looking at the relationships expected to be weaker – is called discriminant validity. Sou­ kakou concluded after finding the expected patterns, “the ICP correlated higher with measures that were conceptually ‘closer’ to its content, while it was more weakly correlated with items that were conceptually ‘distant’ to the ICP constructs” (p. 485). In addition to construct validity, there are other types of validity to consider. To assess concurrent validity the researcher examines how a new measure works in relation to a pre­ viously-developed measure of the same phenomenon. Perhaps the new measure is cheaper to administer, or quicker to use, but researchers would want the new instrument to ‘behave’ similarly to the older one, to be as accurate in measuring the construct. In predictive validity the researcher examines if the new measure predicts outcomes as would be expected theo­ retically. For example, if Soukakou believed that a higher score on the ICP should result in children with disabilities doing better in some way, she could compare ICP scores to out­ come measures for the children in the classrooms being observed. In sum, validity is about how accurately an instrument measures what it is intended to measure. We take time here to return again to Iruka’s chapter. Researchers undertake the procedures just described to ensure that measures “do” what they are expected to do. The worldview of the person developing a measure is not what is being assessed via these processes. It is up to the research consumer to consider what a measure means in relation to a particular community or group. In Chapter 13 we include a researcher perspective examining the use of an instru­ ment developed in the United States as it crossed borders into Italy (both WEIRD in the acronym introduced by Iruka, but different from each other nevertheless). The other half of the equation for quantitative researchers is reliability. The focus here is on consistency in the scores obtained for an instrument. Researchers expect and need to measure phenomena with dependability and stability. Soukakou (2012) used two forms of reliability in her study. The first was inter-rater agreement. Researchers expect that who the data collector is should not affect the score on an instrument – that different raters or data collectors achieve roughly the same scores. On the observational instruments used by Soukakou, observers were trained until they achieved scores on the instruments that were sufficiently close to the “standard” score by expert users of the instrument. This is done before any data are collected for a study. Sometimes researchers will do a check on a percentage of measures during data collection to assure that data collectors have not veered away from using the scoring protocols as they are intended to be used. If researchers cannot reach reliability when their instruments are used, their results will not have much worth. The second form of reliability used by Soukakou (2012) was internal consistency. When researchers combine several items on an observational instrument or a survey to result in one average score, they want all the items to “hang together,” indicating that they are all aspects of the same construct. On the ICP, Soukakou examined how the items were related amongst themselves in consideration of how they contributed to the overall construct of program quality for children with disabilities. The higher the number reported for this measure (Cronbach’s alpha, α), the more consistent the items are with each other.

54

METHODOLOGIES

Another form of reliability not utilized by Soukakou is test–retest reliability. When a direct assessment of individuals is used, the researcher wants to know that the individual would score close to the same in a reasonable amount of time (not too long, so there is no further growth or development). In conclusion, assessments of reliability help to assure readers that the researcher’s instruments operate consistently. Qualitative Research Because the findings from qualitative research specifically focus on interpretation of data rather than on “objective” measurement, it is not uncommon for qualitative research to be perceived as lacking in rigor. Researchers directly address post-modern and critical ideas of objectivity where that story of the data draws on the researchers’ and participants’ inter­ pretations of phenomena, and context (time, place, people) matters. In qualitative research the burden is on the researcher to present enough evidence of the claims about the data that the reader can believe and trust that those claims make sense. One of the major differences between qualitative and quantitative research in this area is the idea and meaning of “truth.” Quantitative researchers search for a larger objective and universal “Truth” and, thus, the focus on researcher objectivity is paramount to that endeavor. Qualitative researchers focus more on the situated and local nature of truth where a variety of explanations and interpretations can all be “true” depending on perspectives and experiences both of the researcher and the study participants. This can make it tricky, and the challenge is to support the reader to understand how the claims can be “true” in a context that may actually be more or less familiar to the reader. We rely on the notion of verisimilitude – or, simply put, “truthiness.” Important ques­ tions include: What is the level of believability of the story that the researcher presents or the claims made? How likely is it that the story or claims could be true based on the context and the participants involved? (Keeping in mind, of course, that the reader of the study comes to the reading with his/her own perspectives.) Does the researcher convince the reader of their expertise to interpret the data of the study? (Note that this relates to the earlier discussion of positionality.) Does the researcher present enough evidence from the data so as to convince the reader of the verisimilitude of the findings and interpretations? This striving for believability can take a good bit of effort and description. Thus, qualita­ tive projects and write-ups tend to be longer and include much detail. Many qualitative researchers have been stymied by the page limits of the journals in which they publish. And many qualitative researchers struggle with peer reviewers who strain to believe the claims made because the researcher cannot present enough data in the space allotted. So too, a good measure of believability in a qualitative study rests on the perceived rigor and con­ sistency of the data analysis, which requires manuscript space to explain. We go into more depth about qualitative data analysis in Chapter 7; we mention it here because part of “rigor” and believability of a study also has to do with convincing the reader that the analysis is sound and reasonable. A technique often used in qualitative research is called ‘triangulation’ where once a claim is made, that claim can be considered more “true” if there are three (or more) sources of evidence from different kinds of data for that claim. For example, the researcher might have observed evidence for the claim, heard evidence for it in an interview, and gleaned evidence from an artifact. Returning to the Souto-Manning (2014) example, she presented a very brief vignette of young children in classroom talking about “poop” and “stink.” This vignette was drawn from her classroom observations and field notes. She ultimately made the claim that these

55

STRUCTURES OF RESEARCH

kinds of conversations can be considered as in conflict with the official curriculum and that these places of conflict can be productively used as learning tools, both in pushing against that official curriculum and in “honoring the breathtaking linguistic and cultural diversity present in today’s classrooms” (p. 612). She used data from her interviews with the teacher to provide the evidence that the teacher initially viewed these kinds of topics as inappropriate fodder for the official curriculum. Then she presented more observational data to tell the story of how the teacher allowed the topics to come into the classroom, and the subsequent learning that took place. She also used data from the teacher’s participation in a teacher learning group to substantiate the ways that the teacher struggled with and ultimately came to differently understand points of conflict in an early childhood classroom. Throughout the manuscript, Souto-Manning (2014) made sure to reference and explain the amount of time she spent in the classroom and the various types of data that she col­ lected – “nine months of ethnographic work…through observations, informal conversations, interviews, and collection of artifacts” (p. 613). In this way, she established her researcher credibility in this setting. In this particular example, while Souto-Manning briefly described her method of data analysis (i.e., writing analytic narratives based in her data), she spent a great deal more time in telling the story of the classroom, presenting her findings, and pro­ viding her interpretations. This give-and -take between brief explanations of data analysis and longer presentations of findings and interpretations are often evident in a journal manuscript, whereas in a book, there is likely a full chapter explaining the data analysis. Making Meaning: Within the Study and Beyond The closing sections of a research report typically help the reader understand what the results or findings mean in a larger context and/or in light of the extant research. Often beginning with a summary or discussion of the findings, researchers then use these sections to consider the meaning of the study, discuss potential limitations of the study, implications for practice, and suggestions for further research. Quantitative Research The discussion section of the report of a quantitative study is typically written around a set of topics: a summary of the findings, a discussion of how the findings fit with previous research, implications of the findings, limitations of this particular study, and directions for future research. Researchers vary tremendously in how much attention is paid to each section. In addition, while the sections previous to the discussion tend to follow longstanding conven­ tions for presenting information, the discussion section is often less prescribed. In our experience reviewing manuscripts for potential publication, we have found it not so unusual for researchers to let go of some constraints, leading to concerns that the discussion goes beyond the actual results in regard to import, for example. This is one reason we do not want students to revert to the discussion section to get to a quick summary of a study; they are then taking the researcher’s interpretations for granted, rather than examining whether those interpretations appear founded. The discussion section Piasta et al.’s (2012) paper is rather extensive. The researchers are cautious in their wording. Many times results are discussed with wording such as “might reflect …,” “it might be …,” “it is possible …,” and “suggesting that … .” It seems clear that the authors are trying to avoid overstatements. The authors also discuss four limitations of the study. Part of the mindset of many quantitative researchers is that results are

56

METHODOLOGIES

somewhat tentative and might be revised upon further study. Remember, though, that there are a variety of stances, and in some studies the researchers appear quite sure of the meaning and importance of the study and less expansive about possible limitations. One of the long-standing assumptions of quantitative research is that it allows researchers to make generalizations to other groups. In fact, the statistical analyses operate with the notion that the sample of participants for a study is a subgroup of a larger population of individuals. The question at hand is: in what other situations might we expect to find the same results? Although this is an assumption of the paradigm, some researchers do not directly address this question, and it is up to the reader to consider. Knowing quite a bit about the particular sample of participants and features of their context is therefore essential to making any educated speculations about how far generalizations might be made. In the Piasta et al. (2012) paper we do find attention to the question of generalizability. Noting that the sample consisted of teachers and children in classrooms targeted for parti­ cular children (e.g., living in poverty), the researchers noted, “It is unclear whether findings would generalize outside of these settings – for instance, in home-based programs or forprofit private day cares – and to children who are not eligible for these subsidized programs” (p. 398). Without saying directly, the researchers do however appear to assume that the findings would generalize to similar programs with targeted enrollment policies based upon poverty status. Qualitative Research Quantitative research studies are specifically designed around the goal of explaining social phenomena in such a way that the findings can be used to predict what will happen in other, similar situations. The ideas of prediction and generalizability are actually antithetical to the goals and aims of qualitative researchers. In qualitative approaches, the context is important to understanding how and why things happen. As a consequence, qualitative researchers are disinclined to suggest the occurrence in one time, space, and context could be applied to another situation or phenomena in a predictive manner. That is not to say that qualitative researchers do not desire to help readers consider phe­ nomena in ways that will be helpful (or disruptive) or applicable to other situations. Quali­ tative researchers seek to provide a variety of interpretations to help readers consider their issues or phenomena in a multitude of ways, stemming from the basic belief that knowledge and understanding are local, situated, and contextualized. As we learned in Chapter 2, dif­ ferent paradigms across qualitative research have different aims and goals. These range from providing new and enhanced interpretations, to providing an alternative (critical) lens to elucidate and push on power structures, to complete deconstruction of the theories behind how and why things are the way they are. However, while a qualitative researcher desires to provide a reader with a new way to think or a new way to act, this would not be to suggest that it could be predicted what that would look like, as it would involve different actors with different experiences in different contexts. This illustrates the generative nature of qualitative research. When a qualitative researcher discusses what their study means in education (or in the world), that discussion should be consistent with the larger theoretical framing of the study and the aims of the interpretation at hand. Returning to the Souto-Manning (2014) example, her goal was to apply conflict theory to bring an alternative understanding of how conflict can be applied to and used in the early childhood classroom. In addition, she wanted to use this

57

STRUCTURES OF RESEARCH

theory as a means to disrupt the official curriculum to authentically connect to the diverse cultural and linguistic assets the children bring to school. After she carefully laid out the story of the children and the teacher as they embarked on the new way of considering and accepting conflict in their classroom, she ended the paper with a section called, “So What? Implications and Invitations” (p. 629). Her discussion centered on how the story could be used to recon­ sider how people think about, understand, and enact the notion of the “ideal” early childhood classroom – consistent with her goal of disrupting the larger dominant narrative that can ulti­ mately be harmful for some children. She did not suggest that what happened in the class­ room could or should be reproduced, but rather invited readers to reconsider their own policies and practices through an alternative lens with the potential for transformation. Many qualitative researchers will discuss potential implications of their work for other similar settings or situations. Souto-Manning (2014) invited preschool teachers to reconsider how con­ flict is addressed and used in their own classrooms. They will often offer ideas for other ways that their study could be considered or suggest areas for further research. It is frequently the case that a good qualitative study will generate more questions than answers – especially given that ‘answers’ are not really the goals of the research in the first place! And sometimes, since qualita­ tive research provides insight into particulars, it offers fodder for quantitative colleagues to iden­ tify potential trends occurring that could benefit from study from a broader vantage point. Summary There is much information to digest here! By contrasting qualitative and quantitative para­ digms, we have uncovered key beliefs at work that take the researchers in each paradigm in different directions. We urge readers who may feel overwhelmed to take a deep breath and continue reading forward. As more information is added, frequently the picture being drawn becomes clearer. It may also help to review this chapter following our in-depth discussion of quantitative and qualitative methodologies.

A Deeper Look Thus far we have discussed varied approaches to research separately via the quantitative and qualitative paradigms. Yet, as usual in the “real world,” distinctions, boundaries, and defini­ tions are not always so clear-cut. Among researchers there are varied perspectives about the existence of distinctions and the importance of differences. In addition, there are varying levels of advocacy for utilizing multiple, or mixed, methods within a single study, in other words, utilizing both qualitative and quantitative data. Determining the Place of Paradigms Before we take up the topic of multiple methods, we briefly explore how the research paradigms have been privileged across time. As described in Chapter 4, the quantitative methods of psychology have greatly impacted educational research. Throughout history, quantitative methods have been viewed as delivering what is needed to solve issues in education. As noted by Rudolph (2014), “In the face of complex and persistent educa­ tional problems, they [quantitative methods] seem to promise objective results, uniform solutions, and standardized interventions less prone to ideological distortion” (p. 16). Thus, when qualitative methods first secured a foothold among researchers, this group was identified as a “highly suspect newcomer” (Howe & Eisenhart, 1990, p. 2).

58

METHODOLOGIES

As qualitative research gained traction, researchers from both paradigms advocated pas­ sionately for the value of their perspective and methods. Sometimes the advocacy moved into critique of the other paradigm, instigating the expression “paradigm wars.” Those operating from a quantitative base advocated for the strength of traditional, objective scien­ tific methods for establishing cause–effect relationships and solving educational problems. They criticized qualitative methods for providing little in the way of explanation, being too relativistic in their orientation, and lacking in “scientific rigor.” Those operating from a qualitative approach advocated for the richness of their data for understanding the com­ plexity of human situations. They criticized quantitative methods as bringing a limited con­ ceptualization of methods borrowed from the natural sciences that was not adequate for the social sciences, providing limited insight into complex situations and individual perspectives, and protecting existing power differentials in social structures (Gage, 1989). Those who identified as interested in the problems best served with qualitative designs were dealt a difficult blow in the early years of the twenty-first century. At that time, federal education policy in the United States, specifically No Child Left Behind, included a definition of “scienti­ fically based research” that greatly narrowed its scope to quantitative designs. In addition, the What Works Clearinghouse privileged evidence from experimental research in its evaluation of curriculum effectiveness. For qualitative researchers, the effect was to separate their work from what was privileged as “scientifically based.” Ongoing debate resulted in some changes but for many the impact was felt as a devaluing of their research perspectives (Eisenhart & Towne, 2003). In trying to reduce the impact of these policies on the field, the National Research Council published a position statement, Scientific Research in Education (National Research Council, 2002), derived from the input of researchers rather than policy-makers. In this state­ ment there were attempts (viewed by some as not effective) to define “scientific research” in broader ways that encompassed qualitative perspectives. Several years previous to these events, as the use of qualitative methods rose, Howe (1988) declared that positivism “has fallen” (emphasis in original, p.13). Clearly, our description of the ascendency of quantitative methods in subsequent federal policy, epito­ mized by the experimental study, alters that conclusion. Interestingly, in that same paper, Howe described how at the turn of the twentieth century there was a surge of support for positivism in psychology and, as a result, “any research methodology that failed to measure up was dismissed as unscientific.” Thus, there have been waves of support for each of the paradigms since the beginning of the twentieth century. The support for quantitative methods has been so strong at times that the word “science” could be denied association with non-quantitative methods. The various ways in which paradigms have been privileged, or not, are an important part of the background story about research. Mixed (Multiple) Methods Within the last couple of decades there has been increasing enthusiasm for the use of mixed, or multiple, methods within single studies. Johnson and Onweugbuzie (2004) offered a formal definition of this work: “the class of research where the researcher mixes or combines quantitative and qualitative research techniques, methods, approaches, concepts or language into a single study” (p. 17). Mixed methods research can range in how researchers attempt to create these hybrids that draw in various ways upon both research traditions – for exam­ ple, combining methods and combining concepts are different ideas. While there are many approaches to mixed methods research, there are also many per­ spectives regarding how it fits into the bigger picture. And here we refer back to Chapter 2,

59

STRUCTURES OF RESEARCH

where Whyte discussed the philosophical roots of research. Some researchers who support mixed methods research draw from the pragmatic approach we presented there. For them, the task of the researcher is to identify the most appropriate research approach for answering the question of interest. Approaches are neither weaker or stronger, or more or less fitting, in their assumptions about ontology and epistemology. When mixed methods are utilized, there are two dimensions along which the research designs might vary. One is the timing of the various methods and the other is the relative emphasis of each (Johnson & Onweugbu­ zie, 2004). Sometimes both qualitative and quantitative approaches are used within a single study. This would be a design with simultaneous data generation approaches addressing multiple questions. Other researchers might use a sequential design approach within a group of studies. Thus, they might begin with either a quantitative or qualitative design and gen­ erate data in connection with one or more questions. The next phase of the research will then shift to the other type of research design to continue the process of inquiry toward other forms of data (Johnson & Onweugbuzie, 2004). Regarding relative emphasis, in some studies the questions being answered through the two methodological approaches are relatively equal in status in the study (or studies, if this is an example of sequential use of mixed methods). In other studies, one methodological approach is dominant, and the other is utilized to generate supporting data, but functions more obviously as a “secondary partner” to the study’s purposes (Johnson & Onweugbuzie, 2004). It should be clear that the use of mixed methods requires a broad skill set from both researchers and consumers. If the researcher is adept within only a single paradigm, it is challenging to do this work well. For this reason we believe that the strongest mixed meth­ ods research comes from teams where experts within each paradigm contribute to the study. Currently interest in this form of research is strong. One pitfall is the generation of studies in which the additional, non-dominant methodology functions as window dressing, and does not have a reasoned place in contributing to answering the research questions. Instead, it may be added to the study to attract attention at a time in which mixed methods are pop­ ular. The onus is therefore on the consumer to consider the effectiveness of the methodol­ ogy – did the mixing of methods result in a coherent investigation? For some, mixed methods studies make a lot of sense, and they can step into the hybrid space rather easily. For others, navigating this is more difficult, such as when one’s position is deeply tied to the underlying epistemology (with, of course, the exception of pragmatism). In a paper published before the current interest in mixed methods, Hatch (1985) proposed that some qualitative and quantitative researchers see “quite different worlds” based upon their “essential assumptions about how the world works” (pp. 162–163). He cautioned that an approach focused on the utility of the two approaches ignored the basic underlying phi­ losophical differences. While not denying the possibility of high-quality mixed methods research, Phillips (2009) cautioned that some methods do not “sit easily with each other” (p. 185). When co-teach­ ing a research course, the two of us operating with different methodological expertise would often somewhat jokingly ask each other, “Well, why would you ask that question?” This further illustrates the point that mixed methods can present challenges. It seems appropriate to end this chapter with a reminder that all forms of research can be done wonderfully, and all can be done poorly. Both single method studies and mixed methods studies must be carefully evaluated by the consumer. In the next set of chapters we explain the basic methods used by qualitative and quantitative researchers to design studies and analyze data – essential knowledge for being a judicious consumer.

60

6

QUALITATIVE RESEARCH

Aims and Methods

In this section of four chapters we separate the qualitative and quantitative research paradigms to closely examine the key methods and terms of each. This prepares readers for the second half of the book, where we compare and contrast the various ways researchers ask questions. In Chapters 6 and 7 we discuss qualitative approaches to research. We describe how a study is conceptualized and designed, how research questions are developed, and the important relationships in qualitative research between the theoretical perspectives employed in the study and the methods used to address the questions proposed. We discuss basic aims for qualitative studies; for example, answering questions such as “How does this happen in practice?,” “What are the experiences of children in this context?,” and “How can I more fully understand this and what it means to the people involved?” We also present information on reading profes­ sional literature, beginning with a focus on the opening sections of a study report Qualitative research as “naturalistic inquiry” is “discovery-oriented” (Guba, 1978, p. 1); there is minimal “investigator manipulation of the study setting,” and there are no “prior constraints on what the outcomes of the research will be” (Patton, 2015, p. 48). Because the goal of qualitative research is to interpret, explain, deeply understand, and reframe phe­ nomena, it can be a less standardized enterprise than quantitative research. Qualitative research certainly employs standards for quality and rigor, and we can identify basic tenets of study design and implementation. However, the interpretive goal means that there are almost endless ways to design a study to further our understanding of phenomena. Our goal here is not to be all-encompassing, but rather to lay out the range of possibilities and give enough insight so readers can appraise the quality and trustworthiness of qualitative projects in early childhood education. Discussing the concepts and ideas of the next two chapters we draw upon three studies as examples. We chose three studies that draw from the variety of theoretical perspectives as outlined in Chapter 2 to represent a range of paradigms within qualitative research. Further, as we explore in the second half of the book, all early childhood research projects, regardless of paradigm, focus on the various elements of the field of early childhood. For instance, researchers can study the impact of early childhood practice; activities of children, teachers, and teachers’ approaches to their work; curriculum and instructional strategies; or policy implementation. The three articles selected engage with policy in early childhood education at various levels – a state-level comparison of policy enactment and the eventual implications for the practices of educators; how the experiences and actions of individuals within a system can be reconceptualized with implications for broader policy; and, an ethnographic study of a classroom with commensurate analysis of how broader educational and political policy shaped the experiences therein. Thus, we highlight a study of policy implementation down

DOI: 10.4324/9781003354499-7

61

STRUCTURES OF RESEARCH

into practice, lived experience up into policy, and practice within a policy environment to show how that larger environment ultimately shaped inequities within the classroom. First, an article by Graue et al. (2018) represents a critical constructivism paradigm. The authors examined the development and then enactment of early childhood standards (policy) in two states. They studied state systems to understand how standards are developed and even­ tually reflected in the practice of early childhood educators in classrooms. They employed a “multisite, comparative case study” (p. 5) in order to understand and compare conceptualiza­ tion of standards at the state level and then enactment and impact of standards-related policies by district administrators and, eventually, by classroom teachers. They approached the exam­ ination through a socio-cultural lens, assuming that local interpretations and contexts would shape policy enactment and capacity. By comparing the approaches and contexts of two differ­ ent states these authors were able to illustrate the “non-linear implementation” (p. 2) of the broader policies. They used these insights to then suggest recommendations regarding the multiple factors necessary to consider how to ultimately “best support young learners and their teachers” (p. 2) via policy development and implementation. Representing a socio-cultural approach with a transformative/liberatory lens, we explore a study by Navarro-Cruz et al. (2023). These researchers examined the decision-making processes of student parents in higher education as they made their childcare/early learning choices. Here the focus was on understanding the processes and lived experiences of the actors (student par­ ents) in order to gain a fuller view. (As we later discuss, the authors apply a framework in an innovative manner so as to analyze the data critically.) Unlike the Graue et al. (2018) piece where the authors studied the policy and how it trickled down, this study focused on the indi­ vidual experiences of participants to illuminate the complexities of their decision-making and then to make recommendations up into policy development. The study provides insight into how individual agents function within in a broader policy context, and makes recommendations to impact policy in order that it might more equitably serve the actors involved. This study shows how the “student parents’ choices [were] shaped by larger social forces” (Navarro-Cruz et al., 2023, p. 217) and provides insight into which social forces were most impactful for the participants. Based on this insight, the authors suggest ways (from a policy perspective) of how to provide childcare options that better attend to the needs of this group and also better support their family-enhancing educational pursuits. The third article, by Ferguson (2021), represents the postmodern paradigm. In this arti­ cle, using sociomaterial/post-humanist analysis (Actor Network Theory), Ferguson studied a letter writing project in a kindergarten classroom. The study of particular instances (the activities and the context of the classroom, and the material artifacts within) led to exam­ ination of the classroom within the larger local and historical context of the school in order to understand how certain literacies became privileged in the classroom space, ultimately impacting the educational opportunities of the children. This is a study of individual activity within a policy context to illuminate how that policy context shaped the activities and what it potentially meant for decentering and deconstructing the privileging of certain literacies in classrooms. Ferguson “proposes [how] sociomaterial accounts of literacy curriculums … may [account] for structural inequities mobilized in and through schools” (p. 1).

Study Design and Context In basic terms, qualitative researchers must explain how they intend to carry out the study or inquiry of interest. Designing a qualitative research project is a personal endeavor, especially since the researcher is often engaged with the people involved as participants (Rhedding­

62

QUALITATIVE RESEARCH

Jones, 2007). And, given that qualitative research focuses on the hows and whys of human interactions and experiences, the study take places in a particular context. Thus, it seems obvious to state that context and study design in qualitative research are intimately inter­ twined. Here we suggest some basic considerations regarding these two constructs, and provide details concerning these within our example studies. Given the range of possibilities in the study of human experience, designing a qualitative project can be considered an art. Janesick (1998) uses the metaphor of art in describing how researchers go about crafting an inquiry project: the qualitative researcher is very much like an artist at various stages in the design process, in terms of situating and recontextualizing the research project within the shared experience of the researcher and the participants in the study … art forces us to think about how human beings are related to each other in their respective worlds … the qualitative researcher, as designer of a project, recognizes the poten­ tial of design. The design serves as a foundation for the understanding of the parti­ cipants’ worlds and the meaning of shared experience between the researcher and participants in a given social context. (p. 37) All qualitative research begins with an area of interest on the part of the researcher, where the researcher is compelled to dig deeper, find out more, and get to the details of social interactions. There is often passion on the part of the researcher around the topic or area of interest. And, as we discuss, the researcher will consistently need to address bias that always is the foil to deep interest in a context or topic. The complexity of any situation or setting means that there might be any number of questions or areas of inquiry possible within any context. This is demonstrated by the three studies used as exemplars. The goal for qualitative researchers is not to reduce that com­ plexity, but rather to embrace and engage with it. Qualitative researchers often begin with larger, overarching questions and make basic initial study design decisions based on those interests. Unlike quantitative research where the design of the study is conceived before­ hand, study design in qualitative research occurs ahead of the study and during the study, and can accommodate changes after data are collected. In reporting on a qualitative project, it is important that the researcher carefully explains the study design, since the research process will shape the eventual outcomes or findings that are reported. This is different from quantitative work where efforts are made such that the research process itself will does not affect the outcome of the study. Since qualitative study design can and does vary, the rea­ soning of the design must be evident. This enhances the trustworthiness of the project. There are several basic study designs available to qualitative researchers. Again, the options we discuss are not formulaic in nature, though we can offer some basic parameters. The important part of the study design is that it follows the threads of consistency of the project. Essentially the study design entails the number of participants involved and the level of analysis the researcher is attempting. The researcher may study an individual or individuals within a setting. The researcher may study a group of people with the goal of understanding group norms and culture. The researcher may study people within a larger structure with the goal of understanding how and why that structure shapes the behaviors of the people within. Qualitative researchers study “cases” of situations and phenomena, in order to provide rich and detailed examples of what may be happening. And, the way that a “case of some­ thing” may be constituted in qualitative research is quite varied. Patton (2015) notes that

63

STRUCTURES OF RESEARCH

“cases can be empirical units” (p. 259) which in early childhood education might be indivi­ duals (children, teachers, childcare directors), families, or organizations (Head Start site, childcare center, school). Patton suggests that cases might be “theoretical constructs” which could be resilience (in children), quality (of childcare), or life in kindergarten, for instance. Cases can be “physically real” (experiences of student parents in locating childcare), “socially constructed” (whiteness, educational inequity), or “historical/political” (policy study across states). Cases can be in-depth study of one setting or individual or they can be studies of several cases within a setting that are then compared and contrasted. With this to consider, we turn to the beginning sections of qualitative pieces that situate a study to familiarize readers with the kinds of questions qualitative researchers ask and explicate the importance of theoretical perspective in the research. Here we remind readers that one standard of quality for qualitative work is that internal con­ sistency is evident and has clearly guided the process of the researcher. The researcher views the world from a particular vantage point (epistemological stance) and with a working set of theories about how the world operates. This ‘theoretical perspective’ should be evident and explicit in the research report. The kinds of questions being asked should make sense in light of this theoretical perspective. In that way, the methods chosen for data collection and analysis will also be consistent. We explicate this further as we work through the parts of a research report. Below and in the next chapter we will walk through what one will typically see in an article-length presentation of qualitative research. However, it is very important to point out that for qualitative research this is not to be seen as formulaic. We will see in later quanti­ tative chapters there is a basic format and expected sections that indicate the quality of the study when research is reported. Qualitative researchers, on the other hand, are committed to interpretation and deconstruction. It is often the case that research reports are written and presented in ways that are meant to purposefully disrupt convention and show the possibi­ lities of alternative, new, or oppositional understandings of the world. This can make for some dense reading for the student of qualitative research! Qualitative projects can be messy and ambiguous. They are invariably as complex as the phenomena under study. This makes explaining findings and drawing conclusions challenging. Hence, internal consistency of any project is one hallmark of quality that the reader should be able to appraise regardless of methods or presentation style, and regardless of whether the reader likes or agrees with the findings of the study.

Phenomena Under Study/Purpose of the Study/Research Question(s) Recall from Chapter 5 that the goals of qualitative research are not rooted in prediction or generalization. The goals of qualitative projects are varied and include: to understand phe­ nomena more deeply; to interpret a situation or set of behaviors differently; to explain phe­ nomena to expose and/or disrupt norms and power relationships; or, to understand the perspectives, desires, or intentions of the people being studied. It makes sense therefore that the questions asked and the purposes for a study can be as individual as the researcher and the context and participants of the study itself. As we will mention this chapter and Chapter 7, the reader should look for the chains of internal consistency that must work across a qualitative study so that the approach (or theoretical framing) of the study corresponds to the questions asked, connects to the context of the study, is reasonable given the data col­ lected and the analysis completed, and is consistent with the conclusions drawn from the analysis and the interpretation.

64

QUALITATIVE RESEARCH

Phenomena Under Study A study introduction should give the reader a clear idea of what the researcher is studying and attempting to interpret and explain. (This is also connected to the context of the study which should be appropriate to the type of phenomena under inquiry – but more on this later.) Similar phenomena can be studied across the paradigms of qualitative research, with each examination having different purposes. It is worth noting that it is possible to see, across a particular researcher’s body of work, data from the same project – or even the same data – analyzed from varying frameworks to produce new or alternative interpretations (recall the interview in Chapter 2 with Kate Delaney). It is helpful for the reader to ascertain the level of analysis (individual, group, or structure) of the study to understand what the researcher is attempting. Generally, there are three levels of analysis apparent across all the paradigms of qualitative research. These levels of analysis can and do share types of data collection and methods of analysis. The reader should discern if what is being studied is: �

�

�

Phenomenological (individual): The researcher is attempting to explain and help the reader understand the phenomena or experience from the perspective of the individual (s) under study. How do the individual(s) in a setting or context experience the situa­ tion under study? What are their perspectives and views? Ethnographic (group): The researcher is providing insight into the experiences, norms, beliefs, and/or the ways of being a part of a group of people. Here the researcher may describe the actions of individuals within the group, and this is in relationship to the established group norms. Why do people act in the ways they do in certain contexts? How can one explain the decisions or behaviors of members of a group? Structural: The researcher is exposing or illuminating the societal institutions or social structures that shape the choices people make, the ways people think, or how people behave. The researcher may describe how individuals or groups within the social structure under examination act or think, and this is specifically in relationship to the structure. How does social class (for instance) shape the educational experiences of children?

These categories are basic and for purposes of illustration. Given the complexities of social/cultural actions and occurrences, there can be a combination of these in any given study. And, in the human experience, one level (individual, group, societal) has impli­ cations for the others. Sometimes a researcher will explicitly state the level of analysis of the study, but not always. The embrace of complexity in qualitative research generally means that the corpus of data for most qualitative projects is quite extensive. Therefore, it is not uncommon for the researcher to write about a small component of what was studied in a larger project. Another reason is that qualitative researchers are often stymied by the page limits of journals, making it necessary for them to choose a small part of a project about which to write. It is also common for researchers to use the same corpus of data across several papers, slicing the data slightly differently for different audiences or to focus on a different interpretation of, or lens for, the data. Unlike quantitative projects where it is unacceptable to “fish” the data for outcomes that were not planned for ahead of time, it is likely that in the course of a quali­ tative project new and unexpected questions will arise. In fact, reinterpretation of data is common and encouraged.

65

STRUCTURES OF RESEARCH

Making a determination of what is being studied is the first part of the thread to ascertain. Understanding the level of analysis and what was actually studied should lead the reader to a discussion of why it was important to study that particular phenomenon. Turning to our three example studies, in these cases, the abstracts and introductions help­ fully spell out the purpose and focus for each study, and the units of analysis. Graue et al. (2018) state in the abstract (p. 1) that they utilized a multi-state comparative case study in New Jersey and Wisconsin. The abstract indicates a “Purpose/Focus of Study” – “to compare the role of standards in Pre-K programs … analyzing standards conceptualization and enactment by administrators and teachers.” The abstract further indicates that the team interviewed “state actors” and then also engaged in weekly observations and teacher interviews in Pre-K class­ rooms in elementary schools, Head Start, and childcare centers across a variety of school districts (rural, midsize, and urban) in the two states. In the Navarro-Cruz et al. (2023) piece the abstract (p. 217) indicates that they “conducted 36 in-depth interviews with student parents attending a 4 year university in the Western United States.” There is an “Introduction” section (p. 217) which also explains the purpose of the study as exploring how the “undergraduate student parents … obtained childcare for their young children.” For Ferguson (2021) the abstract lays out how the study of a kindergarten classroom in New York City supported the study of the network in which it existed to make the case for differently understanding how that network acted to impact educational opportunities within the classroom. In all of our sample studies, the level of analysis and what is being studied are clear, but in some presentations of qualitative research it can take a bit more digging on behalf of the reader to ascertain this. This task is, however, worthwhile. It is important to note here that quality of the project is not necessarily determined by the writing or presentation style of the researchers. We turn next to the purpose of the study and depending on what that is, the presentation style may be more intricate. Again, the apparent threads of consistency are a good marker of quality in qualitative projects, even though there is not a strict formula for presenting data. So, if the reader needs to dig a bit more to ascertain the purpose or level of analysis, this should not necessarily be viewed as problematic. As in Ferguson (2021), researchers may be applying a particular and complex theoretical lens to data analysis, and this is the focus of their write-up rather than the specific activities of the participants. Purpose of the Study Because the same study data can be used for different purposes, describing the intended purpose of both the study in general and of any particular analysis or report of research is important. This discussion can most often be found in the introduction and there may be a specific section indicating the purpose of the study or the specific research questions. There will likely be some accounting of why this study is important or significant and how it helps us to understand a phenomenon more deeply, more complexly, or in a different way. Also, there may be discussion of why it is important that the data or a particular phenomenon are examined from the indicated interpretive framework. If the purpose is not explicitly stated, it may be inferred from the review of the literature, or made evident when the researcher discusses the research methods of the project. Whether specifically stated or not, the purpose of the study will be elaborated upon and justified within the review of literature associated with the study. The Graue et al. (2018) study is a good example of a research report (in this case a journal article) that takes a smaller slice of a much larger project, giving a particular purpose to the part of the study upon which they report. The purpose of the larger study was to explore state early learning standards’ development given “growth in public Pre-K programs guided

66

QUALITATIVE RESEARCH

by state learning standards” (p. 1) and what eventually guided practice in Pre-K programs. Yet, for this particular paper the researchers note that they explored “how Pre-K policy is enacted in policy development and political exchange, constructed through administration and experienced by relevant stakeholders” (p. 5). The purpose of this paper, as mainly noted in the “Methodology” section was to describe the whole landscape of this process across levels of standards’ interpretation and execution to examine what that meant for how teachers in classrooms engaged with or enacted the stan­ dards in their classrooms. With a large data corpus, one can imagine many slices of analysis this research team might take. They could examine many parts of the process in more depth for different audiences. For instance, they might focus on the administrators within the process to understand their knowledge and experiences with enactment of early learning standards. This team also lays out a purpose of utilizing an alternative lens to differently understand the phenomena at hand. They note (Graue et al., 2018, p. 5): “In line with growing recognition that local practices and culture shape the [standards’] implementation process, we are guided by an analytic lens that views ‘policy as practice’ (Levinson, Sutton, & Winstead, 2009).” They describe this as a “critical approach” (p. 5), with part of the purpose of the study to apply this approach to understand the complexity and contextualized nature of policy enactment. This connects to their socio-cultural theoretical perspective that “policy implementation is not linear, with policies directly transported from the intention to imple­ mentation. Instead, policies are taken up in local social contexts where they interact with existing systems and participants’ beliefs” (p.5). Hence, the researchers are employing a particular lens to differently understand how a policy landscape operates and the complexity of its impacts on teachers’ lived classroom experiences. Research Questions Given the wide range of purposes and the goal of interpretation and understanding of qua­ litative research, it is challenging to categorize the types of questions a qualitative researcher might ask. We can say questions in qualitative research do not put forth a hypothesis or presumed outcome as they do in quantitative research – though researchers do enter a pro­ ject with theoretical approaches and assumptions in mind. As mentioned, researchers often begin with a larger or overarching area of interest. Once a study is designed and a context for the research is determined, sub-questions may be developed that connect more specifi­ cally to the setting of the project. However, as discussed earlier, there are various paradigms within qualitative research and the types of questions that interest any given researcher also stem from that researcher’s view of the world (theoretical perspective) and the unit/level of analysis that the researcher seeks to undertake in a project. In some approaches to qualitative research (grounded theory in particular) there is actually a caution about developing research questions at all before col­ lecting data, as the questions and theories are expected to arise from the researcher experi­ encing the phenomena. Agee (2009) suggests that the initial questions posed in a qualitative project can serve as a plan for the research, represent a beginning point for the project, and can evolve. Agee notes, “First iterations of questions are tentative and exploratory but give researchers a tool for articulating the primary focus of the study” (p. 433). Sometimes the research questions in a study are stated explicitly as questions and these are often found at the beginning or end of the introduction. Other times the researcher will express the goals of the research in statement or narrative form. Turning to our three

67

STRUCTURES OF RESEARCH

example studies, for Graue et al. (2018), the research question is found at the end of the introduction. They explicitly state, “In this paper we describe how Pre-K programs across three districts in each of … two states responded to early learning standards to address the following research question: What guides Pre-K programs in two states?” (p. 3). In the Navarro-Cruz et al. (2023) piece, a particular conceptual framework is applied and drives the data analysis, with data collected specifically for this report. Given this, the research questions are stated directly and also found within the “Introduction” section. The authors state: “Our research questions are: (1) How do student parents in higher education choose childcare for their children ages birth to 5, and (2) What factors influence those choices?” (p. 217). The research team also notes here that they are using a framework called an “accommodation model” which allowed a critical perspective different from other studies. In contrast, Ferguson (2021) never explicitly states a research question. This piece is a case where a study was undertaken (a 9-month, ethnographic study of “literacy curriculum enactment” (p. 3)) in a kindergarten classroom in a New York City public school, and the researcher applied an alternative lens (Actor Network Theory) to the data analysis. The study described, emerged from Ferguson acting as a participant observer in the kindergarten as he was observing the enactment of literacy curriculum therein. His application of the theory emerged from his engagement, rather than being determined a priori. Ferguson may have had a very general research question as he entered the study site, and he indicates that he was drawing from post-humanist theory to understand how objects in a setting function as “active mediators” (p. 4). Then, as he engaged, additional questions came to formation. He eventually decides to “look out” from the classroom ethnographic data and then “reconsider how the curriculum was not only constituted by networks of circulating materials, but also by networks circulating students’ bodies into unequal school spaces, fueling changes in the school’s enrollment and funding” (p. 1). As we are beginning to see, it can be challenging in qualitative research to isolate and discuss the various parts of a research presentation, since each part is interconnected and dependent upon other parts. The threads of consistency only make sense when the whole comes together. Indeed, when Janesick (1998, 2000) describes qualitative research design she uses the metaphor of choreographing dance where all of the various parts and movements come together to make a cohesive whole – and attention to each of the movements is necessary, but will not make sense in the absence of placing it within the whole. Next, we discuss the theoretical framing of a study which should help the reader to contextualize it within an approach to how the world works and also within the appropriate body of literature.

Theoretical Framing and Review of Literature One defining characteristic of the qualitative paradigm is researchers embracing that all research is ideologically driven – meaning that the researcher’s frames of reference and beliefs will, in fact, bias the research. There is no attempt in qualitative research to suggest that any research project is value-free. And, given the multiplicity of paradigms that con­ stitute the larger qualitative paradigm, there are a wide variety of theoretical perspectives a researcher can employ in designing a study and analyzing data. Given this, it is important in a presentation of research that authors discuss and explain the perspectives and worldviews upon which they draw and the theoretical framework employed. Although mentioned previously, it is worth reiterating that in qualitative research, literally the same body of data (or parts of) could be analyzed from different conceptual frameworks within the qualitative paradigm, and different kinds of findings or new

68

QUALITATIVE RESEARCH

understandings could be derived. It is thus incumbent upon the authors of a study to make it clear for the reader how these perspectives have shaped the research questions, the design of the project, the findings that emerged, and the conclusions that were drawn. Here it makes sense to define a uniting theme of qualitative research, which is the concept of contextualization. The context of any qualitative research project is a centralizing force of the project. Contextualization is the process through which meaning is assigned to a phe­ nomenon via the context in which it occurs. Also, meaning can be assigned to a context via the way a phenomenon plays out. The context of a project is literally the place, space, materials, and participants from where and whom data are collected. The researcher will interpret and give meaning to the space, place, and phenomena observed by placing their analysis of the data collected within the larger theoretical framing of the study – con­ textualizing the analysis within a larger body of research or literature. Similarly, a theoretical perspective can gain meaning in the practical world via contextualization within the activities or interactions of the phenomena being studied. In the following sections, we will help readers make sense of this by taking a closer look at the sections of a research presentation where the theoretical framing of the study is detailed and the study is placed within the larger body of literature related to the topic of the project. Theoretical Framing In a qualitative write-up, there will often be a section explicitly labeled ‘Theoretical Fram­ ing.’ However, if not, the reader should be able to ascertain the author’s perspectives via the introduction to the study, and through the review of the literature that is related to the justification of the study. Since the threads of consistency of the project should be in play, it may be that the theoretical framing appears as part of the discussion of the methodology. As we detail in the next chapter, the methods of data collection and analysis should make sense given the theoretical framing of the study. The explanation of the theoretical framing can occur on several levels. The author may position him/herself socially and culturally as a means to situate or contextualize the origins of the theoretical approaches s/he takes. This may be more or less important depending on the purposes of the study. For instance, it may be quite pertinent that a research study about the play interactions of children of indigenous origin be conducted by a researcher who is also of indigenous origin. There is no right or wrong way to do this – a non-indigenous researcher can also research this topic – but in that case, social positioning might need attention, with the researcher explaining how they dealt with bias, acknowledged ideology, or blindspots. If the data are taken from a broader study – which they often are – the researcher may explain the purposes and theoretical framing of the broader study and then go on to further explain the theoretical framing particular to the part of the study upon which they are reporting. We will now turn to Ferguson (2021) to illustrate a complex theoretical framing and a unique application of a theory to understand literacy equity in a classroom from a broader socio-cultural framework. Ferguson includes an entire section, “Opening the black box of the literacy curriculum” (pp. 5–6) – about 900 words – to explain and justify his layered approach to the study. Ferguson uses an ethnographic approach to data collection (participant observation in a classroom for an extended period of time). This suggests his valuing of the close study of the lived experiences of humans within a setting. On page 4, he specifically notes that he has applied “explorations of sociomaterial, posthumanist, and new materialist perspectives.”

69

STRUCTURES OF RESEARCH

Ferguson draws from the theory that agency and activity of the humans in a context are important and also that the material aspects of a context actively mediate (in this case) lit­ eracy practices in the classroom. As Ferguson studied the human activities, this theoretical framing also meant that he closely observed the ways that the available material objects in the classroom actively mediated the learning there. He explained, citing Latour (2005), that ignoring the role of materials as having agency results in an inability to examine the internal workings of the enacted curriculum, making “the joint production of actors and artifacts entirely opaque” (p. 183). Ferguson goes on to explain his application of “Actor Network Theory (ANT)” (p. 5) to support the perspective that it is critical to “interrogate the associations of humans and non-human actors” (p. 5) in a setting. For Ferguson, this meant that he considered the relationships of “teachers, students, and materials” to be “symmetrical” and this also allowed a view outward from the classroom to understand how materials can be indica­ tive of the influences of systems or actors not present – a textbook publisher, for exam­ ple. Ferguson then describes a purpose of the study which is to further extend ANT to be able to describe systems of power influencing the classroom under study to specifi­ cally explicate systemic or structural inequities in education that make available the par­ ticular material actors in a classroom that shape learning experiences therein. If the threads of consistency hold, we will see then how the methods he chooses to analyze the data fit with this framing of the world. Review of the Literature The review of literature in a qualitative study serves the basic purpose of making the case for why it is necessary to conduct the current study in the particular way the researcher has planned. If the reader has figured out the basic purpose of the study, a perfectly legitimate set of questions are, “And? So what? Why do we care?” The review of literature should provide these answers. It should provide the justification for deeper examination of the phenomena and for the particular design of the study. The researcher will use existing lit­ erature to help the reader to understand what other related research has been conducted and where there are gaps in our knowledge or views. The extant literature may also be used to demonstrate how or why a more nuanced or alternative understanding of what the researcher is studying is necessary to promote productive change, disrupt power relation­ ships, or problematize what is seen as the norm. Writing a good review of extant literature is a learned skill and not an easy one at that. The reader should be thankful if the author is able to wade through what is usually a huge body of available research and provide a logical roadmap for the reader of why the current study is important to undertake based on what has and has not been discovered before. As an example, let us examine the Navarro-Cruz et al. (2023) piece to see how they make the case of the importance of understanding the experiences of student parents searching for childcare using a particular conceptual framework – the “accommodation model” (p. 219). The authors state early on that they conducted “36 in-depth interviews with student parents,” because “limited research exists on student parents’ childcare choices and the fac­ tors that inform their decisions” (p. 217). However, the reader needs justification of why this phenomenon is important to comprehend in more depth, and then how the accom­ modation model helps to elicit this deeper perspective. What has been studied already about this topic? Why is this particular conceptual approach and set of methods important? It is key to note here that this is a qualitative piece published in a high-status journal where

70

QUALITATIVE RESEARCH

quantitative studies have predominated. Hence, for this audience, it was also likely important for this team to explain why this more detailed approach is beneficial. Noting that student parents in higher education settings are an under-researched group in general, the authors set up five sections for the literature review to make the case for a more complex understanding of the experiences of this group, particularly in relation to the childcare choices they make. A stated goal of this study is to inform policy given that “pre­ vious studies have found that ensuring student parents have access to affordable quality childcare can triple timely graduation attainment” (p. 217). The five sections review available literature related to the following: � � � � �

Student parents in higher education Student parents’ financial obstacles Childcare affordability for student parents Student parents’ childcare choices Institutional support for student parents

In Section 1.1, the authors explain the importance of studying this topic: adequate support services in higher education for student parents dramatically impact their ability to persist in their studies and to complete their degrees. They make the case that child­ care is a consequential factor for student parents, and understanding their decisionmaking processes could encourage higher education providers to build more effective structures to ensure success of this group. Following on from this, Section 1.2 provides the evidence that financial concerns are “one of the biggest obstacles student parents face” (p. 218), and that childcare comprises a large proportion of the financial landscape for families. Section 1.3 continues the logic trail to provide research evidence that access to “secure and consistent childcare” is a central priority for student parents. Difficulties with finding affordable childcare are related to the potential for withdrawing from higher education. The authors go on in Section 1.4 to review available literature related to available childcare options (indicated as family, friends, and neighbors, home-based childcare, and center-based childcare) for student parents and how they go about making the choices they do. The authors give overall proportions of which options were accessed in the studies presented and in connection with work schedules, race and eth­ nicity, and income status. Here the authors make the case that while the concept of “choice” in this endeavor is often invoked in studies examining this phenomenon, choice conceptualized in this way centers the onus for positive outcomes within an individual actor, rather than accounting for systemic factors that also impact what is available as the “choice.” Finally, Section 1.5 supplies information about the decreasing availability of childcare on campuses despite the federal law that “schools cannot discriminate against pregnant and parenting individuals” (p. 219). Finally, the authors provide two examples of federal funding programs that address the issue at hand. Hence, as can be seen from this example, the theoretical perspective and the situating in the larger body of research should give clues to the reader about what to expect from the data collection and data analysis methods. Navarro-Cruz et al. (2023) provide the reasoning in the literature review for application of an alternative conceptual framework to understand the phenomenon in a more nuanced manner. The authors guide the reader to understand why the methods chosen for this study are justified and important and, in this way, the reader is easily led into the next two sections of the report – “Conceptual Framework” and “Methods” (p. 219).

71

STRUCTURES OF RESEARCH

Now that the reader has a good understanding of the beginning steps of qualitative study design and the importance of the initial framing of a study, the next chapter delves into the nuts and bolts of qualitative projects – methodology, methods of data collection and analy­ sis, and reporting of the findings of the project.

Reading and Understanding a Qualitative Study Phenomena under Study/Purpose of the Study/Research Question(s) Task: Determine what the phenomenon under study is. What did the researcher actually study? Look for: This is often explained in the abstract or at the end of the introduction Task: Figure out why the researcher is undertaking the study. This should help you further determine what they are attempting to explain, deconstruct, more fully understand, or reframe. Look for: The study introduction should provide an accounting of why this study is important and why using the indicated interpretive framework is important. How will this study help us to understand the phenomenon in question more deeply, more critically, or in a different way? The actual research question is often explicitly stated, but not always. One can often find the research question(s) at the end of the introduction, but they may also be found within or at the end of the review of lit­ erature, or in the section explaining the research methods. Theoretical Framing/Review of the Literature Task: Determine the theory or theories from which the researcher is drawing to examine, interpret, or differently explain the phenomena under study. Look for: The theoretical framing of the study is often explicitly stated and discussed. This most often will occur within the review of the relevant literature. Explanation of this sometimes also happens at the beginning of the ‘Methods’ section. For some studies, the theoretical framing is implicit and needs to be inferred from the works cited or from how the review of the literature is discussed. Task: Map out how the study is framed within the literature base. Look for: To begin to build understanding of the literature review, look for the headers used to identify topics. Read the topic and concluding sentences of sections or paragraphs carefully to consolidate your understanding of main points. Examine how the researcher set the stage for this study by describing what had been found in previous research and how this study addresses gaps, provides alternative inter­ pretations, deconstructs mainstream or canonical ideas, or helps us to understand something more deeply.

72

7

UNDERSTANDING ANALYSES IN

QUALITATIVE RESEARCH

This chapter describes the primary data sources, and methodological and analytic strategies used in qualitative research. Readers will gain an understanding of qualitative researchers’ approaches to the process of interpreting data, including the use of descriptive and inter­ pretive coding schemes, organization of data, and eliciting meaning from the data. We also offer information on understanding the presentation and discussion of research findings in the professional literature. We begin with a discussion of quality and rigor in qualitative methods and projects. Keeping in mind the threads of consistency that are the hallmark of a good qualitative study, the reader might consider that there is no “right” method for data collection and analysis in qualitative research. The goal of the qualitative researcher is to tell the story of what is occurring in the setting – to understand the actual experiences of the people involved. The types of data collected and how data are analyzed should make sense when considering the theoretical framing and stated perspectives of the researcher. They should have a logical flow from the types of questions that are being asked, or the goals of the project. The story that is finally reported should flow from and make sense given the framing of the project and the data collected. That said, unexpected findings can emerge from a qualitative project. The threads of consistency support the believability of the findings, whether anticipated or unexpected, and the trustworthiness of the interpretations. The researcher might not find what they looking for and certainly will find things they were not looking for. But even so, the project must be designed and the findings conveyed so the reader can trust what is being reported. This brings us to the important issues in qualitative research of trustworthiness and believability, which are related to the quality and rigor of a project (Loh, 2013). For the qualitative researcher, trustworthiness and believability must flow through and be carefully considered for the entirely of the project, and are particularly important in choosing and implementing the data collection and analysis methods. Qualitative researchers often turn to the concept of verisimilitude, which the American Heritage Dictionary defines as, “the appearance or semblance of truth; likelihood; probability.” In order for findings of a quali­ tative project to carry this characteristic, many would argue that there needs to be evidence of care, thoughtfulness, and rigor taken with the data analysis and the interpretations. And, this is of course up for interpretation by the reader. One way that qualitative researchers attain verisimilitude (and quality and rigor) is through what is called triangulation of data, where the researcher uses several kinds of data to provide evidence for the claims they might make. Janesick (1998, pp. 46–47) (citing Denzin, 1978) explains five types of triangulation:

DOI: 10.4324/9781003354499-8

73

STRUCTURES OF RESEARCH

1 2 3 4 5

Data triangulation: using a variety of data sources Investigator triangulation: using different researchers or evaluators to examine data Theory triangulation: using multiple perspectives to interpret a single data set Methodological triangulation: using multiple methods to study a single phenomenon Interdisciplinary triangulation: using other disciplinary perspectives to inform research processes

With this in mind, we now turn back to the sections of a research report and to our exemplar studies. Recall that Graue et al. (2018) studied development of standards in early childhood education at the state level and compared how these were eventually enacted in classrooms. Navarro-Cruz et al. (2023) interviewed student parents attending a 4-year institution to ascertain how they made choices concerning childcare. Ferguson (2021) studied the literacy practices in an urban kindergarten classroom and reported on how outside forces and structures shaped access to opportunity for these children.

Data Sources and Methods of Data Collection As mentioned in Chapter 5, it is the case in qualitative research that the researcher is con­ sidered the “tool” of data collection and analysis. Qualitative researchers (particularly in postmodern times) seek to explain the phenomenon under study from the perspectives of those participating in the project; to pose a description and an interpretation of what, how, and why something is happening, how relationships play out, and why people behave how they do. Qualitative research seeks to describe and deepen understandings of the reality as experienced by the participants of the study. This puts a great deal of responsibility on the qualitative researcher to collect data that allow them to obtain a picture of phenomena that is as close to the actual happening as possible. And, since interpretation is key, the ability to make accurate or appropriate interpretations is predicated on a high quality data set that provides enough information that the interpretations are believable. Following are key considerations and key methods of data collection. Time in the Setting Time is a very important factor in the design of a qualitative project. The researcher will be spending a great deal of time in the research setting and with the research participants in an effort to tell the story of the phenomenon in a comprehensive and trustworthy manner. The researcher must spend enough time in a setting and enough time with the study participants so as to be able to gain their perspectives and to understand the hows and whys of the situation. There is no prescription for how much time is enough, but it is evident in research findings when the time spent is not enough. Projects can be point-in-time in nature or can be longitudinal (data collected over longer period of time), and this decision is made in the study design process we discussed in Chapter 6. For example, a researcher may study how a policy is initially implemented in a childcare center and may spend only a few weeks at the site collecting data. Or, a researcher may want to see how the policy implementation takes place and evolves over a longer period of time and study how that happens in the site over the course of a year. Ethnographic studies where the researcher is trying to explain the behaviors and norms of a group may take place over years. For example, referring back to Heath’s Ways With Words (1983), she

74

UNDERSTANDING ANALYSES IN QUALITATIVE RESEARCH

spent the better part of a decade living in the three communities she studied, collecting data and coming to understand the literacy practices of each. The research design must include spending enough time and care, for instance at the onset of a project, so that trust and rapport can be established with the participants. In a research report, the author may talk explicitly about this process, particularly if it is especially important to the kinds of data that were captured. When children are involved, as in the case of research in early childhood, developing age-appropriate rapport and trust is especially important. The researcher(s) will be collecting data within the lives and spaces of the parti­ cipants. Therefore, the participants need to feel comfortable sharing this space, and need to trust that their stories will be considered and told in a thoughtful manner. In a journal-length presentation of qualitative research, the author may simply explain how long was spent collecting data and what kinds of data were collected, but there may not be enough space for lengthier discussions of the study design issues that arose rela­ ted to time in the setting. In many cases the reader has to trust that the amount of time was appropriate, though if findings seem inappropriate or off in some way, it is not a bad idea to look to see if the amount of time spent seemed reasonable given what the researcher was studying. As examples, the Graue et al. (2018) study was a large-scale qualitative project with a large corpus of data and involved several researchers. The manuscript explicitly notes (p. 9) that data were collected over an 18-month period with teams working in two states. Data included document analysis of state-level and district-level data, in-depth interviews with various actors, classroom observations, and lesson plan analysis. A case report was developed for each state. Conversely, the Ferguson (2021) study was conducted by the author himself, in the kindergarten classroom he examined. Ferguson reports collecting data for two days per week in the classroom over a span of 7–8 months. During this time, using “a network case study approach” (p. 6), he interviewed the classroom teacher regularly, interviewed administrators and parents at the school, and analyzed text related to the history of the school and the neighborhood. Finally, the Navarro-Cruz et al. study team conducted one­ time interviews with participants, but the process of recruiting and interviewing new indivi­ duals occurred over several months. As is evident, both larger scale projects with teams of researchers and the model of the “lone” researcher are appropriate for qualitative projects. Context of the Study and Study Participants The context of a qualitative project is very important as it provides connections and meaning to the phenomenon being studied, and thus to the findings of the project. For instance, if one desires to study play interactions amongst preschoolers (which could be studied for a variety of reasons), there may be key differences if those interactions are studied in children who attend a Head Start in a low-income, urban neighborhood vs children in attendance in a home childcare setting in a small town vs children in attendance in a private, high tuition childcare center in a wealthy suburb. The qualitative researcher seeks to capture the details and nuances of the interactions within each of these settings. It is possible that a qualitative researcher might study play interactions in different contexts in order to make comparisons but s/he might also find it equally compelling to deeply examine the interactions in just one setting. The important part of the research design is that there is careful thought about and rationale for the context of the study, and then careful consideration of how the context shapes what the researcher sees and interprets.

75

STRUCTURES OF RESEARCH

Access to and entry into a site to conduct qualitative research can be fraught. Qualitative researchers try to understand how and why people think and behave in the ways they do, which automatically entails that trust, rapport, and relationship are key parts of the research process. There is an intimacy involved in qualitative data collection, which can raise a whole host of both methodological and ethical considerations with which the researcher must grapple throughout a project. In fact, a regular critique of this paradigm is that qualitative projects occur in settings of convenience for the researchers. This means that a setting for a project is chosen based on the ease of access to the setting on the part of the researcher. For example, the researcher who wants to study play interactions in preschoolers might choose a site either of a teacher already known to the researcher, or in a school close to the researcher’s university campus with whom the researcher already has an established rela­ tionship. This kind of “convenience” decision can have both positives and pitfalls. Since a journal-length reporting of a project with large amounts of data can be con­ straining for a qualitative research report, there may not be ample space for deep descriptions of the context of the study and the participants therein. However, there must be enough description that the reader can understand the situation and why these participants were chosen as the ‘case’ for this phenomenon, Simply, the reader should determine what the context of the study is and who participated in the project. For the reader, this might be intertwined with the determination of the purpose of the study that was discussed in Chap­ ter 6. This information may be found in the introduction to the study, and is often explained in more detail in the section about study methods. The reader should also look for an explanation (even if brief) of why a particular setting and why the specific participants were chosen. And, this explanation should make sense in light of the indicated theoretical framing and the questions for the project. Further, if the project is a smaller part of a larger study, or the data collected are from one part of a larger system, the researcher must provide enough information for the reader to make sense of the reporting. The Graue et al. (2018) study is a good example of a larger project from which a smaller slice of data was chosen for the study we are examining. These authors carefully explain the contexts and participants from the larger study, and then provide the details as to why the data from particular participants were reported upon in this article. These researchers col­ lected data over 18 months, in two states, and across multiple settings. For this research report, they focused on how state-level standards sifted down into classroom practice to provide evidence that it was not a linear process, and that contexts mattered greatly. Indeed, with the amount of data collected they would have the opportunity to focus on a single district or even a specific classroom. They note they also “conducted case studies of two focal children in each site, following them and their families for the pre-kindergarten year” (p. 9), though this was not the focus of this research report. The reader can make sense of how and why the contexts and participants were chosen through consideration of the study design, questions, and theoretical framing of the study. Graue and team eventually found that standards’ development and eventual enactment occurred in interactions between or amongst people and was spontaneous given a particular context. The threads of consistency for this project are very apparent. This is not a phenomenon that could have been captured through a survey. It is a very distinct kind of interaction that needed to be studied carefully across actors and in the context of classroom teaching. A final note here on participants is exemplified quite well in our Ferguson (2021) study. We have already mentioned the importance of developing relationships with participants in qualitative studies so as to attain their trust and their willingness to share their experiences with the researcher(s), and ultimately with a public audience via publication. While this

76

UNDERSTANDING ANALYSES IN QUALITATIVE RESEARCH

relationship is always important, in some studies it is more key to the study outcomes and thus is explained in more detail. Ferguson, for example, indicates that he was a participant observer in the classroom for the time of the data collection. This meant that he both observed and took notes and images, but he also participated with the children and the teacher in the day-to-day activities. Ferguson takes time to explain his relationship-building with “Clare,” the classroom teacher in his study, which was a key association. Ferguson essentially shadowed her (“walking interviews,” p. 6), asking questions, taking notes, and taking photographs as she prepared the classroom for the children to arrive in the fall. He shares that he took, “copious images of classroom space and materials, recording our conversations while moving furniture and materi­ als into place, over a 5-day period before the school year began” (p. 6). Ferguson eventually uses the data to highlight how inequities seeped into the fiber and function of Clare’s classroom. It would have been important for Clare to trust Ferguson to be a part of the lives of the children in the classroom, and that he would accurately interpret the happenings he observed. Studies like these make for interesting fodder for the consideration of ethics and how the design and questions of a research project need to be considered throughout the project. Key Methods of Data Collection Here we provide a more detailed, yet still summary-driven, overview of the kinds of methods of data collection that qualitative researchers may use. Again, recall that the kinds of data being collected should make sense given the questions being addressed, the theoretical fram­ ing that the researcher has indicated, and the type of context that is under study. Qualitative researchers are always on the lookout for new and better ways to collect data that will help them to more closely or accurately capture the actual happenings that they are studying. Technology has been incredibly useful in this endeavor. In the sections that follow we discuss some of the main types of data that are collected across qualitative projects. Observation Skilled observation of the phenomenon under study is the foundation of much qualitative research. Researchers will spend a great deal of time with people and in settings (e.g., class­ rooms) and carefully observe the happenings. It is important to qualitative researchers that the settings be as natural as possible. There is what Patton (2015) calls “folk wisdom” about observing in a setting, that the researcher must overcome in order to engage in the thorough and systematic observation required for rigorous qualitative work. Making assumptions about what is going on in a setting must be avoided to the best of the observer’s ability. Observation is more than simply going in and watching what is going on. The observer must carefully “see” and capture in written (or in some cases video or audio recorded) form all of that which is occurring in a setting. And, because there is so much that could be observed, this is where a good study design will help the observer focus their observations. However, the observer must also be open to noting the unexpected or to “seeing” that which was not anticipated. While observing, researchers employ various methods to capture what it is they are observing. More traditionally the researcher will take written “jottings” while observing and then write up more lengthy and descriptive memos and field note reports immediately fol­ lowing an observation. The idea is to capture as accurately and descriptively as possible what one is observing. Nowadays, given the flurry of technological advances, researchers may also videotape and/or audio tape during an observation. Remote video observations are possible as are observations of virtual behaviors.

77

STRUCTURES OF RESEARCH

Depending on the goals of the project, researchers will be involved with the participants to varying degrees while observing within the settings. This ranges from complete non-par­ ticipatory observing to being a full participant-observer. In the research reporting, it is expected that the researcher will explain and justify the chosen level of involvement, the impact of that involvement on the participants, and the implications of the extent of their involvement on the types of interpretations that are made. Interviewing Qualitative researchers spend time talking with their participants and others involved in the context of the study, asking questions, clarifying what they have observed, gaining insight and information, understanding background experiences, and checking their interpretations of the data. Interviews are a staple of qualitative work. Interviews can take many forms from formal to informal, from scripted to semi-structured to open ended (Strauss, 1987). Interviews can occur at any phase of a qualitative project and can serve a variety of purposes. They can be conducted with individuals or with groups of people. Sometimes researchers will interview participants individually and then also bring them together into groups to deepen discussions. Focus groups can also be drawn together to obtain a representative interpretation of a phenomenon when individual interviews are not feasible. Interviews are very often recorded in some way and then later transcribed ver­ batim for use in data analysis. Often researchers will develop a pilot interview that they try out with people outside of the possible participant pool to ensure that the questions are designed to capture the appropriate information. Questions are then revised as necessary. The important part of interviews is that there is some logic to how they are conducted in relationship to the necessary consistencies and goals of the project. The decisions about what kinds of interviews are used, how often, and how in-depth need to be explained and justified by the researcher. In a journal-length write up of a study it is rare to see the actual interview questions that were used, though one may see reference to the kinds of questions asked and to the larger goals of the interviews for the project. Interestingly in the Graue et al. (2018) exemplar study, they do share the actual interview protocol used for the classroom teachers (p. 8) to give an idea of the types of questions they were asking the teachers to ascertain the implementation of standards in their classrooms. Collection of Artifacts Artifacts are often collected as a part of the data corpus in order to provide further evidence of the researchers’ interpretations of the phenomena under study. Cultural artifacts can also provide insight into what participants value (or not), how they go about their lives, their relationships to/with the contexts in which they exist, and clues as to how a participant is experiencing or interpreting phenomena. In educational research, artifacts are often textual or graphic in nature and can include such things as notes, drawings, flyers from events, diary entries, emails, lists, artwork, posts from social media, photographs, course assignments, journals, etc. If the study is of organizational structure, the supporting documents to the functioning of the organization may be collected and examined. These could be bylaw documents, policies and procedures documents, forms, evaluation materials, websites, etc. We return to the Ferguson (2021) piece to provide an example of how a researcher explains the types of data collected, particularly in connection to the post-humanist theo­ retical framework he employed. This is a critical and unique approach to studying inequity in

78

UNDERSTANDING ANALYSES IN QUALITATIVE RESEARCH

a classroom. Ferguson does an exemplary job of explaining why this approach is necessary to provide the “networked” understanding he intended. In his write up, the threads of con­ sistency amongst the theoretical framing, the study questions, and the methods for data collection are clear. Ferguson devotes two full sections – “Opening the black box of the literacy curriculum” (pp. 4–6) and “Contextualizing the network case study” (pp. 6–8) – to detailed explanation of how his methods derived from the theoretical framing and why this was a critical way to design the study. He explains that post-humanist and socio-material approaches suggest the interconnectedness of the human actors and the materials in any setting; a “joint co-pro­ duction of actors and artifacts” (Latour, 2005, p. 183 as cited in Ferguson, 2021). Ferguson discusses that: socio-material perspectives have not only recast notions of who/what is included as an actor, but also how agencies get distributed across both local and distant net­ works, and how those boundaries become blurred as disparate actors come together in moments of enactment. (p. 5) Given this framing, Ferguson uses “Actor Network Theory, (ANT)” (p. 5) as his metho­ dology directly connected to his theoretical approaches, and as a way to address critiques of post-humanist studies. ANT is a “set of tools,” used “to interrogate the [detailed] associa­ tions of human and non-human” actors in a setting. The approach assumes that the “rela­ tions of teachers, students, and materials [are] symmetrical, insisting that all actors are doing something.” Ferguson notes a critique of these tools has been the close attention to the particulars of a case using these methods, which has “come at a cost to sufficient attention [being given] to … social structures … context and reflexivity … and politics.” Thus, Fer­ guson strove to address these critiques by “looking down at the particulars” and “looking out for other forces privileging school literacies” (p. 6). Ferguson provides an exemplar of carefully highlighting for the reader the threads of con­ sistency of a project. These kinds of sections can help a reader to determine if the logic of the project is consistent and evident, and that the explanation lends support to the believability of the findings. A reader may not agree with what the author ultimately finds; however, the quality of a project rests both on a solid study design, but also on the author’s ability to nar­ rate and articulate how and why the design and study process decisions were made.

Making Meaning of the Data – Analysis and Interpretation So, here the researcher sits with what seems like mountains of data, and their job now is to tell the story about these data so that they authentically represent the participants’ experi­ ences of the phenomenon. It can be a daunting task! Observation field narratives have been written up and organized. Audio recordings of interviews are transcribed verbatim. Video recordings are transcribed and perhaps cued or cut into episodes. Artifacts are cataloged and organized. The process of data analysis is where these data get interpreted and then con­ verted into findings – into the story of the phenomenon, according to the researcher. Truth be told, there is not a specific time when a qualitative researcher officially “starts” data analysis. From the moment data collection begins it is likely that preliminary analyses have also begun. Analysis of qualitative data is time consuming and detailed, and is as indi­ vidual as the researcher and the design of the project. It is very non-linear and it will be

79

STRUCTURES OF RESEARCH

difficult to describe here in a way that reflects what actually happens as data are analyzed. There is no checklist or recipe for how analysis goes, although most qualitative researchers can agree on some basic principles of rigor and quality. While a researcher can lay out a plan as one gets into the data, twists and turns can occur that require different approaches or additional phases of analysis that were not pre-planned. It can be the case that initial analyses occur that then suggest new or more in-depth data collection. As we have seen, parts of a data corpus can be analyzed using a different lens than other parts. As Patton (2015) notes, “Direction can and will be offered, but the final destination remains unique for each inquirer, known only when – and if – arrived at” (p. 521). Despite this complexity, there are some basic processes and procedures that many researchers use that we outline here. In journal-length write-ups it can be argued that explanation of data analyses often gets short-shrift, unless the researcher is using a particularly innovative or unusual process that requires more in-depth explanation. This occurs because explaining the analyses means describing a very detailed and messy process with many starts, fits, and unexpected turns of events. Yet the way the analyses proceeded is the hallmark of rigor and quality for the pro­ ject. And, if the methods for analyses are not explained, how the researchers arrived at the interpretations may not make sense to the reader. It is important, therefore, for the reader to ascertain as much information as possible for there to be a sense of trust that the analyses were carried out with open-mindedness, thoughtfulness, detail, and organization. Identifying Patterns and Themes Beginning analyses are carried out by reading and organizing the data. Researchers some­ times talk about this phase as ‘reducing’ the data. This does not mean that one is trying to reduce the complexity of the information. But, given the amount of data that is possible in a qualitative project, there must be a means for the researcher to examine the data in man­ ageable chunks or units. Oftentimes this starts by the researcher reading and rereading a set of interview transcripts to begin a process of identifying themes and patterns that arise. Careful notes are kept during this process and the researcher may begin to write what are called descriptive memos so that they have a way to track emerging thoughts and ideas. A common method is called “constant comparison,” as described by Glaser and Strauss (1967). As the researcher begins to examine a new piece of data (the next interview for example) more closely, the new information is being compared to the ideas generated from the last piece of data. As emerging patterns and themes come to light, this comparison process continues to confirm and disconfirm initial theories and interpretations. As this initial analysis process carries forward, many researchers will use this phase to begin to develop a list of codes to be used for more detailed analysis. This phase also provides a structure for the next phases of analysis, which we will describe shortly. The researcher is trying to create an evidence trail that supports the interpretations they are making about the data and ultimately about what was observed. This work provides credence for the scaffolds of believability of the analysis and interpretation. Throughout the whole process the researcher makes claims about what was observed, and provides data to buttress those claims. Coding Data and Developing Categories Coding of data is a common method of qualitative analysis (though certainly not the only way to go about data organization). While the researcher is working through the initial identification of themes and patterns, they begin to develop a list of codes that can be

80

UNDERSTANDING ANALYSES IN QUALITATIVE RESEARCH

applied to help describe the data in greater detail. Coding is an organizing framework for the researcher to use as the analysis grows and becomes more complex. It is a way to mark common instances across a data set to see how similar occurrences might play out in differ­ ent parts of the project. Saldaña (2012) describes a code as “a word or short phrase that symbolically assigns a summative, salient, essence capturing, and/or evocative attribute for a portion of language based or visual data” (p. 3). Once the researcher develops the list of codes to be used – and this is via inductive analysis of the data – one process is to return to the data and go through it line by line and assign a code to words, phrases, and chunks of narrative to mark when one of these is an instance of something the researcher believes they are finding. For example, one code might be “Race” and anytime anyone talks about race or the situation has to do with race, the data chunk would be coded as such. This allows the researcher to go back to these instances as a group and further analyze and interpret what might be going on within the data set in relationship to that code. The descrip­ tive codes can then also mark where further interpretation is required. So, in the example of “Race” the researcher may assign an additional analytic sub-code that describes more specifically what the instance was referring to. Is the instance an example where someone is talking about his/her own racial identity? How race operated in a certain situation? Racism? There are many ways to approach and think about coding and this is determined through the conceptual framework of the project. Some codes will be determined ahead of time (a priori), and other codes will emerge through the process of identifying themes and patterns and are then applied to the rest of the data. Qualitative analyses are not complete after one time perusing through the data. Oftentimes data are analyzed many times, and potentially via different perspectives. Once coding is ‘done’ researchers may begin to develop broader descriptive and analytic categories to frame what they see in the data. Writing Analytic Memos Broadly writing analytic memos is a process akin to journaling. As the researcher moves through the data analysis process they might keep memos or write-ups about what they are seeing, or ideas about what is emerging from the data. This memo-ing process can occur throughout data collection and all throughout the analyses. In some cases, these memos are then organized and used as additional data. Sometimes researchers will use the codes and categories they developed and analyze the information in the memos. Sometimes the memos will be used as data triangulation to support claims the researcher makes. The memo-ing process can be especially important when there is a team of researchers analyzing the data. A memo provides the breadcrumb trail that explains what the researcher was thinking while reading through or coding data and others can use this either as a springboard for further analysis or as confirming/disconfirming evidence of emerging themes and patterns. This memo process is very rarely described in detail in journal-length reporting of research. However, it can be a very important part of the data analysis process. It is some of the “behind the scenes” work that is not usually discussed but was likely an important organizing feature of the study in question. Within-Case and Across-Case Comparisons Once data are coded, the researcher can begin to examine the ongoing analyses both within and across the data in the various “cases” that may be a part of the data collection. How this might occur depends on the design of the study, of course. We use the Graue et al. (2018)

81

STRUCTURES OF RESEARCH

article to provide an illustrative example of how this might have played out. Note that description of this level of analysis is not likely to appear in a journal article, so we surmise the process here to provide the example. Data collected for this study included: state-level documents; interviews of state officials and legislators; identification of three sample districts in each state that “would illustrate the state’s mature policy implementation” (p. 7); interviews of administrators from each sample district; weekly “ethnographic” observations of pre-school classrooms in each district; and multiple semi-structured interviews with the classroom teachers. A within-case analysis would have been carried out as follows, beginning with the determination of each “state” as the case. All of the data related to one of the states were organized together. The activities and artifacts were examined over time to suggest patterns and themes depicting the policy implementation for each state. Within each state, districts were chosen and similarly examined, and then classroom observations and interviews ensued. Coding of the data would be ongoing to pro­ vide a structure to the within-case analysis. A “case” is then developed, in great detail, which describes the activities of each state in relationship to the questions of the study. The across-case analysis could then proceed. A case as described above is developed for each state, and then sub-cases developed for each district and classroom. Further analysis ensues where those cases are compared with each other within the larger state cases, given the established codes and categories. Then, the researchers would have conducted cross-case ana­ lysis across the two states. It is possible within this process that different categories emerge and the cases can be reanalyzed in light of those categories. The analysis, which one can imagine to be layered and complex, proceeds to the point at which the researchers feel a report of the findings is valid. And findings can be drawn from various phases of a project. For the article we have used here, the findings reported are of the policy implementation in each of the states. Reports could be generated that detailed the policy implementation of each state separately. A report could compare different kinds of districts within or across the states. A study that focuses on practices in individual classrooms, or the approaches of individual teachers is pos­ sible. Additionally, the researchers collected data such that they could develop and report on case studies of individual children. Thus, the level of analysis for any report of findings can be varied. This is how and why qualitative analyses can be said to be generative. That is, good, detailed, and rigorous analysis should produce a variety of reportable findings. As we see, qualitative analysis is a completely iterative process. One can think of the pro­ cess as an investigation where the researcher is searching for evidence (both confirming and disconfirming) to support conjectures about what is going on. Researchers talk about fol­ lowing leads, hitting dead ends, and “breaks” in the analysis. Researchers have to be open to interpretations they did not expect and to find disconfirming evidence of things they did expect. The analyses themselves often end up producing more data than expected. The range of data analysis methods and tools used across the three exemplar studies is a good representation of the variety of processes that are used. The Graue et al. (2018) and Ferguson (2021) studies were both projects that gathered large amounts of data over longer periods of time. No doubt that for both of these studies, the analyses took place in phases. And in both reports, a shorter section of the paper (four paragraphs in Graue et al. and five paragraphs in Ferguson) is devoted to describing the analysis. The Navarro-Cruz et al. (2023) project was shorter term and relied mainly on interview data, supplemented by document and observational data at the campus childcare center. They conducted a total of 75 one-hour interviews of student parents, and then culled to the 36 parents with children under the age of five. (Thus, these researchers also would have the opportunity for further analyses.) This research team was applying a novel and critical

82

UNDERSTANDING ANALYSES IN QUALITATIVE RESEARCH

conceptual framework – the “accommodation model” (p. 219) – suggesting that it allowed for “a deeper understanding of the processes and outcomes of decisions.” A section explaining this choice is included along with a section “Methods” (six paragraphs) devoted to outlining collection and analysis methods. Analysis methods are relatively standard for this study – open coding, intentional application of the accommodation model, development of sub-themes, and then line-by-line axial coding. We have described above a range of data collection and analysis methods. However, it is always up to the reader to determine if sufficient information has been provided that one can trust the analysis procedures.

Findings and Discussion of Findings This section of a journal article is where the researcher tells the story that emerged from the data. Qualitative research is always narrative and gives details and description of the phenomena under study. Additionally, the researcher will provide interpretation of what occurred – the “so what?” part of the explanation. It is always a balancing act for a qualitative researcher to include enough description and enough interpretation to make the story believable. From a consumer’s perspective this is usually the most interesting part of the article; however, the reader should be diligent (not just read the discussion) in going through the steps described above to ensure that the story is accurate and credible. Also, in this section(s) it is important that the researcher help the reader to see how threads of consistency of the project have been drawn through the project. This means that it should be clear in the tell­ ing of the story how the theoretical framing generated the question that should be apparent to and connected to the findings. In many cases the findings narrative is organized around the themes or categories that came from the analysis. The researcher will explain what occurred in a part of the data (make a claim) and will provide evidence from the actual data to support that claim. Quotations and data excerpts are common elements, and this is where qualitative researchers can strug­ gle. One must provide enough data evidence so that the claim is plausible and makes sense to the reader. Here also the researcher may draw from extant literature to support claims, particularly in light of the theoretical/conceptual framing. Given page constraints it is common for readers to want more data to support the claims. Hence, the author must do a very careful job of explaining how and why the evidence supports the claims being made. And, there is never enough space to do this! It can be the case that graphic representations (of various kinds) of the data are presented as a means to help the reader understand the way the researcher presented the interpretations of the phenomenon. A graphic representation can be used to present how the data were utilized to develop new theory and can show how a theory (generated from the data) might work on a larger scale in a more universal manner. (Though recall that for the most part generalizability is not a goal for the qualitative researcher.) The Navarro-Cruz et al. (2023) article illustrates the process of making claims, providing evidence, and providing interpretation, and in a bit more structured format than is typical in qualitative reporting, probably given the type of journal in which it was published. The sec­ tion in this article is titled “Results,” and begins on page 220. The discussion is structured around two main themes – “4.1 Childcare obtained” and “4.2 Factors that influence child­ care decisions” (p. 221). Section 4.2 is further divided into sub and sub-sub sections as we can see below:

83

STRUCTURES OF RESEARCH

4.2.1 Family necessity 4.2.1.1 Class schedule 4.2.1.2 Work schedule 4.2.1.3 Combining care due to school and work demands 4.2.2 Family financial resources 4.2.2.1 Financial concerns about the cost of daycare 4.2.2.2 Qualifying for subsidized care 4.2.2.3 On-campus subsidized childcare availability 4.2.3 Family beliefs and preferences 4.2.3.1 4.2.3.2 4.2.3.3 4.2.3.4

Mother is best Family is best Socialization and academic preparation Desirable traits in a childcare center

4.2.4 Community context 4.2.4.1 4.2.4.2 4.2.4.3 4.2.4.4

Proximity to home On-campus childcare Limited infant care Time is fixed at a childcare center

4.2.5 Social networks 4.2.5.1 4.2.5.2 4.2.5.3 4.2.5.4

Social networks as a source of support Social networks sharing information Misinformation Faculty support

In each sub-section, the authors follow a consistent pattern of making a claim from the data related to the theme (telling part of the story), giving data excerpts that provide examples of the claim (providing evidence), and then explaining to the reader how that data excerpt is an example of the claim they are making (providing interpretation). Throughout, the authors also draw from outside literature to help explain to the reader why the claim is important, and how the interpretation enhances or extends our current understanding of the phenomenon. In these final sections of qualitative work, authors will often draw conclusions, explaining why and how the claims and interpretations presented are important or how they may be applied to other situations. The authors may suggest implications for their work. They also may discuss the limitations of their study. Finally, they may propose how their study is a springboard for further research. Often these sections are shorter. The reader should look to see if the conclusions make sense in light of the data, are not overstated, and fit with the threads of consistency related to the original theoretical framing of the project. NavarroCruz et al. draw the conclusion and make the case that family decisions are not characterized by making individual choices from wide-ranging options: “Student parents’ childcare choices are complex. Drawing upon the accommodation model, our findings indicate how student parents’ choices are shaped by larger social forces, particularly family necessity, family finan­ cial resources, beliefs and aspirations, community context, and social networks” (p. 226).

84

UNDERSTANDING ANALYSES IN QUALITATIVE RESEARCH

These researchers go on to suggest that these detailed finding might “inform higher education policy and practices” (p. 226) to better support student parents. They also suggest several avenues for further research. So, welcome to the messy, complex, and non-linear world of qualitative research. In this and the previous chapter we have provided an overview of the main approaches, methods, and processes of qualitative work. This should not be considered an exhaustive treatment but, rather, as a guide to the basics to help a reader appraise the quality and believability of qualitative work. Next, in similar fashion, we move to an overview of the basics of quantitative research.

Reading and Understanding a Qualitative Study Methods: Context of the Study and Participants Task: Determine what the context of the study was and who participated in the research. Look for: Note the descriptions given of the participants in the research. How much does the researcher tell you about their backgrounds? What do you wish you knew that isn’t reported? How sufficient does this group of participants seem (e.g., number involved, their local contexts)? How did the researcher find the participants, and, if pertinent, group them (note that if this is not detailed, most likely it was a matter of convenience)? Task: Appraise the description of the context(s) of the study. Look for: Does the researcher give you enough information about the context? Remember that context is very important in qualitative research, so is there enough detail provided that the reader can understand how a phenomenon may play out? What is missing that leaves the reader with questions? Note that in some case study research, the context may be included in the description of the cases, rather than in a separate section. Task: Determine the relationship between the researcher and the study participants. Look for: Does the researcher discuss this, and what might this relationship mean for the later interpretations of the phenomenon? Do there seem to be any conflicts of interest or biases, and how does the researcher explain these? Was the researcher an outside observer or a participant observer? And, if a participant, what was the level of participation and does the researcher account for the impact of this participation on the outcomes of the study? Methods: Data Collection Task: Identify the kinds of data that were collected, and what methods were used for data collection. Look for: Do the data seem to be relevant to the questions asked? Do the data seem to fit within the theoretical framework and the purposes of the study? Is there enough data to provide evidence connected to the questions? How long did the researcher spend in a site or with the participants? How did the data collection process impact the researcher’s relationships with the participants? Did the data collection methods change at all over the course of the study? Why? Look for explanations of triangulation of data where various types of data are collected to provide evidence of later claims.

85

STRUCTURES OF RESEARCH

Methods: Data Analysis Task:Identify the methods that the researcher used to analyze the data in order to create the story or narrative that will later be told about the data. Look for: While there are some standard methods of analyzing data used in qualita­ tive research, the methods are not standardized. Especially in journal article length presentations, the explanation of data analysis may not always reflect the complexity of what the researcher actually did. Does the researcher provide an adequate accounting of the methods used to analyze the data? Can the reader make sense of how the researcher may have come to the conclusions that the researcher later draws? Does the analysis seem to make sense given how the researcher has framed the study? Does the analysis engender enough evidence to support the claims the researcher makes? The reader may have to read the findings and discussion of the study and then come back to the analysis to see if the analysis provided the evidence to make the researcher’s explanations and interpretations plausible or reasonable. Findings and Discussion of Findings Task:Appraise the story/narrative that the researcher tells that emerged from the researcher’s analysis of the data. Look for: Study findings are written in narrative form, and tell the story of the data as interpreted by the researcher. There may be graphical representation in the form of descriptive tables or other visuals/models included to enhance the reader’s under­ standing of the story. Look for dialogue and data excerpts that support the claims the researcher makes. Do the findings make sense given the kinds of data collected? Does the story provide insight into the research questions? Do the findings make sense within the theoretical framework proposed? Does the story provide insight into the phenomenon in ways that connect to the stated purpose of the study? Do the findings seem plausible given the types of data and analysis conducted? Task:Evaluate the researcher’s discussion of the findings which indicates the “so what” of the study. Look for: Are the claims made about the research consistent with the purposes of the study, the research questions, and the theoretical framing of the study? Do the findings presented support the claims that the researcher makes? Does the discussion adequately explain how the study enhances our understanding of the phenomenon, reframes the phenomenon, or pushes the reader to understand the phenomenon from a different lens? The researcher may provide further discussion of the possible implications of the findings and/or the limitations of the study. Do these make sense given the kinds of data collected?

86

8

QUANTITATIVE RESEARCH

Aims and Methods

As with the qualitative chapters, our aim in this and the following chapter is to explain the basic principles of conducting research in the quantitative paradigm, including determining the research questions, enrolling a group of participants, deciding upon instruments and measures, collecting data, and analyzing the data. Along the way we add to the terminology central to understanding the work of quantitative researchers. Recall from Chapter 5, quantitative studies are developed out of the existing literature. A researcher typically begins with an area of intellectual curiosity. At this point the researcher’s role is to find out “what we know” about this phenomenon. Researchers look at both the findings of previous studies and how the studies were conducted – the types of instruments used, the group of participants involved, and so on. From there the researcher designs a new study to add another installment to the account of what we know. Essentially the process can be pictured as, “We know this, and what about …?” In quantitative research most decisions about how the study proceeds are determined at the outset. This reflects the ideal of objectivity, in that the researcher works along the lines of logic established within the research questions and study design, rather than reacting to what happens along the way. Occasionally, during the analysis of data the researcher makes some decisions to conduct post-hoc analyses (defined as analyses conducted to follow-up the analyses that were pre-planned at the start of the study). However, there are two reasons why this is done very cautiously. First, the more analyses conducted, the more likely the researcher will report a result that is deemed to be statistically significant but in fact is not. We will explain this in the next chapter, but for now remember that analyses are planned and conducted to be parsimonious – more is not better. Second, when researchers decide upon analyses based upon how the data appear after-the-fact, skepticism can arise. One colloquial term for this is a “fishing expedition.” The analyses should be part of the logic established within the research design and not depend upon what looks fruitful to explore after exam­ ining the data. Any post-hoc analyses should thus be supported within the logic model – for example, finding that some data are highly related and then taking statistical means to account for that pattern of relationships without a change in the research question. As we discuss concepts in this chapter and the next, we draw upon three studies for examples. Here we briefly describe each study. Denham et al. (2015) examined preschoolers’ executive control in relation to the chil­ dren’s age, gender, and the level of their mothers’ education (heretofore the executive function study). They used two measures of executive control: cool executive control (orderly and flexible responses for shifting attention and focus) and hot executive control (regulation of emotional arousal and impulses). They hypothesized that older children, girls, and children from higher socioeconomic levels would score higher on these measures.

DOI: 10.4324/9781003354499-9

87

STRUCTURES OF RESEARCH

Secondly, they examined how executive control was associated with teachers’ reports on children’s social competence and adjustment in the classroom. They expected that cool executive control would be more highly related to children’s classroom adjustment and hot executive control would be more highly related to children’s social competence. Sheridan et al. (2014) randomly assigned Head Start classrooms to participate, or not, in an intervention designed to engage the families in understanding their children’s develop­ ment, forming goals for their children in partnership with program staff, and responding sensitively to engage their children to meet goals (referred to as the family engagement study). During the two years that children and families were participating in the program, the researchers collected data by periodically observing the parent and child as they played together in tasks set up by the researchers (e.g., book reading, puzzles, free play, and clean up). They expected that as the intervention proceeded, the children whose families participated would show increases in learning-related behaviors (e.g., persistence, positive affect) compared to children whose families did not participate. In addition, they tracked the family members’ self-reported depression to examine if this had any influence on how the intervention impacted the children’s learning-related behaviors. In our third study, a group of European researchers studied early childhood teacher education students in four different countries – the Netherlands, Wales, Germany, and Finland (the play study). van der Aalsvoort et al. (2015) showed four video clips of young children engaged in activity to their participants and had them write responses asking if they would label the activity as play. They subsequently categorized the participants’ explanations for their decision (play/not play) by identifying if they had drawn upon four characteristics commonly attributed to play – it is fun; children participate voluntarily; rules are applied by the children; and there is no externally-imposed goal for their activity. The categorization process served to convert the participants’ written responses into numerical data; in this case, the number of times a characteristic was evident in the responses. These data were then analyzed to examine differences according to country.

Determining Research Questions The purposes of quantitative studies fall into a small number of major categories. Researchers’ questions can be descriptive, relational, comparative, or causal in their nature. A key term to introduce here is the variable. Variables represent those factors and concepts that are measured within the study. Examples of variables can include the children’s age, their teachers’ score on a self-efficacy survey, or teaching methods used in the classroom. Descriptive Questions and Purposes Quantitative studies are invariably descriptive. In presenting numerical data for a variable, researchers are describing a phenomenon or characteristic. The question at the heart of this approach is along the lines of: how much of this and that do we observe? For example, Holmes et al. (2012) were interested in how often teachers in kindergarten through thirdgrade classrooms used concrete teaching materials in their vocabulary instruction. They observed lessons using a coding scheme that categorized teaching materials within a hier­ archy that began at one end with oral description/definition of vocabulary, progressed through other categories (e.g., models), and ended with real-word materials in their natural context. The data in this study were reported as the percentage of observations within each of the levels of the hierarchy with regard to various teaching contexts (e.g., grade level,

88

QUANTITATIVE RESEARCH

content area of lesson). In the presented data, there were both differences and similarities in the percentages reported among the categories. But this is all we know – that vocabulary teaching was approached differently, and sometimes not so differently, in various contexts. No further statistical analyses were performed to examine whether these differences might reflect the random variation one would expect in any real-world phenomenon or might be patterned in identifiable ways that allow interpretation (e.g., there is greater use of certain teaching materials in certain grade levels). The Holmes et al. study provides us with a window into classrooms, but when we see differences we are not able to determine if they are random or systemically related. Because of this, quantitative researchers most often take that next step with statistical analyses. This allows them to ascribe more meaningful interpretations to their data. Yet, even though more statistical analysis may be given, most quantitative articles begin with descrip­ tive data in the presentation of the study’s results. Such descriptions are offered to char­ acterize the study’s participants and to provide information about the study’s variables – for instance, the average score and range of scores on instruments. Relational Questions and Purposes Many quantitative studies focus on the relations among variables. These relationships are referred to as correlations. The research questions might include: � � �

How strong is the relationship between this variable and that variable? How does a third variable affect the relationship between this variable and that one? How accurately can we predict a variable based upon knowing these other variables?

In Denham et al.’s (2015) executive function study, one of the research questions was relational in nature. Specifically, the researchers wanted to “describe associations of … [children’s execu­ tive control] with teachers’ reports on children’s social competence and classroom adjustment” (p. 215). For example, were children rated as more socially competent if they scored higher on executive control? Denham et al. complicated the relationships they explored by estimating how social competence was related to children’s age, maternal education, and gender. Then, they looked at how much of the remaining variation in children’s social competence ratings was associated with the executive control variables. We return to this example in Chapter 9 to examine how to read and understand the results of the analyses. It is important to note that a correlational relationship found between two variables does not imply that one variable causes another. For example, among children, height and weight tend to vary together – taller children tend to weigh more and shorter children tend to weigh less. The relationship is not perfect, certainly, but it exists. Note we did not use adults as our example because adult weight at any one point in height tends to differ more, meaning there is less correlation. But still, height does not cause weight. In correlational research, even if one of the variables comes temporally before the other variable, the first cannot be assumed to cause the second; it could be that another, unmeasured, variable was involved. For example, suppose a researcher recorded how often teachers provided pretend play for children during the early months of the school year and then assessed children’s vocabulary in the later months. After finding a positive correlation, that higher levels of pretend play were associated with higher vocabulary in children, the researcher proclaims that pretend play builds vocabulary. One thing not measured was how often teachers talked with children during pretend play versus other activities such as block-building. It may not

89

STRUCTURES OF RESEARCH

be pretend play in itself that “causes” vocabulary growth, but how the teacher participates in that activity, which ostensibly could be applied to other activities. Therefore, a maxim to always keep in mind is “correlation is not causation.” With a correlational analysis we can say that one variable tends to ‘go along with’ another variable, and if we know one variable we can make predictions – with limited certainty – about another, correlated, variable. Comparative Questions and Research Just as prominent in the literature are studies where the research questions focus on differ­ ences, or comparisons, among groups. The nature of this question is whether one might expect to see the differences found between or among groups by chance alone or whether this is unlikely, implying that a variable is operating systematically to generate a difference. Returning to the executive function study by Denham et al. (2015), one of the research purposes was to “evaluate differences in [executive control] according to child characteristics (age, socioeconomic risk status, and gender)” (p. 215). The researchers compared the scores of girls and boys, and of children at different age levels to see if the group scores were sys­ tematically different from each other. Similarly, van der Aalsvoort et al. (2015), in the play study, examined how the responses of teacher education students varied across the countries represented in their sample. However, knowing that groups are different does not tell us why they differ. For example, in the executive function study Denham et al. (2015) found that boys and girls differed on the measures of executive control, with girls scoring higher. But we cannot determine what led to these differences. Are girls naturally more inclined toward these skills? Or were they more likely to have been expected and supported to develop these skills? Or were they rewarded more for demonstrating the skills? Causality cannot be assumed and the limits of the answers to these research questions are important to remember. Causal Questions and Research As we have described, many research questions cannot address causality. It is not surprising, however, that researchers (and policy-makers) do want to ask questions about whether a variable causes change in other variables. Yet, this type of research is more complex to undertake, and often more costly and difficult. It requires the researcher to control some variables (e.g., what is taught, how it is taught, to whom it is taught). For these reasons, it is not the major form of research within the complex operation of education, although many might wish for the type of answers it appears this research could provide. Causal questions, such as does an intervention (e.g., a particular curriculum) lead to desired outcomes, are addressed through quasi-experimental and experimental (true) research. In quasi-experimental studies, researchers use existing groups of participants but work carefully to ensure that the individuals in one group (i.e., intervention group) are similar in key char­ acteristics to the individuals in the other group (i.e., comparison group, who do not receive the intervention). To illustrate, Winter and Sass (2011) tested an intervention designed to prevent obesity and increase children’s school readiness. They used four Head Start centers located within the same agency, with the intervention conducted in two of the four centers. Comparisons among the four centers were conducted to assure that the participants were similar. Thus, the researchers were able to state that the two groups of centers (intervention and comparison) did not differ systematically in regard to children’s gender, age, ethnicity, and family income. Furthermore, all centers used the same curriculum.

90

QUANTITATIVE RESEARCH

The findings of a quasi-experimental study are generally regarded with some caution because within existing groups there may be an unmeasured phenomenon impacting the outcome. Let’s return to the study by Winter and Sass (2011): the obesity prevention pro­ gram involving parent education. We do not know if there were any systematic differences among the four centers in regard to the effectiveness of their general family engagement efforts that might have impacted the intervention. If, perhaps, an intervention-site center had a very effective family engagement process that altered how families responded to the intervention, it could impact the study findings in undetected (unmeasured) ways. In a true experiment, rather than quasi-experimental research, random assignment functions to ensure that groups are as comparable as possible, and that factors vary at random within each group. This is because participants have an equal chance of being in the treatment or control groups. Thus, any background variables should range in similar ways and not be sys­ tematically different between the groups. Still, this form of research should not be taken as establishing definitive proof – problems may arise as we describe below, and what is not con­ trolled or measured by the researcher may exert an influence (Deaton & Cartwright, 2018). That said, it is difficult in educational settings to randomly assign children to schools and classrooms. As an alternative, then, it is the classrooms or programs that are randomly assigned. It is important that the units (e.g., classrooms) being randomized be numerous enough for comparable groups to result. If too few units are randomized, it is more likely the groups might not be so comparable. Consider tossing a coin. If tossed 50 times, there is a reasonable expectation of approximately 25 each of heads and tails. However, if tossed only six times, five or even six heads might result. Odds tend to even out with more repetitions. In the family engagement study, Sheridan et al. (2014) used a true experiment, or as it is sometimes called, a randomized control trial. They examined an intervention designed for engaging families with their children’s learning and development to support social-emotional and language/literacy skills. The researchers randomly assigned Head Start classrooms in 21 different buildings to either treatment or control groups. Obviously, these 21 research sites reflect a far more expansive, and expensive, study than the four centers that made up the quasi-experiment performed by Winter and Sass (2011). In sum, we have described the major types of questions posed by quantitative researchers. Each type of question has its own set of possibilities and limits – what it tells us, what is left unanswerable, and where we should exercise judicious caution in making meaning of the story. In regard to equity, it is important for the reader to attempt to understand the assumptions that underlie the research questions. For example, are deficit perspectives perceptible in the mechanisms that make up the phenomena being studied? Are groups marginalized within a society assumed to reflect, or expected to reflect, mainstream norms?

Enrolling Participants While the literature review is a section of its own in a study report, the other topics we discuss in this chapter are typically described within a section entitled methodology or methods. These sections describe the nuts and bolts of how the research was conducted. The participants recruited into a study are referred to as a sample. The use of statistical analyses requires a sufficient number of individuals to contribute data to the study; too few results in an “underpowered” study in which potential differences may not be detected. Therefore, quantitative studies usually have more participants than qualitative studies, and often the difference is quite large.

91

STRUCTURES OF RESEARCH

In theory, the participants in a quantitative study represent a larger population of indi­ viduals. Remember previously we have explained that quantitative researchers assume the results of their study, using a sample, can be generalized to a larger group of individuals, the population. Of course, for this assumption to stand, the sample must adequately reflect key characteristics of the population. Yet, researchers rarely define the link between their samples and a population, so often this is a judgment the reader must make. Who is the larger population represented by this sample? Obviously, we would not generalize findings from a study using a sample of 3-year-old children to children who are 6 years old. Much research has been criticized for being conducted with samples of White children while operating under the assumption that the findings could be generalized to children uni­ versally. Nowadays the scientific community is more likely to acknowledge that unique cultural and contextual experiences make these assumptions questionable. Often, however, it is not so clear-cut. Can we generalize a study of children attending the American Head Start program in one geographic location (and thus meeting eligibility requirements con­ cerning family income) to represent a population of all children who attend programs with income eligibility guides, or even all Head Start programs? We hope readers realize how fraught the issues are – how do we understand what larger group might be represented by a particular study sample? It is still up to individual readers to consider how far they might go in assuming that what was found true for this particular sample might apply to other individuals outside of the study. Ideally, the sample is carefully chosen by the researcher. In reality, locating individuals, programs/schools, and/or families who are willing to take part is challenging. Research requires a willingness of the participants to open themselves up to the researcher’s time requests, tasks, and observational gaze. Also, the researcher’s own time and expense con­ straints often limit the bounds of what is possible. Thus, many samples are reflective of the most suitable arrangements possible, given time, expense, and willing volunteers. Samples in quantitative research studies come in many forms and are chosen by various methods. For example, study samples may truly be formed by convenience, more often than supposed given the ideals of science. Sometimes researchers exert some criteria, or purpose, for determining who will be in the sample. Finally, sometimes researchers work to create what is called a stratified sample, attempting to replicate within a sample some characteristic of the population. As an example, in Milwaukee, where we both have lived, the demographics of individual schools in the public school system reflect the highly segregated living patterns in the city. Thus, many schools have a majority of one racial or ethnic group. If we wanted to create a stratified sample of Milwaukee schools, we would do the following. We would select a number of schools with a majority of Black children roughly equivalent to the percentage of such schools in the overall school district population (e.g., 60% of the sample compared to 60% of the district overall as a hypothetical). As well we would want the number of schools serving largely Latino/a children also roughly equivalent to the percentage of such schools in the overall district. If we selected only schools in Black neighborhoods, we would not repre­ sent the overall district well, because exactly who attends schools in the various neighborhoods of the city varies tremendously. Single schools do not look the same as the overall district demographics. In conclusion, researchers typically work with constraints in forming a sample for their studies and it is important that they describe this sample as thoroughly as possible for read­ ers. However, it is up to the reader to evaluate the sample. Who is represented in the sample? Who is not represented? Does the sample seem adequate to test out the researchers’ questions? What are the limits of generalizability with this sample?

92

QUANTITATIVE RESEARCH

Deciding Upon Instruments and Measures In quantitative studies data are collected via the measures utilized by the researchers. Deciding upon the instruments and measures to be used in a study is a key task impacting the potentials and challenges inherent in any piece of research. Before going any further, it is important to define how researchers conceptualize variables for their studies. Recall that in Chapter 5 we explained that quantitative researchers measure the concept of interest via the use of numbers. The concept of interest, or construct, being studied, is denoted by a variable which is represented numerically via measurement strate­ gies. The theory underlying the research describes how the variables are connected. We will use Denham et al.’s (2015) study of executive function to explain. In Denham et al.’s study there are several variables. Executive control was one construct of the study. The researchers used one instrument as a measure of hot executive control, one variable of the study. They used a second instrument as a measure of cool executive control, another variable. Denham and colleagues were also interested in several other areas of children’s functioning. These constructs included social competence and classroom adjustment. These two constructs were measured by two different instruments completed by the teachers of the children. In the study, the measures of executive control were used to predict children’s social competence and classroom adjustment. Because they are predictors, or come first in the research question, the measures of executive control are independent variables. Indepen­ dent variables are considered to impact or have some influence on other variables. The measures of social competence and classroom adjustment are dependent variables because they come last in the research question, and the researchers are interested in how they are impacted, or influenced, by the independent variables. The researchers theorized: “we expected that CEC [cool executive control] would be more highly associated with classroom adjustment than HEC [hot executive control], with the converse true for social compe­ tence” (Denham et al., 2015, p. 215). Remember that the independent variables (CEC or HEC) come before the dependent variables (classroom adjustment and social competence) and are considered to influence them. While science is based upon the concepts of independent and dependent variables, in research articles these terms often are not used explicitly. Rather, readers have to infer which is which based upon the question of what variable(s) is being assumed to influence or impact what other variable(s). However, this portrayal of independent and dependent variables is the simplest form for research questions. Frequently the equation is made more complex with the use of intervening variables, such as moderating variables and control variables. A moderating variable operates as its name implies, impacting the relationship between the independent and dependent variables from “in the middle.” A control variable is used in the statistical analyses to isolate the impact, or influence, of the variables in the study. In essence, a control variable is used to test out the following proposition – let’s remove the influence of the control variable from the picture and see what is left in the relationship between the independent and dependent variables. We explain more in the next chapter on understanding these analyses. Considering the Denham et al. (2015) study on executive function, we can see in the research questions that control variables were used. One aim of the study was to “describe associations of PSRA [the measure of executive function] components with teachers’ reports on children’s social competence and classroom adjustment, after holding child characteristics constant” (p. 215). From this we can determine that the control variables here are child characteristics, in this case gender, age, and maternal education.

93

STRUCTURES OF RESEARCH

Moving to Sheridan, et al.’s (2014) family engagement study, we find the following research questions: “What are the effects of the Getting Ready intervention on preschool children’s learning-related behaviors (e.g., agency, persistence) …?” (p. 750). Here, the independent variable is the intervention received by those in the treatment group, which is being compared to the control group – families who did not receive the intervention. The dependent variables are children’s learning-related behaviors. The second research question was, “Does parental depression moderate the effects of Getting Ready on children’s learn­ ing-related behaviors?” (p. 750). Parental depression is the moderating variable. The researchers could control who got the intervention and who did not; they could not control who was experiencing depression. On believing it might be important, they measured the level of depression and included it in the analyses. We must note that Sheridan et al. also utilized two control variables: the child’s gender and the presence of an identified disability for the child. These control variables are not included in the research questions, but instead are mentioned in the methods section under a header “study variables and instrumentation” (p. 756). This exemplifies how important it is to read a research report carefully and thor­ oughly as sometimes information is located in various places. Comprehending how variables are defined and related to each other, as we have just described, is the first step in understanding the study methods. From there readers must examine how researchers chose to measure the variables. They may make use of instruments that have been devised by others or develop new instruments for a study (this should be accompanied by more information about that process). In Denham et al.’s (2015) study on executive function the researchers used several estab­ lished instruments. Hot and cool executive functioning were each measured with an instrument directly administered with children. The researchers described what children were asked to do and how scores were established. For example, for hot executive function the children were asked not to peek while an assessor spent a minute wrapping a toy. The variable was measured as the number of seconds (up to 60) before the child took a first peek. We assume that children who did not peek before the time limit ended were scored as 60. Social competence and class­ room adjustment were measured by surveys completed by the children’s teachers. The researchers described how many items each survey contained as well as the subscales contained within each by naming the subscale and offering an example item. When researchers describe the instruments used, they should provide enough information to help readers understand the nature of the instrument. In addition, harkening back to our discussion in Chapter 5 regarding validity and reliability, readers should expect researchers should also offer information about the validity and reliability of the instruments. In the experimental study of family engagement by Sheridan et al. (2014), recall that the independent variable was the Getting Ready intervention program. In the Methods section the researchers provided a description of the strategies used within components of that program. In addition, they reported they measured intervention fidelity (how closely tea­ chers adhered to established guides when delivering the program). However, instead of going into detail, they referred the reader to other published papers in which they more thoroughly described how fidelity was assessed. Sheridan et al.’s (2014) dependent variables were scores on ratings of agency, persistence, activity level, positive affect, distractibility, and verbalizations shown by children in interac­ tion with their parents. The researchers provided the definitions established for each of these variables. These variables were rated from videotapes, and the researchers described the process of establishing inter-rater reliability. To measure depression, the moderating variable, the researchers had participants complete an established survey about how frequently they

94

QUANTITATIVE RESEARCH

had experienced depressive symptoms. The researchers explained that this instrument results in a range of scores (in the case of this instrument, participants rate 20 items on a scale of 0 to 3, resulting in a possible range of scores from 0 to 60). They could have chosen to use the score from each participant in their analyses (e.g. 10, 14, 8, etc.). Instead, Sheridan et al. described how they relied on other studies to help them set a “cut-off” score which served to divide the participants into two groups: those scoring above the cut-off and thus showing depressive symptoms and those who scored below and did not show depression. In this case, the researchers used an established instrument but derived their own way of using it to measure their variable of interest, dividing the participants into two groups. In addition to the use of instruments, researchers take smaller bits of information to create measures used within their studies. For example, demographic information (age, gender) is frequently used as well as background information such as family members’ or teachers’ level of education and teachers’ years of related experience. Researchers sometimes simply name the variables used and sometimes provide information on how variables were created. Returning to Denham et al.’s (2015) executive function study, the researchers used measures of child gender and age. The researchers also surveyed parents about their educa­ tional level and created an education variable that had two levels: mothers who had a high school diploma or less (low education) and mothers who had an associate degree or higher (high education). We might question, incidentally, how a mother with some college cour­ sework, but not an associate degree, would be categorized on this dichotomous (two-level) variable as it was described. Often educational level is used to denote just that – how much education a participant has completed. Denham et al. (2015) stated that they were utilizing maternal education as a “proxy for socioeconomic risk status” (p. 216). Thus, they moved back and forth in their paper, refer­ ring to “children of mothers with less formal education” and “children at risk for living in pov­ erty” (p. 220) in reference to the same variable. Proxy variables are used when something more easily measurable (such as education level here) is used to signify a more complex variable, here the chances of living in poverty. The researchers explained that some of their participating children were enrolled in Head Start programs, which do serve children living in poverty. Yet, some children were enrolled in private childcare centers, and this tells us little about families’ potential socioeconomic situations. Also, it would be reasonable to consider if in two-parent families (we don’t know family or household members in this study) the mother’s educational level is a reliable indicator of the likelihood of the family living in poverty. The larger question is: what can the mother’s educational level signify and stand for? We chose the van der Aalsvoort et al. (2015) play study as one of our exemplars because their measures are somewhat unique among quantitative researchers. In this case students studying to be early childhood teachers were asked to evaluate and explain if clips of chil­ dren’s activities reflected play. This contrasts with the typical closed-ended options utilized in quantitative research. After data collection the researchers used an instrument to assign the participants’ comments to categories, reporting on the reliability of their efforts. Con­ sider how this is different from the data analyses processes used by qualitative researchers. In the end the data were subjected to a standardized process for “reducing” them to numerical form, i.e. the number of responses in each category. In other words, the details of responses were left behind as each response was recorded as representing one of the four possible categories. Before leaving this topic we note the researchers wrote they found the confines of written responses frustrating at times, being unable to code a response based upon what was written. They realized that had they used individual interviews, they could have probed more during participants’ responses to “fill” them out. However, they also

95

STRUCTURES OF RESEARCH

acknowledged their method was time-efficient in gathering responses from a large number of individuals. It should now be clear that deciding upon instruments and measures for a study is a com­ plex and important process. Researchers aim to use the best available instruments but also must compromise in those decisions because of the resources and constraints of the study. In addition, decisions must be made about how data from the instruments will be transformed into scored variables. It is up to the reader to evaluate the logic of the researchers’ decisions. Does an instrument seem an appropriate measure of a variable? Are the instruments ageappropriate for children? Are reliability and validity adequately assessed? Are the instruments meaningful and equitable for all participants? For example, in iterations of a major study of the Head Start program (Family and Child Experiences Survey – FACES), family members were asked how often they took their children to a zoo or “art gallery, museum or historical site.” Clearly, opportunities to do so would vary dramatically across geographical locations and impact the findings. To highlight the issues of using instruments in different areas, in Chapter 12 readers will find an essay by Valentina Pagani, an Italian researcher who examined the meaning of an instrument developed in the United States for her own cultural context.

Collecting Data Once measures and instruments have been defined, researchers then need to make decisions about data collection. Once again, the decisions should harken back to the logic underlying the research questions. In the Denham et al. (2015) executive function study, the researchers tested how chil­ dren’s executive control predicted (varied with) their teachers’ ratings of social competence and classroom adjustment. The research team assessed children’s executive control first, in the middle part of the school year. They waited until the end of the year to collect the tea­ cher reports. By that time, if indeed the independent variables influenced the dependent variables, there would have been time for this to occur. Thus, decisions about collecting data are made in light of expected relationships of the variables (e.g., temporal order) as well as in consideration of when participants can best offer the information needed for the study. For example, if a researcher wants children’s scores on a pre-test and post-test measure to investigate the impact of a particular intervention, it would be important to time the assess­ ments as close to the beginning and ending of the intervention as possible. Too late after the intervention begins, or too soon before its ending point, and the scores might not represent the full impact of the intervention (i.e. from before it began to when it ended). Too late after the intervention ended, and other things may have intervened. As might be expected, procedures here are also as standardized, or uniform, possible. If, for example, a researcher was observing children’s peer relationships in preschool classrooms, similarity of the timing of the observations matters. Collecting data in some classrooms in the spring and other classrooms in the fall may affect the data in a biased manner, as children have had more time by spring to form relationships with their peers. Also, collecting data in some classrooms during play opportunities and in another during planned large group activities can impact what can be observed of children’s relationships with their peers. While qualitative researchers typically spend plentiful time with those participating in the study, this is rarely true of quantitative researchers. A researcher can often leave a classroom after a single day’s observation with ample data, depending upon the design. The question here is: Does the data collection plan allow for a fair representation of a phenomenon to develop? In complex educational settings, this is a knotty question.

96

QUANTITATIVE RESEARCH

Conclusion We began this chapter by noting that much is done upfront in a quantitative study. These decisions begin with forming researchable questions. They continue to enrolling a sample of participants, deciding upon measures and instruments, and planning for data collection. In a well-done study, there will be consistency and connectedness at each step of the process. When researchers veer from the direction of their logic (e.g., an instrument that does not quite reflect the construct of interest, a data collection process with inopportune timing decisions), the limits and challenges associated with the study increase, while the potentials decrease. Each study is a balancing act of trying to maintain the logic as closely as possible and minimize the limits. It is up to readers to evaluate the study methods. Processes are messy and sometimes unpredictable in educational research, even in the best-planned stu­ dies. Teachers leave positions unexpectedly, children leave programs, surveys come back only partially completed. Thus, any critique of research must also keep in mind the real-world issues that intervene, no matter how meticulous the planning. In the next chapter we move into the processes involved after data are collected.

Reading and Understanding a Quantitative Study Purpose/Research Question Task:Identify the research question Look for: The question may be introduced early in an article but it is stated most clearly at the end of the literature review. Examine the question(s) to determine who this study involves, what is of interest, and how the researcher expects the phenom­ ena of interest to operate (e.g., differ from each other, impact each other). What assumptions do you think this and other researchers in this area have about how the world works? Task:Map out how the study is framed within the literature base Look for: To begin to build your understanding, look for the headers used to identify topics. After reading the literature review, re-read the introductory and concluding sentences of sections or paragraphs carefully to clarify your understanding of main points. Determine how the researcher set the stage for this study by reviewing what has been found in previous research and addressing how this study addresses gaps and/or ongoing questions. Do you think this literature base has left perspectives unexplored? Methods: Sample Task:Record who is represented in this study Look for: Note the descriptions given of the participants in the research. How much does the researcher tell you about their background? What do you wish you knew that isn’t reported and may be important? How appropriate does this group of participants seem (e.g., number involved, their local contexts), given the research question(s)? How did the researcher find the participants, and if pertinent, group them? Note that if there are few details about identifying participants, the sample may have been a matter of convenience rather than deliberate selection. If the

97

STRUCTURES OF RESEARCH

researcher intends for this sample to represent a larger group, who would be represented in the larger group? Who has been disqualified from participation in the study or is unrepresented, and what meaning do you make of that? Methods: Variables and Measures Task:Identify what is being measured and how it is being measured Look for: Note the concepts that are involved in the study and their role in the research questions. It might be helpful to consider what is upfront in the research question and what is resulting in the question. How clearly and concisely did the researcher define the variables? Read the descriptions of each instrument used in the study. Are reliability and/or validity addressed? How closely tied, or representative, of the concept is each instrument? Look for potentials (the instrument is closely tied to the concept), challenges (there is some disconnect between what the researcher wants to measure and this particular instrument, or the data being used by the researcher from an existing database), and limitations (some connections of the logic – this relating to that, or this causing these groups to differ – are not well represented by the instruments and data). Do the instruments allow a clear and equitable understanding of each potential participant to be reflected? Methods: Data Collection Task:Identify the procedures used to collect data Look for: Note when the data were collected, how, and by whom. Does the timing of when the data were collected make sense for the research questions (e.g., if teachers are making reports about children, have they had enough time to get to know the children). Is there a sufficient amount of data to assure that the phenomena of interest are represented (e.g., how often and when data were collected via classroom observation). Were the procedures standardized to ensure that data collection remained the same throughout the study and for all participants? Consider what perspective is gained when instruments utilize reporting (e.g., who is reporting, what they might fairly know about the person and/or events being queried, what factors might impact their perceptions).

98

9

UNDERSTANDING ANALYSES IN

QUANTITATIVE RESEARCH

It is likely that some readers had a sharp intake of breath as the title of this chapter appeared. A lack of comfort with statistical analyses is a challenge to feeling confident as a consumer of research. We acknowledge that many or most readers have not studied statistics. Our goals for this chapter are therefore to acquaint readers with major types of analyses and to scaffold an understanding of how to locate and comprehend pertinent information about the results of analyses. A deeper understanding, acquired through further study, allows for more complex evaluation (e.g., What are the assumptions and limits of this analysis? Why does an analysis work as it does?). For now, our aim is to help readers identify what is communicated in the findings, relate this back to the research question(s), and evaluate the claims being made. We continue to make use of the studies in the previous chapter and follow the same organization, explaining first descriptive data presentations, then correlational analyses, and finally, comparative analyses. There is a wide array of possibilities for statistical analysis, and we focus on only some predominant types.

Descriptive Data Presentations Presentations of descriptive data frequently start off the explanation of research findings. Hence, we begin with an examination of the nature of data, as this determines how data are presented and analyzed. Some forms of data are categorical, meaning that there are discrete groups for the vari­ able. For example, gender has been invariably coded with limited choices by researchers. The number assigned in coding is meaningless; it doesn’t add understanding. In other words, males could be coded as 0 or 1, researcher’s choice. When data are categorical, researchers report the frequencies of each category or type within the variable. Sometimes a sort of shorthand is used in researchers’ presentation of frequencies. The researcher may report in text that 83% of teachers were White, leaving the reader to consult a table to ascertain how the remainder of the sample reflected other racial/ethnic categories. Other data are continuous in nature. For example, age progresses along a continuum of possible data points, as do possible scores on many standardized assessments used with children. Similarly, but in a more restricted range, participants’ responses on a one- to fivepoint Likert scale result in a set of data points located along a continuum. One of the most important descriptors for continuous data is the mean, the arithmetic average. The symbol used for a mean is X̅ , or the abbreviation M. For a continuous variable, the range is also often presented. This represents the lowest and highest values recorded for this sample. It is also helpful to know the possible lowest and highest values.

DOI: 10.4324/9781003354499-10

99

STRUCTURES OF RESEARCH

Another important descriptor of continuous data is the standard deviation. This number, abbreviated as SD, reflects how “spread out” the data are. Given a sample of sufficient size, there is an assumption that data tend to spread around the mean in a predictable manner, resembling a normal bell curve, with more responses clustered around the middle of the curve and fewer in the tail ends. The standard deviation is calculated such that approximately 34% of the data points will fall within one standard deviation above the mean and another 34% will fall within one standard deviation below the mean. Approximately 13.5% of the data points fall between one and two standard deviations above the mean and, conversely, another 13.5% fall between one and two standard deviations below the mean. Thus, about 95% of data points are within two standard deviations in either direction from the mean. See Figure 9.1 for a representation of the normal bell curve. Means and standard deviations allow us to make sense of what these variables tell us about the participants. For instance, if the mean on a one- to five-point Likert scale is reported as 1.34 or 2.52 or 4.43, we can visualize that responses tended to gather in the lower end, middle, or upper end of the response options, respectively. If this was a job satisfaction survey for teachers, and the mean was 4.43 on a response scale where “5” corresponded to “strongly agree,” we could construe that generally the participants had very positive feeling about their jobs. Note that to make this interpretation, we must know the mean and range of a variable, as well as how it was measured via the instrument. In making sense of the standard deviation, look for how it compares to the mean and range. The smaller the standard deviation figure, the more closely packed in the bell curve is; the larger the standard deviation, the more spread out the curve becomes. When the data are more closely packed in, smaller differences in means and standard deviations can be found to be significantly 99.7% of the data are within 3 standard deviations of the mean 95% within 2 standard deviations 68% within 1 standard deviation

μ −3σ

μ −2σ

μ −σ

μ

μ +σ

μ +2σ

μ +3σ

Figure 9.1 Bell Curve and Standard Deviation (note: in the figure, µ stands for mean and σ stands for standard deviation). Source: Kernler, Dan. Empirical Rule. Digital image. Wikimedia Commons, 30 Oct. 2014. Web. 13 Jan. 2016. https://commons.wikimedia.org/wiki/File:Empirical_Rule.PNG. This file is licensed under the Creative Commons Attribution-Share Alike 4.0 International license.

100

UNDERSTANDING ANALYSES IN QUANTITATIVE RESEARCH

different from each other, even though on the overall scale the difference may not be large. Suppose a 10-point scale measures teachers’ job satisfaction, with higher scores reflecting greater satisfaction. In two groups of teachers being compared, the mean (M) and standard deviation (SD) for Group A are 8.9 and .6, respectively, while for Group B they are 7.8 and .5 respectively. Going further, let’s suppose that statistical analysis reveals that these two means are significantly different (more on what that means later). The researcher then asserts that teachers in Group B are less satisfied with their jobs than teachers in Group A. On the face of it, this is true; the mean for Group B is lower than Group A’s mean. Remember, though, that approxi­ mately 95% of scores are located within two standard deviations above and below the mean. For Group B (M = 7.8, SD = .5), that range would be roughly 6.8 to 8.8 (the M with 2 SDs sub­ tracted and 2 SDs added). While scoring lower as a group compared to Group A, the vast majority of teachers in Group B are still scoring above the midpoint of the scale and would appear to be satisfied, to some extent, with their jobs. Knowing this helps the reader to see that even if the groups are different, it would not be accurate to portray the second group as unsa­ tisfied. Furthermore, Group A teachers seem to be incredibly satisfied. Looking at the data in this way allows readers to add meaning to numerical presentations. By examining the descriptives, consumers of research can begin to understand the nature of the research variables. To answer most research questions, however, the researcher per­ forms statistical analyses on the data. The analysis begins with an assumption the data do not differ between or among the comparisons being made, with the results of the analysis showing if that seems likely or unlikely to be the case. Within the review of literature, the researcher makes the case of where differences might be expected, and this can be stated as a hypothesis or question. Bear this brief description in mind as we dig further into statistical analyses in the following sections.

Correlational Analyses Correlational analyses indicate how strongly related variables are to one another or the degree to which some variable(s) predict another variable. Sometimes researchers are interested in questions focused on relatedness as their research goal; other times researchers might make adjustments in the planned analyses to account for correlations found in the data. Relationships: Bivariate Correlations How two variables relate to each other is measured by a bivariate correlation, or the Pearson correlation, sometimes referred to as Pearson’s r or just presented as r. The scores on the two variables are compared for each participant, and the Pearson correlation reflects the relationship calculated across the overall sample. Values of r range from –1.00 to +1.00. In a positive correlation, the higher the value of one variable, the higher the other variable tends to be – they vary in the same direction. In a negative correlation, the larger one variable is, the smaller the other variable tends to be – they vary in opposite directions. For example, children’s height and weight would be positively related to some degree. However, chil­ dren’s age and amount of time spent crying in a day would (hopefully) be negatively related to some degree. Values closer to zero reflect the lack of a systematic relationship between the two variables. Because it bears repeating, we again make the point that these relation­ ships cannot be assumed to indicate that one variable causes another. To further explore bivariate correlations, please refer to Table 9.1. This is a table of cor­ relations from Tominey and McClelland (2011) who researched self-regulation interventions

101



102

0.01

0.06

0.1 −0.12

0.18 −.24†

0.05

0.17

−0.1

0.04 −0.13

−0.08

0.05

0.14

0.07

0.12

−0.07

0.14

−0.12

0.14

__

−0.13

0.03

0.01

−0.03

0.11

__

−0.05

.28*

0.01

0.01

.28* −0.04

0.06

__

0.07

.94***

__

__

−0.08

__

13









Note: HTKS = Head Toes Knees Shoulders task. a Child gender: 0 = female, 1 = male. b Head Start status: 0 = not enrolled in Head Start, 1 = enrolled in Head Start. c For correlations including maternal education, n = 55. d Intervention group: 0 = control, 1 = treatment. † p < .1. *p < .05. **p < .01. ***p < .001. Source: Tominey, S. L., & McClelland, M. M. (2011). Red light, purple light: Findings from a randomized trial using circle time games to improve behavioral self-regulation in preschool. Early Education and Development, 22, 489–519.

−0.07

−0.09

0.05

0.08 0.03

−0.15 −.00

.32**

−0.08

12



13 School ansences

−0.11

0.02

.29*

0.19

11



12 Number of sessions

11 Intervention group

−0.06

.45***

0.04

−.28*

−0.02

10



d

−0.12

−0.3

.52*** __

9



10 Vocabulary difference

−0.03

−0.20

−.46***

−0.12

−0.16

__

8



9 Letter-Word Identification difference

0.16

.50***

__

0.22

−.52***

__ .33*

7



8 Applied Problems difference

−0.04

−0.02

−.37**

−.065***

__

6



7 HTKS difference

−0.05

−0.17

−.31*

0.18

5



6 Spring HTKS

−0.01

−0.09

0.09

__

4



5 Fall HTKS

c

.31*

3



4 Maternal education

b

__

2



3 Head Start status

a

1

2 Child gender

1 Child age (months)

Variable

Table 9.1 Bivariate Correlations for Children in the Overall Sample (N = 65)

STRUCTURES OF RESEARCH









UNDERSTANDING ANALYSES IN QUANTITATIVE RESEARCH

with young children. The table presents the correlations between all of the study variables, which include the child’s age, gender, participation or not in Head Start, and mother’s education level. In addition there are variables representing measures of self-regulation and math and language/literacy skills. Finally, there are variables to represent whether the chil­ dren were in the intervention or control groups, the number of intervention sessions they experienced, and the frequency of their absences from school. Each of these variables is listed in the left-hand column labeled “Variable.” Readers can note that some of these variables are categorical (e.g., whether in intervention or control group) and some are continuous (e.g., number of intervention sessions). To read the table, look at the column labeled ‘1.’ This represents the variable numbered ‘1’ from the far left-hand column, child age. There is no number in the space that repre­ sents the intersection of row ‘1’ and column ‘1’ because the variable is the same for column and row. Continuing down the column, the next number (.31) is the correlation between ‘1’ (the column – child age) and the row within which it lies, child gender (2). Child gender, a categorical variable, is marked by the superscript, a, which corresponds to a footnote stating that for gender, males received a score of 1 and females a score of 0. To make sense of the relationship, think of it as “boys are high, girls are low.” There is a positive relationship between age and gender, meaning that the older the child, the more likely the child is a boy. The value for this relationship (r = .31) is closer to zero than to 1.00; this signifies the relationship was not particularly strong. Continuing down this column, it is evident that the remaining variables are correlated negatively with child age (except for Head Start status), but the strength of each correlation is quite weak, as most of the r values are less than –.10. Now move to the column labeled ‘3.’ The first correlation is between maternal education (4 in the first column, denoting our row) and Head Start status (3, the column we are reading), another categorical variable. The footnote for the Head Start variable, marked by the superscript b, tells us that enrolled in Head Start is higher, 1; not enrolled in Head Start is lower, 0. Here the relationship between the variables is stronger than other correlations on the table (r = –.65), and it indicates that children in Head Start (high) were more likely to have mothers with lower levels of education (low). One final detail is important in understanding this table. There are a series of symbols in the last footnote, which present numerical equations around the value p (e.g., *p < .05), which represents statistical significance. As we noted above, an analysis starts with the assumption that there is no relationship between the variables. Once the analysis is per­ formed, there is a calculation of how likely it is that the result obtained would have occurred if indeed there is no relationship between the two variables. Obviously, in the case of a bivariate correlation, the further a value gets from zero, in either direction, the more likely we can expect there is indeed a relationship. But we can’t assume this from the value itself; whether it was a likely or unlikely value depends upon the nature of the particular set of data. Thus, the p value tells us how likely this value was for this particular set of data. In research, there are accepted conventions about what value of p is noteworthy. There are three p values marked with asterisks on the table (p < .05, p < .01, and p < .001). The asterisks are used in the table to denote which analyses reached these levels of statistical significance. The r value of .31 between child gender and age reached a significance level of .05 and the r value of –.65 between maternal education and Head Start status reached a significance level of .001. What these significance levels indicate is the chance of a correlation of this value occurring by chance if in fact there was no relationship between the two variables. We interpret the

103

STRUCTURES OF RESEARCH

finding between child gender and age this way: if there was indeed no relationship between child gender and age, the chances for the r value to .31 are less than .05 (p < .05), or less than five in a hundred (read the p value as you would a decimal). This finding has reached the level of ‘unlikelihood’ that there is general agreement to accept the result is likely due to something other than the operation of only random chance. But how should we make meaning of this? Applying what we know about gender and age in young children, we might speculate that the finding may well be due to the luck of the sample. Consider it like the situation where a classroom ends up with ten boys and four girls, although we know that boys and girls are about equally distributed in the population of young children. It is not very likely, but it does happen every so often. Thus, we shouldn’t assume that it generalizes to “older children are more likely to be boys” in the overall population. In a case such as this, researchers may make adjustments in subsequent analyses to “adjust” for the fact that boys tended to be over-represented among older children. Regarding the relationship between maternal education and Head Start status the inter­ pretation is: if there was no relationship between maternal education and enrollment in Head Start, the chances for the r value to be –.65 are less than .001 (p < .001), or one in a thousand. This affirms that mothers whose children were in Head Start were more likely to have less education. Since this program serves families based upon income eligibility, and we know there is a relationship between education and income, this finding would appear to reflect something that could be expected in the larger population. To summarize, for researchers to accept that something is indeed “happening,” the odds for the result to occur simply by chance have to be quite low. In addition, if there is a sig­ nificant finding, the reader must make sense of what it means, as we have just illustrated. Finally, if we accept the odds that the finding could have occurred “five times out of one hundred” by chance alone (unlikely, but possible), it is possible this is the rare occurrence where random chance is operating, rather than a valid systematic relationship among vari­ ables. This is referred to as a false positive. Hence, among researchers, no result is perceived as absolute proof. Furthermore, the more analyses that are conducted, the more likely it is that a false positive crops up; this relates to the odds. Recall that in Chapter 8 we explained that quantitative researchers aim for a parsimonious set of analyses – that is, more is not better; this is due in part to the nature of the statistical analyses, in which a false positive is a potential issue. Before leaving this topic, we want to note that Tominey and McClelland (2011) also identified results with a level of p < .1 in the table reproduced in Table 9.1. This corresponds to a finding that will occur by chance one time in ten. As we noted above, the standard convention accepted by researchers is the .05 level (5 in 100). However, some researchers choose to report the p < .1 level to indicate that the relationship in the data is approaching significance. The message tends to be a suggestion that something might be happening, as the finding is close to the accepted significance level. Others choose not to report this .1 value of p. Their thinking is guided by the idea that “it was close, but it didn’t make it.” It is up to the reader to decide what meaning to attach to results that are in the margins. Prediction: Regression Analyses In multiple regression analyses the researcher aims to determine how accurately a depen­ dent variable can be predicted based upon some set of independent variables. The logic is as follows. For a dependent variable there is a range of values found for the participants in the study. We will be using Denham et al.’s (2015) executive control study for our example, so

104

UNDERSTANDING ANALYSES IN QUANTITATIVE RESEARCH

let us consider one of the study variables, social competence as measured by teacher ratings. Children’s social competence scores will differ in a group based upon a number of factors. The totality of all these differences amongst scores is referred to as variance. We could say that social competence varies among individuals due to … (imagine the many possible influences). If we picture the social competence scores residing within the area enclosed by a circle, the aim of regression analyses is to “rope off” areas of that circle by determining some percentage of the total variance that is related to each of the independent variables. In other words, how much of the total variance of the dependent variable can be explained by knowing independent variables? For example, if 38% of the variance is explained by our independent variables, we have some ability to predict a dependent variable by knowing the independent variable values, though obviously much is operating that is still unknown. For quantitative researchers, the ability to predict suggests understanding something about the phenomenon of interest. Importantly, it is always only a portion of the total variance that is explained in these analyses. This knowledge, however, is viewed as valuable in that we potentially have some ability to impact the phenomenon of interest in the world outside of research. Regressions are correlational analyses, and we need to offer the reminder that correlation does not imply causation. For example, it could be the case that the relationships found in regression analyses are related to some other, unmeasured variable. Even if one variable is measured prior to the other (e.g., one measure in the fall and one in the spring), the variable occurring “first” in the study cannot be assumed to cause the one measured later in a correlational study. As explained, the search for relationships among multiple independent variables and a dependent variable is the goal of multiple regression analyses. They are performed in a couple different ways. In a standard multiple regression, all of the independent variables are analyzed as a set; added to an equation at once, in a single step. In a hierarchical multiple regression, independent variables are entered in sequential groups, resulting in analyses that are reported as consisting of two or more steps. If researchers are using a hierarchical mul­ tiple regression, they must determine their logic for how to sequence the steps to enter the independent variables. For example, in the first step researchers may enter variables that cannot change (e.g., a child’s age), while in the second step they may enter variables related to phenomena that can be controlled or impacted (e.g., children’s development of executive function, which can be supported via teaching). Finally, in a multiple regression analysis, researchers also have the option of inserting interactions as one of the predictors. An interaction is a unique relationship between two independent variables. Say, for example two independent variables are gender and age. The researchers may want to examine if gender has a specific relationship to social competence at certain ages; as a hypothetical, perhaps previous research has shown the differences are wider between boys and girls at a younger age than at an older age (or vice versa). By entering an interaction as a predictor, the research question becomes, does the relationship of gender and social competence in a prediction of a dependent variable depend upon the age of the child? This is a relationship that implies some conditions; as in how well gender and chil­ dren’s social competence predict the dependent variable is conditional upon the children’s age. When interactions are used in a regression analysis they are usually represented as “variable 1 � variable 2.” Denham et al. (2015) used hierarchical multiple regression analysis in their study of executive control. One of their research aims was to “describe associations of [executive control tasks] with teachers’ reports on children’s social competence and classroom adjust­ ment, after holding child characteristics constant” (p. 215). Denham and colleagues entered

105

STRUCTURES OF RESEARCH

child characteristics (gender, age, and maternal education) into the regression analyses first. They determined how much of the variance in social competence and classroom adjustment was predicted by these variables; effectively, they “roped off” this portion of the variance. Then they entered the scores on executive control tasks to see how much of the remaining variance of the dependent variables (social competence and classroom adjustment) could be predicted. Essentially, they examined how executive control tasks added predictive potential above and beyond what was already shown to be predicted by child characteristics. By “holding child characteristics constant,” as the researchers said they did (p. 215), they are in effect saying, “let’s determine that prediction first.” In Table 9.2 we have reproduced the table from Denham et al. (2015) in which the hierarchical multiple regression analyses are displayed. Let’s examine the results for social competence. The child characteristics are listed under “Block 1,” meaning that they were entered into the analyses first. Note the symbol β (beta) along the headings under social competence; this refers to beta weight. The beta weight value indicates how much social competence scores increase or decrease (depending upon if a negative sign is indicated) if the independent variable increases by one standard deviation. Thus, the numbers are deci­ mals. Again, asterisks are used to indicate the significance levels found in the analyses. Chil­ dren’s age positively predicted social competence at the level of p < .001. There was an increase of .217 (approximately 22%) in the measure of social competence for each increase of one standard deviation in children’s age. In other words, older children tended to be rated by their teachers as more socially competent, and this difference was very unlikely to occur by chance alone (less than one in a thousand). Denham et al. (2015) did not discuss the individual beta weights in their report, instead focusing only on the fourth column, marked by ΔR2. This symbol is read as “delta R squared.” The value of R2 indicates how much variance in the dependent variable is predicted by the group of variables entered in the block. The ‘delta’ refers to the change in R2 that is achieved by that block of variables. In this example, the value of ΔR2 is .162 for Block 1. The interpretation goes like this: approximately 16% of the variance in social competence was predicted by the child characteristics (transform the Table 9.2 Prediction of Social Competence and Classroom Adjustment, Given Age, Maternal Educa­ tion, and Gender Social Competence B

Classroom Adjustment SE B

β

ΔR

Block 1

2

B

SE B

β

ΔR2

.162*** Age Maternal education Gender

0.021

0.005

−0.095

0.033

0.42

0.82

.217*** −.157** .280***

Block 2

.122*** 0.012

0.004

−0.033

0.025

0.312

0.061

.173*** −0.076 .285***

.019* CEC HEC

0.005

0.36**

0.002 −0.019

0.002

0.001

.118

0.002

0.004

0.002

.149**

.156*

+

+ p < .10. * p < .05. ** p < .01. *** p < .001.

Source: Denham, S. A., Bassett, H. H., Sirotkin, Y. A., Brown, C., & Morris, C. S. (2015). “No-o-o-o peeking”:

Preschoolers’ executive control, social competence, and classroom adjustment. Journal of Research in Childhood

Education, 29, 212–225.

106

UNDERSTANDING ANALYSES IN QUANTITATIVE RESEARCH

figure to a percentage by moving the decimal point two places to the right). This means that knowing all of the variables in Block 1 allows a researcher to predict 16% of the variance in social competence scores. Clearly, that leaves quite a lot of the variance unexplained. Moving down­ ward in the table to Block 2, the ΔR2 value is .019. This means that an additional 2% (approximately) of the variance in social competence scores was predicted by executive function. This is obviously a small value, but the asterisk indicates it was significant (an example of how the value of a number itself is not necessarily an indicator of its likelihood of reaching sig­ nificance). When we examine the individual variables for executive function, we see that only one, hot executive control, was a significant predictor, with a beta weight of .156. The positive value indicates that children who scored better on this measure were rated as more socially competent by their teachers, with an increase in score of approximately 15% for each standard deviation increase in hot executive control scores. The analyses in this table are described this way by Denham et al. in the text of the paper: “after controlling for significant contributions of age, maternal education, and gender, HEC [hot executive control] contributed to a significant increment in variance explained for social competence” (p. 218). These findings point to an obvious question about quantitative research – the findings may be significant, but how does this matter? With regression analyses, the importance is intuited by the R2 value. In the Denham et al. (2015) study, both child characteristics and executive control were significant predictors of children’s social competence ratings. By using hierarchical regres­ sion, they showed that hot executive control related to social competence scores above and beyond the relationship of child characteristics to these scores (the additional ΔR2). However, executive control predicted only a small portion of the variance in social competence (approxi­ mately 2%) when child characteristics were entered first. If a publisher claimed that their curri­ cular materials that focused on nurturing executive control skills would make a difference in children’s social competence, we might use this research to support the potential merit of this idea, although we might expect modest results at best. Concerning other significant findings from the study, it might be important to equity concerns, as discussed by Iruka in Chapter 3, to speculate more about the finding that boys were rated lower than girls on social competence (a not uncommon finding). Is this an inevitable developmental phenomenon? Something resulting from how social competence is measured in research? Might it be influenced by how adults (e.g., in this case, the teachers) perceive the behaviors of boys and girls? Or might it indicate something about how preschool classrooms function to support or censure ways of being? As we have pre­ viously stated, research results inexorably raise further questions. It behooves consumers of research, therefore, to regularly consider if the results might be related to how research is con­ ducted (e.g., our questions and measures) or how the world functions in unexamined ways. From this set of four chapters on qualitative and quantitative methods, readers might discern that the former question (what haven’t I measured) is more commonly raised by quantitative researchers while the latter (what haven’t I examined) is more commonly raised by qualitative researchers. We contend the former question is more limiting (we tend to measure what we consider measurable), and the research world would benefit more from the latter question. We also refer readers back to Iruka’s discussion of the concepts of race and racism in Chapter 3, exploring what has not been examined in the simple assignment of a measure of race.

Comparative Analyses Recall from Chapter 8 that some research questions are comparative in nature. In this case researchers are interested in comparing two or more groups, examining one or more dependent variables.

107

STRUCTURES OF RESEARCH

Comparing Categorical Dependent Variables As we explained earlier, categorical data are summarized by noting the frequencies (the numbers or incidences) occurring within each group or level. In the play study conducted by van der Aalsvoort and colleagues (2015), the data were recorded for two variables: the country of the participants and the characteristics of play detected in their written responses after watching videos. For each of these variables, frequencies make sense (i.e., how many), but a mean or average cannot be calculated. To examine the question of whether teacher education students in the four countries differentially drew upon the various characteristics of play in their writing, the researchers used a chi square analysis. The symbol for chi square is χ2. In a chi square analysis, the distribution of the data (e.g., in this case, how many individuals from each country uti­ lized the various characteristics of play in their responses) is recorded in a matrix. As we described about statistical analyses earlier in this chapter, the assumption being tested is that there are no systematic relationships among the data. To test this, the chi square analysis compares the frequencies observed (recorded in the matrix) with the frequencies that would be expected if there were no systematic relationships among variables. The expected frequencies under this assumption are determined by using the row and column totals. If a chi square analysis is significant (denoted by the p value), this means the observed frequencies in the data are different from the expected values, at the level of chance indicated by the p value. For the van der Aalsvoort et al. study the frequencies for each cell in the matrix were recorded (e.g., number of Germans who referred to play as fun, number of Germans who referred to play as voluntary, and so on, to include all combinations of the two variables). The observed values are noted in the research report, although the calculated expected values are not recorded. Remember, if the chi square analysis does reach significance, the interpretation is that there are systematic differences among the cells at a level unlikely to occur by chance. In Table 9.3 we have reproduced a chi square table from van der Aalsvoort et al. (2015). The chi square analysis reached significance at p = .002. In the table, frequencies of responses are recorded (the authors noted in the text that participants could and did use more than one characteristic in their responses). Using the totals from each country in the bottom row to convert these numbers to percentages allows for a sharper picture of similarities and differences among countries. Examine the row showing responses that used child-applied rules as a characteristic of play. The highest percentage of instances, 62%, occurred among participants from Finland, reflecting 31 of the 50 Finnish respon­ ses. In the Netherlands, 49% of responses included references to rules, 52 of 106 Table 9.3 Number of Play Characteristics Mentioned by the Respondents per Country Characteristics of play

Netherlands

Wales

Germany

Finland

Fun

9

24

17

5

Voluntary

39

57

77

13

Rules

52

42

64

31

No External goal

6

6

17

1

Total number of remarks

106

129

175

50

Source: Van der Aalsvoort, G., Prakke, B., Howard, J., Konig, A., & Parkkinen, T. (2015). Trainee teachers’ per­ spectives on play characteristics and their role in children’s play: An international comparative study in the Neth­ erlands, Wales, Germany and Finland. European Early Childhood Education Research Journal, 23, 277–292.

108

UNDERSTANDING ANALYSES IN QUANTITATIVE RESEARCH

responses. In Germany and Wales these responses made up 37% (64 of 175 responses) and 33% (42 of 129 responses) of the total number of responses per country, respec­ tively. It appears that rules were a highly salient category for the participants, but, fur­ thermore, that use of this characteristic in explanations differed among the countries represented by the participants. The expected frequencies under this assumption are determined thusly: for a single cell the expected frequency equals the total frequency for the entire row, multiplied by the total fre­ quency by the entire column, divided then by the total frequency for the entire table. This establishes an expectable proportion for the cell of interest. For instance, we calculated the expected frequencies for child-applied rules for Finnish participants, and this was 20 responses, versus the 31 made by the participants. Similarly, the expected responses in this category from Welsh participants was 53, compared to 42 actual responses. Importantly, a significant p level for a chi square analysis only tells us that the responses were very likely not random. However, the overall analysis does not provide information about which observed frequencies contributed to that finding (we would assume some did and some did not when there are a number of obser­ vations involved). Hence, readers should not assume that our noted expected response fre­ quencies reflect differences that can assumed to be statistically significant. Comparing Continuous Dependent Variables

Gender

To compare two or more groups using continuous data, the analyses operate under the assumption that the group means are equal. If indeed the dependent variables being ana­ lyzed between or among groups are found to be sufficiently different that an accepted level of significance (p) is reached, the researchers are able to conclude that there is reason to believe that there are differences between or among the groups. In simplest form, two groups might exist based upon an independent variable, for example gender. The scores for these two groups can be compared on the dependent variable, say executive function. In this case, the researcher may use a t-test. This is a simple comparison of two means. However, the same comparison can be made (and is often made), using an analysis of variance, abbreviated as ANOVA. The score for an ANOVA is represented as F. The analysis for a simple ANOVA or t-test can be repre­ sented visually this way:

Girls

Mean score on measure

Boys

Mean score on measure

F = Comparison of boys and girls

Analysis of variance can also be used for more complex configurations, such as three groups formed from a single independent variable (for example, children at 3 years of age, 4 years, and 5 years) being compared on a dependent variable. Comparisons such as these are called a one-way analysis of variance. The possible comparison here is, first, whether the scores for the three groups are significantly different, and, then, where those differences are, for example between 3-year-olds and 4-year-olds, between 4-year-olds and 5-year-olds, and/or between 3-year-olds and 5-year-olds. In yet another variation, the participants might be categorized using two independent variables, such as age and gender, again being compared on a dependent variable. This is

109

Age

STRUCTURES OF RESEARCH

3-year-olds

Mean score on measure

4-year-olds

Mean score on measure

5-year-olds

Mean score on measure

F= Comparison age groups

referred to as a two-way analysis of variance. In this example, the comparisons can be made in several ways, between boys and girls and between children of different ages. These are referred to as main effects; this is because they represent analysis of the independent vari­ ables. In addition, a comparison can be made of what is referred to as an interaction: whe­ ther there are differential patterns within the groups. For example, perhaps for boys age matters in different ways than it does for girls. This is represented visually below. The group comparisons can become more complex with other ANOVA models. There may be two or more dependent variables in the comparison (such as two forms of executive control),

Boys

Girls

3-year­ olds

Mean score on measure

Mean score on measure

4-year­ olds

Mean score on measure

Mean score on measure

5-year­ olds

Mean score on measure

Mean score on measure

Comparison of all age groups

Age

Gender

Interac�on Comparison of all boys to all girls Effect: boys at different ages compared to girls at different ages

referred to as a multiple analysis of variance, or MANOVA. When the dependent variable has been measured at more than one point in time, for example before and after an intervention, the analysis used is a repeated measures analysis of variance. In an analysis where the researchers want to account for the variance associated with a variable, they use ANCOVA or MANCOVA, the analysis of covariance. Covariance refers to a variable being controlled in the analyses. For instance, say that gender has shown a significant correlation to an independent variable, yet the researchers are not interested in gender as an independent variable. When they control the var­ iance associated with gender, they remove it “from the top.” In effect, the analyses from that point on reflect a situation where gender is no longer involved – as if the differences associated

110

UNDERSTANDING ANALYSES IN QUANTITATIVE RESEARCH

with gender are no longer in the picture. While this may be difficult to wrap one’s mind around – in fact, gender never disappears – by controlling its impact, the researchers can assess what is happening among the independent and dependent variables without the effect of the variable that is being controlled. What is important to remember is this: two or more groups can be formed (sometimes reconfiguring the participants along two different independent variables) to compare on one or more dependent variables. To assist in understanding a study in the future, it may be helpful to draw a figure such as we have provided above to visualize the comparisons being made. The Denham et al. (2015) study of executive control utilized a MANOVA analysis. The independent variables included children’s gender (boy or girl), age (3-, 4-, or 5-years-old), and maternal education (high school or less, or associate degree or more). The two depen­ dent variables were hot executive control and cool executive control. We have reproduced their table of results in Table 9.4. Main Effects While they have not explicitly named it as such, Denham et al. (2015) reported the main effects of a MANOVA analysis. The main effects are the results for the analysis of the inde­ pendent variables, gender, age, and maternal education on two dependent variables, the measures of executive function. When there are three or more independent variable groups and/or more than one dependent variable, follow-up analyses must be conducted to deter­ mine more specifically which of the possible comparisons were significantly different. In this example, the table from the original article does not provide all of the information needed to understand the results; it is complemented by the text information. In Table 9.4, look first at each of the three comparisons labeled at the top of the table. Immediately under each heading is the value of F, along with asterisks indicating the level of significance. There was a significant result for the age analysis. In the text reporting the research, Denham et al. (2015) wrote, “Older children showed more EC [executive control]; follow-up one-way ANOVAs showed that age differences were significant for CEC and HEC” (p. 218).

Table 9.4 Age, Gender, and Maternal Education Comparisons for HEC and CEC Gender

Age

F(2,272) = 4.08*

F(4,544) = 22.70***

F(2,272) = 10.24***

Partial ɳ = .029

Partial ɳ = .143

Partial ɳ2 = .070

Boys

Girls

3-yr

4-yr

5-yr

High School or Less

Associate Degree or More

52.41

57.83

28.14

61.32

75.90

47.12

63.13

(2.57)

(2.64)

(3.73)

(2.49)

(3.23)

(2.69)

(2.51)

38.64

45.98

29.50

46.08

51.36

42.08

42.54

(1.83)

(1.88)

(2.66)

(1.77)

(2.30)

(1.92)

(1.79)

2

CEC HEC

Maternal Education 2

Notes: CEC = cool executive control; HEC = hot executive control. Fs evaluated by Pillai’s Trace. Standard errors

in parentheses.

*p ≤ .05. ** p ≤ .01. *** p ≤ .001.

Source: Denham, S. A., Bassett, H. H., Sirotkin, Y. A., Brown, C., & Morris, C. S. (2015). “No-o-o-o peeking”:

Preschoolers’ executive control, social competence, and classroom adjustment. Journal of Research in Childhood

Education, 29, 212–225.

111

STRUCTURES OF RESEARCH

What this means, is that first the researchers ran the largest, most complex analysis (everything in the middle part of Table 9.4). Finding a significant result allowed them to return to examine the dependent variables, CEC (cool executive control) and HEC (hot executive control), sepa­ rately. In both cases the results were significant. This means that among the three age groups, there were differences in both measures of executive function. But, given that there are three age groups, we still do not know if the differences were between every possible pairing of age groups or only some pairs. Denham et al. continued in the text, “Bonferroni multiple compar­ isons for age differences in CEC showed a linear progression, with groups scoring higher as age increased. Similar comparisons for age differences in HEC showed only differences between 3­ year-olds and both other age groups” (p. 218). To put the results another way, the follow-up analysis showed that for CEC, the 4-year-olds scored significantly higher than the 3-year-olds, and the 5-year-olds scored significantly higher than the 4-year-olds. Each year in age was marked by a significantly higher mean score. On the other hand, for HEC, there was a sig­ nificant difference between the 3-year-olds and the other two groups, the 4s and 5s. There was no significant difference in the scores between the 4s and 5s. Rereading the text description while following the values in Table 9.4 can be helpful. In this section of their report, Denham et al. (2015) state the results of other follow-up analyses, and we suggest readers consult the table and deliberate on these portions of the text: SES [socioeconomic status] differences favoring children less at risk [recall that the authors have used maternal education to measure socioeconomic status and assume low maternal education is a risk] were found only for CEC … and gender differ­ ences favoring girls were significant only for HEC. (p. 218) In summary, a MANOVA table will alert readers to which of the overall analyses reached significance. It is important to locate the follow-up analyses that pinpoint the specific dif­ ferences; careful reading and comparison of the tabled and textual information is essential. Interaction Effects To briefly illustrate the concept of an interaction, we move to another study. Anthony et al. (2014) conducted a study of the impact of several different models of literacy support for young children. They compared children’s scores on print knowledge at the beginning and end of the year and in between these assessments they implemented versions of the literacy support interventions in different classrooms. In Figure 9.2, we have reproduced a figure from the study illustrating an interaction. An interaction indicates that groups within an independent variable showed different results for a dependent variable. Here Anthony et al. showed how children with differing levels of pretest scores were impacted by the interventions. The figure presents the post-test scores (marked on the left, or y, axis) of children from three different groups, each of which had a different interven­ tion condition (indicated by the lines being dotted, dashed, or solid). On the bottom, or x, axis are three points indicating children grouped by their pretest scores, as either low, average, or high. Overall, children who started with lower pretest scores were lower at the end of the year, while children with higher pretest scores had higher end-of-year scores (indicated by all the lines sloping upward). The interactions are ‘read’ in the distance between the lines for three groups. Looking at the right side of the figure, the children who began with higher scores at pretest, the three groups, labeled Control, Family, and

112

Raw Score at End of Year

UNDERSTANDING ANALYSES IN QUANTITATIVE RESEARCH

21

20

19

18

17

16

15 14

13

12

Control Family RAR Low

Average

High

Pretest Score

Figure 9.2 Interaction Effect for Pretest Scores

Source: Anthony, J. L., Williams, J. M., Zhang, Z., Landry, S. H., & Dunkelberger, M. J. (2014).

Experimental evaluation of the value added by raising a reader and supplemental parent training in

shared reading. Early Education and Development, 25, 493–514.

RAR, are at close to the same point for end-of-year scores, with only a slight advantage for the Family group. Moving to the left side of the figure, the children who began with low pretest scores, there is a larger gap between the Family group and the Control and RAR groups for end-of-year scores. In this case, children who began with lower scores benefited more by being in the Family intervention group – their scores were higher than the other two groups by a greater margin. The interpretation of this interaction is that the Family intervention was particularly effective, compared to the other interventions, when children began with lower scores. Researchers often use figures such as the example to illustrate interactions because the visual so effectively shows the differential effects indicated by a significant interaction. In summary, comparative analyses are used to find significant differences between and among groups. Groups are determined by the independent variable(s). These comparisons can be made more complex by the addition of more variables, whether independent, dependent, or control variables. Comparisons can be made between groups and also within groups when there are more than two subgroupings. As mentioned, the question lurking beneath significant findings is how important those results may be in reality. With comparative analyses, researchers are more often expected to publish a statistic called effect size. For example, assume researchers ran a study of two different early literacy interventions and included a third comparison group conducting “business as usual.” Both Intervention A and Intervention B were found to have significant impact on the study’s dependent variables measuring literacy skills. The question then becomes, should we institute either of these interventions on a wide-scale basis? Do either of them have practical significance? Effect size can help in those determinations. Effect size is calculated based upon the standard deviation of the sample, most commonly using a statistic called Cohen’s d. The effect size indicates how much the mean of the groups of interest “moved” away from the mean of the comparison group. This movement is expressed as a decimal of the standard deviation. Standard conventions call for considering an effect size of .2 to be “small.” In this case the intervention group mean would have moved 20% of a standard deviation unit away from the comparison mean. Effect sizes of .5 are considered to be “moderate” and of .8 or more to be “large.” Thus, if Intervention A in our example had an effect size of .11, and Intervention B had an effect size of .62, we would

113

STRUCTURES OF RESEARCH

weigh those differences in practical significance in our recommendations for which inter­ vention to recommend knowing that Intervention B had a greater impact (considering other factors, such as cost, ease of use, cultural responsiveness, and so on).

Hierarchical Analyses The analyses described in the previous section have been in long use in educational research. More recently, hierarchical analyses are being used because they account for a unique feature in educational settings. When researchers conduct studies in early childhood settings, they come upon a situation in which children are already divided into groups based upon class­ rooms, or on another level, within schools. How a group of children and a teacher come to function together is a complex and unique experience, and most importantly it is shared within that group while within another group other experiences are shared. Hierarchical linear modeling (HLM) takes into account that children are “nested” within classrooms. As well, classrooms are nested in schools or programs. Below we explain a study in which hierarchical analysis was used. Poulou (2017) conducted a study in Greece to examine influences on preschool children’s emotional and behavioral difficulties in the classroom. Four specific types of emotional/ behavioral difficulties were reported by the teachers; they are the dependent variables. The independent variables examined for their contributions to dependent variables were the teachers’ own emotional intelligence scores and their perceptions of the children’s social skills and behaviors. Poulou was interested in how each of these independent variables influenced children’s levels of emotional and behavioral difficulties, as well as how they possibly interacted in this influence. In this study, not only were the children participating in the study nested within class­ rooms, but all of the information gathered about them was generated by the same indivi­ dual, their teacher. Thus, Poulou (2017) first examined how the data were influenced by this nesting. To put it simply, was there variance indicating that there were similarities among children within a classroom while also showing differences between the classrooms. Poulou reported the results of this analysis as thus: I computed the intraclass correlation (ICC). ICCs ranged from .09 to .38 for the behavioral outcomes. That is, between 9% and 38% of the variance in students’ behavioral outcomes was attributable to the preschool classroom the students were in. All ICCs were significant at the p