The Palgrave Handbook of Political Research Pedagogy (Political Pedagogies) 3030769542, 9783030769543

Citation preview

POLITICAL PEDAGOGIES

The Palgrave Handbook of Political Research Pedagogy Edited by Daniel J. Mallinson Julia Marin Hellwege Eric D. Loepp

Political Pedagogies

Series Editors Jamie Frueh, Bridgewater College, Bridgewater, VA, USA David J Hornsby, The Norman Paterson School of International Affairs, Carleton University, Ottawa, Canada

The purpose of the series is to create a new space for conversations between scholars of political pedagogy, and between such scholars and those looking for guidance on their teaching, and become the main recognizable authority/series/conversational space in this field. The proliferation of journals, conferences, and workshops devoted to teaching attest to the accelerating interest in the pedagogy of Political Science and International Relations over the past two decades. While research scholarship remains the dominant criterion for hiring and promotion at top tier institutions, almost all academics in these disciplines spend most of their energy teaching, and more than two-thirds do so at institutions where effective teaching is the primary factor in career success (Ishiyama et al 2010). Even those at research-intensive positions benefit from more effective classroom environments, and institutions across the world are building centers devoted to improving teaching and learning. The challenges of teaching span sub-disciplines and connect disparate scholars in a common conversation. Indeed, teaching may be the only focus that academics in these disciplines truly share. Currently, most writing about teaching politics is published in journals, and is therefore dispersed and restricted in length. This series will provide a much needed platform for longer, more engaged contributions on Political Pedagogies, as well as serve to bring teaching and research in conversation with each other.

More information about this series at http://www.palgrave.com/gp/series/16526

Daniel J. Mallinson · Julia Marin Hellwege · Eric D. Loepp Editors

The Palgrave Handbook of Political Research Pedagogy

Editors Daniel J. Mallinson Public Policy and Administration Penn State Harrisburg Middletown, PA, USA

Julia Marin Hellwege Political Science University of South Dakota Vermillion, SD, USA

Eric D. Loepp Political Science University of Wisconsin–Whitewater Whitewater, WI, USA

ISSN 2662-7809 ISSN 2662-7817 (electronic) Political Pedagogies ISBN 978-3-030-76954-3 ISBN 978-3-030-76955-0 (eBook) https://doi.org/10.1007/978-3-030-76955-0 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Cover image: © DrAfter123/DigitalVision Vectors/Getty Image This Palgrave Macmillan imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland

Acknowledgments

The editors would like to thank all the authors for their generous contributions to this project. We also thank Jamie Frueh for his support and encouragement, which helped to get this project off the ground. Finally, we thank Cohl Turnquist for his expert help in formatting and preparing this book for production.

v

Praise for The Palgrave Handbook of Political Research Pedagogy

“In this Handbook, instructors will find a diverse and inspiring resource for teaching any course that has a research component. The breadth of personal knowledge in the chapters provide insights into the pedagogical thought process from a variety of perspectives, sparking both ideas for innovative assignments and rethinking of course goals.” —Dr. Rebecca Glazier, Associate Professor of Political Science, University of Arkansas at Little Rock, USA “The Handbook provides an enjoyable and reflective read about the journeys of colleagues who have taught research and writing. You will be inspired by their wisdom and creativity. Undoubtedly, your own course design and how you approach learning will be impacted.” —Dr. Janet Box-Steffensmeier, American Political Science Association President (2020–21), Distinguished University Professor, The Ohio State University, USA “How do we thwart the authoritarians? One way is to teach our students to distinguish fact from fiction, quality research from social media rabbit

vii

viii

PRAISE FOR THE PALGRAVE HANDBOOK OF POLITICAL RESEARCH …

holes. Mallinson, Marin Hellwege, and Loepp, along with their wideranging team of scholars, have written a book that should be on the desk of every political science teacher.” —Dr. Mark Carl Rom, Associate Professor of Government and Public Policy, Georgetown University, USA, member of the Executive Board of the Political Science Education section (ASPA) and Editorial Board of the Journal of Political Science Education

Contents

1

Introduction to Teaching the Research Process Daniel J. Mallinson, Julia Marin Hellwege, and Eric D. Loepp

Part I 2

3

4

5

6

1

Teaching Information Literacy

Information Literacy Development of Undergraduate Political Science Student Researchers: The Instruction Librarian’s Role Emily Reed

29

Should Research Methods Teach Information Literacy or Statistics? Why not Both? Chelsea N. Kaufman

43

The Politics of Identity and Teaching Information Literacy in Political Science Jennifer Epley Sanders

55

Using K12 Foundations to Teach Scientific Literacy in College Research Methods Kristina M. W. Mitchell

67

Designing a Research Methods Course for a Skeptical Classroom Tavishi Bhasin

79

ix

x

7

CONTENTS

Journeys Beyond Information Literacy: Applying a Metaliteracy Framework to Political Science Sally Friedman and Trudi E. Jacobson

8

The Savvy Consumer of Political Science Research Jonathan Ring

9

Zen and the Art of Teaching Methods Without a Methods Course Verónica L. Reyna

Part II 10

Building Qualitative Methods Skills Through Research Design Jessica Hejny Teaching Research Design with Authenticity Christina Fattore

12

Research Design as Professional Development and Empowerment: Equipping Students to See, Analyze, and Intervene in Political Realities Kelly Bauer

14

115

129 141

151

Teaching Multidisciplinary Research Methods at a Small Liberal Arts College Jarrod T. Kelly

161

Less Can Be More: Encouraging Mastery of Research Design in Undergraduate Research Methods Jennie Sweet-Cushman

173

15

“Research Methods: Who Am I and Why Am I Here?” Robert Postic

16

The Inquiry’s the Thing: Teaching Quantitative Research Without Teaching Statistical Software Debra Leiter

17

103

Teaching Research Design

11

13

91

Teaching Research Design: The Gender and Politics Lab and Reflections on the Lab Model for the Social Sciences Amanda Bittner

183

193

205

CONTENTS

18

19

Researching & Teaching Political Science Through Arts-Based Inquiry Methods Michaelene Cox Embedding Feminist Pedagogy in Political Science Research Design with Reflections on Critical Theory and the Social Construction of Reality J. Cherie Strachan

xi

215

227 239

20

Black Lady Classroom Nadia E. Brown, Jasmine Jackson, Aayana Ingram, India Lenear, and Ariel D. Smith

21

How the Research Design Can Be a Structure, a Process, and a Product for Learning Political Science Erik Cleven

249

The Success of Research Methods at the Department Level Neil Chaturvedi and Mario Guerrero

261

22

Part III 23

24

Teaching Research Methods

Traveling Along with an Accidental Academic: Doing and Teaching Research John A. Garcia

275

Statistical Skills for the Workplace: A Practical Approach to Teaching Methods with Excel Lisa A. Bryant

289

25

Sneaking in Statistics Andre P. Audette

26

Pedagogical Recommendations for Applied Statistics Courses Jennifer Bachner

27

The Accidental Methodologist Christopher Zorn

28

From Step-Child to Innovative Leader: Political Science Research Methods Over the Decades Andreas Sobisch

301

311 323

335

xii

CONTENTS

29

Teach Me If You Can: Teaching Political Science Majors Statistics at a Hispanic-Serving Institution Dongkyu Kim

30

Excel, in More Ways Than One Whitney Ross Manzo

Part IV 31

32

33

34

35

36

37

38

345 355

Teaching Research Writing

Research Articles, Not Research Papers: Empowering Students Through Research Writing William O’Brochta

367

Integrating Research Writing and Research Methods: Toward a More Seamless Curriculum Martin S. Edwards

379

Empowering Students by Teaching Research-Paper Writing as a Foundational Methods Course Lisa A. Baglione

389

From “Good” to “Effective”: Teaching Writing Skills Explicitly in Political Science Colin M. Brown

401

Revising the Revising Process of Writing in Upper Level Political Science Research Methods Emily M. Farris

413

Systematic ELA Challenges at Post-secondary Institutions: Why Many Two-Year Students Aren’t Prepared for College-Level Writing Lauren Grimes Teaching Research Writing to Undergraduates in Political Science and Public Administration in the Online Environment Darrell Lovell Teaching Methods in the Context of a Writing Intensive Course Jessica A. J. Rich

425

435

447

Notes on Contributors

Dr. Andre P. Audette is an Assistant Professor of Political Science at Monmouth College (USA). His main areas of teaching and research are in American Politics, including political behavior, identity politics, and political inequality, especially among religious and racial and ethnic groups. He is also interested in constitutional law and educational psychology. Dr. Jennifer Bachner is the Director of the Data Analytics and Policy program at Johns Hopkins University. She is the author of America’s State Governments: A Critical Look at Disconnected Democracies (with Benjamin Ginsberg, Routledge), What Washington Gets Wrong (with Benjamin Ginsberg, Penguin Random House) and editor of Analytics, Policy, and Governance (with Kathryn Wagner Hill and Benjamin Ginsberg, Yale University Press). Her report, Predictive Policing: Preventing Crime with Data and Analytics, has been published by the IBM Center for the Business of Government. Bachner received her Ph.D. in Government from Harvard University. Dr. Lisa A. Baglione is a Professor of Political Science at Saint Joseph’s University in Philadelphia, PA. Her research interests include teaching research-paper writing, gender- and intersectional-sensitive pedagogies, gender and leadership, and authoritarianism in Russia. Dr. Baglione is the author of Writing a Research Paper in Political Science which is in

xiii

xiv

NOTES ON CONTRIBUTORS

its fourth edition with CQ/Sage, and she is currently finishing a theoretically informed introductory comparative text which examines democracy, authoritarianism, and inclusion around the world. The book gives special attention to gender, ethnicity, and intersectionality, while also encouraging active learning through data literacy. Dr. Kelly Bauer is an Assistant Professor of Political Science at Nebraska Wesleyan University. Her research explores how Latin American states govern rights and citizenship as global trends destabilize state sovereignty; recent work on state responses to international Indigenous rights, irregular migration, and human security regimes appears in Journal of Ethnic and Migration Studies, Latin American and Caribbean Ethnic Studies and Canadian Journal of Development. She also researches the best practices of teaching and learning, and has been recognized for her excellence in teaching. Dr. Tavishi Bhasin has a Ph.D. in Comparative Politics from Emory University. She is an Associate Professor in the School of Government and International Affairs at Kennesaw State University. She studies internal conflict processes at the intersection of comparative politics and international relations. She works on dissent and state repression/human rights violations in democratic and authoritarian contexts. She uses both quantitative and qualitative methods in her work and is a proponent of mixed methods. She has published inthe British Journal of Political Science, Electoral Studies, Journal of Conflict Resolution, Terrorism and Political Violence, and the Journal of Turkish Studies. Dr. Amanda Bittner is a Professor in the Department of Political Science at Memorial University, where she also founded and directs the Gender and Politics Lab. She studies public opinion and voting, and her broad research interests include the effects of knowledge and information on voter decision-making, as well as the institutional and structural incentives affecting voting behavior in both Canadian and Comparative contexts. She has published on a wide variety of topics, including voters’ evaluations of party leaders and the effects of those evaluations on vote choice, voters’ attitudes about women in politics, the role of parenthood in politics, and the measurement of gender in survey research. Dr. Colin M. Brown is an Assistant Teaching Professor in Political Science at Northeastern University, where he teaches a number of introductory courses as well as leading writing-based senior capstone classes.

NOTES ON CONTRIBUTORS

xv

His primary research interests are in migration, minority representation, political incorporation, and citizenship, particularly in Western Europe. In addition, he is part of a Boston-area research team investigating the effectiveness of common writing instruction techniques when used in political science and social science classrooms. He received his Ph.D. in Political Science in 2015 from Harvard University and his B.A. in 2007 from the University of Rochester. Dr. Nadia E. Brown (Ph.D., Rutgers University) is a Professor of Government and Director of the Women’s and Gender Studies Program at Georgetown University. She specializes in Black women’s politics and holds a graduate certificate in Women’s and Gender Studies. Brown’s Sisters in the Statehouse: Black women and Legislative Decision Making (Oxford University Press, 2014) has been awarded the National Conference of Black Political Scientists’ 2015 W.E.B. DuBois Distinguished Book Award, 2015 Anna Julia Cooper Award from the Association for the Study of Black Women and Politics, and the 2015 Center for Research on Diversity and Inclusion at Purdue University Faculty Research Award. Dr. Lisa A. Bryant is an Associate Professor of Political Science at California State University, Fresno. Her research focuses on election administration, political behavior, public opinion, gender and politics, and research methods, specifically survey and experimental research. She has published in American Politics Research, Political Behavior, Electoral Studies, and Publius: The Journal of Federalism. She currently serves on the editorial board for Political Analysis and on the national research advisory board for the Electronic Registration Information Center (ERIC). Dr. Neil Chaturvedi is an Associate Professor of Political Science at Cal Poly Pomona and a research fellow at the Brennan Center for Justice. Aside from research methods, his teaching and research focuses on American institutions, race and ethnicity, and elections. Dr. Erik Cleven is an Associate Professor in the Department of Politics at Saint Anselm College in Manchester, NH where he teaches international relations, comparative politics, and research methods. He has been actively involved in developing research opportunities for students and has co-authored research with undergraduate students. He has also mentored many students that have presented their research at undergraduate sessions at regional and national political science conferences. His own research interests are in political violence and dialogue and

xvi

NOTES ON CONTRIBUTORS

peacebuilding and have been published in journals like Ethnopolitics, International Interactions, and Conflict Resolution Quarterly. Dr. Michaelene Cox is an Associate Professor in the Department of Politics and Government at Illinois State University where she teaches courses in international relations and European Politics. She holds a Ph.D. (2002) from the University of Alabama. Her research interests are generally transdisciplinary in nature and include topics such as human security, digital government, corruption, and visual politics. Dr. Martin S. Edwards is a Professor in the School of Diplomacy and International Relations at Seton Hall University. His research on the International Monetary Fund has been supported by the National Science Foundation, and he has been a Fulbright Visiting Research Chair in Global Governance at the Balsillie School of International Affairs. He is currently a Research Fellow with the University of Southern California’s Center for Public Diplomacy. He currently serves as an Associate Editor for International Studies Quarterly, and his book on IMF and WTO economic surveillance was published in 2018. Dr. Jennifer Epley Sanders works at Texas A&M University-Corpus Christi in the USA as an Associate Professor of Political Science. She graduated from the University of Michigan with an M.A. and Ph.D. in Political Science as well as a Certificate in Southeast Asian Studies and a Certificate in Survey Methodology. Her B.A. in Political Science is from Vassar College. Her primary research focuses on the meaning and significance of identity in politics. She specifically analyzes the interrelationships between religion, gender, public opinion, and political behavior. She has a secondary research interest in effective teaching and student learning. Dr. Emily M. Farris is an Associate Professor of Political Science and Core Faculty of Comparative Race and Ethnic Studies at Texas Christian University in the USA. She studies and teaches on issues of urban politics and racial and ethnic politics. Dr. Christina Fattore is an Associate Professor of Political Science at West Virginia University. She is also a faculty advisor to WVU’s International Studies program. She was a 2020 finalist for the university-wide WVU Faculty Award for Distinction in Mentoring Undergraduates in Research and served as a WVU Honors College Faculty Fellow.

NOTES ON CONTRIBUTORS

xvii

Dr. Sally Friedman is an Associate Professor in the Department of Political Science at the University At Albany. Her book, Dilemmas of Representation (2007), provides an in-depth examination of the choices legislators make as they go about the business of representing their constituents. Using selected representatives from New York as case studies, the book also shows how the balance of local and national factors in the representational process has changed over time. Sally’s teaching has focused on American politics (intro classes, legislatures, women, and politics), methodology, and as the chapter in this volume describes, critical thinking. Dr. John A. Garcia is Emeritus Professor at School of Government and Public Policy (University of Arizona) and Research Professor Emeritus—ICPSR-Institute for Social Research—(University of Michigan). He received his Ph.D. in Political Science from Florida State University (1971). His publications focus on race (i.e., multidimensional conceptualization of race, validity of skin color measures, street race, examination of Latino His primary areas of research and teaching are minority group politics, especially Latinos; political behavior; political mobilization, urban politics; social survey research; and public policy. He has published eleven books and over seventy articles and book chapters. Lauren Grimes is a Political Science professor, public diplomacy analyst, author, educational consultant, and youth advocate located in Washington, D.C. Professor Grimes attained her Sociology B.A. at the University of Maryland and her M.P.P. at George Mason University. During her time in academia and as a consultant she has developed courses, curricula, scholarly articles and has hosted workshops and delivered conference presentations throughout the USA. She also founded “The Community Enrichment Project,” a youth civic engagement non-profit organization that focuses on empowering young leaders to create community-based solutions to socio-economic issues. Dr. Mario Guerrero is Department Chair and an Associate Professor of Political Science at Cal Poly Pomona. His teaching and research focus on political communication, American presidency, and political behavior. Dr. Jessica Hejny is an Assistant Professor of Political Science at Middle Tennessee State University (MTSU) in Murfreesboro, Tennessee. She received her Ph.D. in political science from the University of Oregon. Her research and teaching focus is US environmental policy. Her work

xviii

NOTES ON CONTRIBUTORS

has been published in the Journal of Environmental Studies and Sciences, Administrative Theory & Praxis, and the edited volume, Pragmatist and American Philosophical Perspectives on Resilience. She is currently working on a book project that examines the relationship between partisanship and environmental political development in the twentieth-century USA using historical methods. Dr. Aayana Ingram received her B.S. in Political Science at Alabama Agricultural and Mechanical University and is currently a second-year Political Science Ph.D. student at Purdue University. At Purdue, she serves as the Political Science Student Government Association senator. Her research interests include higher education policy, social media’s influence on the black college choice process, and the portrayal of HBCUs in media and how it affects state funding. Jasmine Jackson is a fourth-year Ph.D. Candidate in the Department of Political Science at Purdue University. She graduated summa cum laude with a Bachelor’s degree in Political Science from Jackson State University. Jasmine’s areas of focus are American politics and public policy. Her research interests include Black political behavior and identity politics. In her dissertation, she evaluates how the current conceptualization and measurement of political knowledge may exacerbate the racial gap between Blacks and Whites. Jasmine is the 2017 recipient of the George Washington Carver Fellowship and a 2017–2018 American Political Science Association Minority Fellow. Trudi E. Jacobson ([email protected]), Distinguished Librarian, is the Head of the Information Literacy Department at the University at Albany and also Extraordinary Professor, Research Unit SelfDirected Learning, Faculty of Education at the North-West University at Potchefstroom. She has been deeply involved with information literacy throughout her career and co-chaired the Association of College & Research Libraries Task Force that created the national Framework for Information Literacy for Higher Education (2015). She is the coauthor or co-editor of 14 books, including three about metaliteracy, and numerous scholarly articles. Her most recent work positions metaliteracy as scaffolding for open pedagogy. She and her students participate in the Wiki Education student editing program, in order to practice what she preaches.

NOTES ON CONTRIBUTORS

xix

Dr. Chelsea N. Kaufman is an Assistant Professor of Political Science at Wingate University, Wingate, NC, USA. Her research primarily focuses on rural public opinion and political behavior in the USA. Pedagogically, she is interested in how open access materials, technology, and high impact practices can create an inclusive and accessible classroom, as well as how she can best prepare students to be informed and civically engaged members of society. Her work appears in Journal of Rural Studies,Rural Society, and Journal of Political Science Education. Dr. Jarrod T. Kelly is an Assistant Professor of Political Science at North Carolina Wesleyan College, where he also serves as Dean of Accreditation and Institutional Research. He completed his Ph.D. at the University of Pittsburgh in 2017. His research interests include political psychology, public opinion, and electoral behavior. He has published in American Politics Research, Research and Politics, and Electoral Studies. At NCWC, he teaches courses on American Politics, including federal institutions and electoral politics, as well as research methods. Dr. Dongkyu Kim is an Assistant Professor of Political Science at the University of Texas Rio Grande Valley. His research interest lies in the intersection of domestic politics and international relations. Research topics include foreign direct investment, social welfare spending, income inequality, electoral participation, social protest, social capital, and interagency collaboration. His teaching interest includes International Political Economy, Comparative Political Economy, International Relations, and Conflict Studies. Dr. Debra Leiter is an Associate Professor with the Department of Political Science at the University of Missouri-Kansas City. She received her B.A. from the University of California, San Diego, and her Ph.D. from the University of California, Davis. An award-winning researcher and teacher, she studies comparative politics, primarily in Western Europe. Her research examines the intersection of elections, parties, and voting, and how context shapes political decision-making. Her teaching focuses on cross-national institutions and political representation, with a strong emphasis on undergraduate research. She also plays trumpet in Kansas City’s own Mid-America Freedom Band. India Lenear received her B.A. in Political Science at North Carolina Central University magna cum laude and is currently a doctoral student

xx

NOTES ON CONTRIBUTORS

at Rutgers University at New Brunswick. She studies Women and American politics, focusing on Black politics, identity politics, and Black feminism(s). Her research examines Black women’s political behavior and participation through Black feminist theoretical lenses. Dr. Eric D. Loepp is an Associate Professor of Political Science at the University of Wisconsin-Whitewater, where he teaches courses in American government, political behavior, and research methods. His disciplinary research focuses on candidate evaluations and electoral decisionmaking, particularly in primary elections. He also studies pedagogy, with an emphasis on data- and technology-enhanced teaching techniques, one of which won the American Political Science Association’s 2018 CQ Press Award for Teaching Innovation. This work has been featured in such journals as Electoral Studies, the Journal of Political Science Education, the Journal of Elections, Public Opinion, & Parties, Research & Politics, American Politics Research, and PS: Political Science & Politics. Dr. Darrell Lovell is currently an Assistant Professor of Political Science at West Texas A&M University. Pedagogy has been a part of his research agenda examining the use of simulations and methods in undergraduate education. His academic research focuses on public policy and administration in higher education and K-12. Currently, he is working on studies examining state takeover policies, the impact of the COVID19 pandemic on organizational communication and relationships and studying co-production at community colleges. Daniel J. Mallinson is an Assistant Professor of Public Policy and Administration at Penn State Harrisburg. His research examines patterns of policy diffusion and learning and the policy process, more broadly speaking. He has done deeper topical research on cannabis, environmental, and energy policies. His work has been published in Policy Studies Journal, Policy & Politics, Evidence & Policy, American Review of Public Administration, State Politics & Policy Quarterly, Publius: The Journal of Federalism, State and Local Government Review, Interest Groups & Advocacy, Information Polity, PLoS One, Research & Politics, the Journal of Political Science Education, and Teaching Public Administration, among others. He teaches courses in public policy process and analysis, research methods, and state government. Dr. Whitney Ross Manzo is an Associate Professor of Political Science at Meredith College in Raleigh, NC, USA. At Meredith she also serves as

NOTES ON CONTRIBUTORS

xxi

the campus pre-law adviser and Assistant Director of the Meredith Poll, an academic survey research organization. She studies women in politics, North Carolina public opinion, and political science pedagogy. Manzo earned her Ph.D. from the University of Texas at Dallas. Dr. Julia Marin Hellwege is an Assistant Professor of Political Science at the University of South Dakota. She is also affiliated with the Women, Gender, and Sexuality Studies program. Broadly speaking, her research examines the interaction between institutions and identity, particularly the representation of women and people of color. She has published in American Politics Research, Journal of Interpersonal Violence, Journal of Political Science Education, PS: Political Science and Politics, Policing: An International Journal, Politics, Groups, and Identities, and Social Science Quarterly. She teaches various courses in American political institutions, as well as Women & Politics, Politics of Inequality, and Writing & Research. In 2021, she won the Belbas-Larson Award for Excellence in Teaching, the highest award the University of South Dakota bestows upon its educators. Dr. Kristina M. W. Mitchell is a Lecturer at San Jose State University and the Regional Manager for West Coast Sales at Activate Learning, a K12 Science Curriculum publishing company. She conducts research on gender and diversity, pedagogical best practices, and political economy. She earned her Ph.D. from the University of Texas at Dallas in 2012. William O’Brochta is an Assistant Professor of Political Science at Louisiana Tech University. He studies ethnic representation and political violence in developing democracies as well as ways to make teaching comparative politics more relevant and impactful both for students and the community. His work has been published in British Journal of Political Science; Research & Politics; and Political Studies; among other outlets. [email protected], williamobrochta.net. Dr. Robert Postic is Professor of Political Science and Chair of the Department of Behavioral Sciences at The University of Findlay. His main research interests include religion and politics as well as the scholarship of teaching and learning. He received his M.A. in theology from Fuller Theological Seminary (1990) and his Ph.D. from Wayne State University (2007). Dr. Postic has an extensive teaching background, having taught for 25 years at the secondary and post-secondary levels. For over a dozen of those years, he has taught research methods.

xxii

NOTES ON CONTRIBUTORS

Emily Reed is a Reference and Instruction Librarian for the Penn State University Libraries at the Harrisburg campus. She is the subject specialist librarian for Penn State’s School of Public Affairs. Emily currently resides in south central Pennsylvania. She has ten years of experience working in libraries and specializes in providing library instruction. She enjoys working with faculty and students to provide valuable information resources and develop information literacy skills in students to produce independent researchers. Dr. Verónica L. Reyna is Associate Chair of the Department of Government at Houston Community College (Ph.D. Political Science, University of Michigan). She teaches American Government and Texas Government, as well as Mexican American/Latinx Politics. Besides teaching, Dr. Reyna’s research and community partnerships work toward community empowerment through youth political mobilization. Dr. Reyna started the Center for Civic Engagement in 2017 to empower Houston Community College students and surrounding communities to use their political power to create change on their own terms. Dr. Reyna has been nominated for teaching awards and is a 2019 NISOD Excellence Award recipient. Dr. Jessica A. J. Rich is Assistant Professor of Political Science at Marquette University. Her research and teaching focus on social movements, NGOs, social-welfare policies, HIV/AIDS, and state capacity. Regionally, she specializes in Latin America, with a focus on Brazil. Rich earned her Ph.D. and M.A. degrees from the University of California, Berkeley, and she has held postdoctoral fellowships at Tulane University, in the Center for Inter-American Policy and Research, and at the London School of Economics and Political Science. Her book, State-Sponsored Activism: Bureaucrats and Social Movements in Democratic Brazil, was published by Cambridge University Press in 2019. Dr. Jonathan Ring is the Director of Student Programs and global security fellow in the Howard Baker Jr. Center for Public Policy at the University of Tennessee, Knoxville. His research has appeared in the Journal of Peace Research, European Political Science, Asian Survey, and the Journal of Political Science Education. He regularly teaches research methods in the Department of Political Science and heads the public policy analytics minor program at the Baker Center.

NOTES ON CONTRIBUTORS

xxiii

Ariel D. Smith, M.Ed. is an American Studies Ph.D. Candidate and instructor of American Studies and African American Studies at Purdue University, where she also serves as Scholar-in-Residence for the Purdue Black Cultural Center. Ariel’s research interests are Entrepreneurship, Service Learning, Food Studies/Foodways, and Digital Media. Her research at Purdue contributes to scholarship on African American entrepreneurs engaged in street vending by exploring the community cultural wealth of African American food truck owners and their representation. Ariel is the founder of The Food Truck Scholar, a media and educational platform for the food truck industry, and the host of its podcast. Dr. Andreas Sobisch is an Associate Professor and Chair of the Political Science Department at John Carroll University, in Cleveland, Ohio. Besides research methods and introductory statistics for political science students, he teaches courses in comparative politics with an emphasis on European politics and health policy. His current research and recent publications focus on effective strategies for global and experiential learning. Dr. J. Cherie Strachan is Professor of Political Science at Central Michigan University. Her most recent work addresses concern for low levels of political engagement among specific demographic groups— particularly women and young people. These include her co-authored CQ-Sage textbook, Why Don’t Women Rule the World, and a forthcoming youth-focused political behavior textbook.Her research also focuses on partisan polarization, civility, civic engagement, and deliberative pedagogy. Strachan currently serves as the review editor for the Journal of Political Science Educationand is co-director of the Consortium for Inter-Campus SoTL Research (CISR), which facilitates multi-campus civic engagement and political science pedagogy research. Dr. Jennie Sweet-Cushman is an Associate Professor of Political Science at Chatham University in Pittsburgh, Pennsylvania where she teaches courses in public policy, American politics, women and politics, and research methods. She was the recipient of APSA’s 2020 CQ Press Award for Teaching Innovation. Jennie earned her Ph.D. in political science from Wayne State University in Detroit in 2014. Her recent work has appeared in PS: Political Science & Politics, Politics, Groups, and Identities, and

xxiv

NOTES ON CONTRIBUTORS

Politics and the Life Sciences. She currently serves as a commissioner on the City of Pittsburgh’s Equal Opportunity Review Commission. Dr. Christopher Zorn is the Liberal Arts Professor of Political Science, Affiliate Professor of Law, and a faculty affiliate of the Institute for Computational and Data Sciences at Pennsylvania State University. He holds a Ph.D. in political science from Ohio State University and a B.A. in political science and philosophy from Truman State University. Prior to coming to Penn State, he was Professor of Political Science at the University of South Carolina, a Visiting Scientist and Program Director for the Law and Social Science Program at the National Science Foundation, and Winship Distinguished Research Professor of Political Science at Emory University. His research focuses on courts and judicial politics, and on applied statistics for the social and behavioral sciences.

List of Figures

Fig. Fig. Fig. Fig. Fig. Fig. Fig.

1.1 1.2 1.3 1.4 1.5 8.1 8.2

Fig. Fig. Fig. Fig.

13.1 16.1 31.1 35.1

Distribution of contributors by institution type Highest degree offered at contributors’ institutions Number of methods courses available Number of methods courses required (undergraduate) Contributors’ program’s main priorities An inaccurate model of learning Hypothetical distributions of students before and after a successful research methods course Mean course grade by major Poster template Research article writing course structure Research process

17 17 18 18 19 105

112 166 202 370 418

xxv

List of Tables

Table 16.1 Table 21.1 Table Table Table Table

22.1 22.2 24.1 35.1

Sample online analysis tools Research Design as Structure, Process, and Product for Learning How the Methods Sequence Evolved Over Time at CPP Summary of New Methods Sequence Topics and Functions Covered in Excel Exercises Comparing Syllabi in terms of Lecture Topics and Assignments

198 254 268 269 298 420

xxvii

CHAPTER 1

Introduction to Teaching the Research Process Daniel J. Mallinson, Julia Marin Hellwege, and Eric D. Loepp

“We need you to teach research methods. Do you feel comfortable with that?” It is our experience hearing some version of this question at a job interview that motivated this book. For the many faculty who have been asked that question, they most likely said “yes,” even if they had never taught the research process before. As Ph.D. candidates in political science, we are trained in various facets of the research process, statistics, and writing. However, we are often not taught how to teach those skills to others, particularly to undergraduates and Masters-level students. As political science has evolved in the recent decades, many undergraduate

D. J. Mallinson (B) Public Policy and Administration, Penn State Harrisburg, Middletown, PA, USA e-mail: [email protected] J. Marin Hellwege Political Science, University of South Dakota, Vermillion, SD, USA e-mail: [email protected] E. D. Loepp Political Science, University of Wisconsin-Whitewater, Whitewater, WI, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_1

1

2

D. J. MALLINSON ET AL.

and Master’s degree programs now require their students to complete at least one course, if not more, in research methods. But the vast majority of faculty teaching these courses were likely never trained how to teach research, nor do many of them consider themselves “methodologists.” This Handbook compiles the experiences of over 40 instructors and librarians representing a broad range of institutions and career stages. There will be an instructor teaching at an institution like yours. Our contributors each reflect on their pedagogical journey in teaching an aspect of the research process: information literacy, research design, research methods, or research writing. For many it is a journey of trial and error; of being handed a syllabus from a retiring faculty member or someone that is tired of teaching methods and sent forth to teach. These stories document the growth experienced by the contributor. Beyond the formative process, each chapter provides practical advice and insights into teaching through the research process. Bear in mind that this is a Handbook, but not a Manual. It is not so much a How To for teaching research methods; rather, it contains hard-earned wisdom about designing and delivering content. Although individuals will find some chapters more relevant to their teaching context than others, there is a gem in here for every reader. There are personal experiences and pedagogical approaches that will resonate with methods instructors across the discipline and academy. We think it will resonate with you like this tweet from @poliscibitches resonated with many (including us): Teaching Methods. ONE STUDENT: fretting bc her measure of poverty rates is from 2017 and graduation rates is 2015. ANOTHER STUDENT: my control variable is California.

Since research methods classes are typically required in political science programs today, there tends to be a lot of variation in students’ preparedness for these classes. It seems to us that everyone has a valuable story to tell about teaching the research process. That is what this book is all about: sharing those stories, and learning from each others’ experience, both positive and negative. Before discussing the goals of the book, how we reach those goals, and introducing the contributors, we—the editors, Dan, Julia, and Eric—will take a few pages to tell our stories.

1

Dan:

INTRODUCTION TO TEACHING THE RESEARCH PROCESS

3

When I arrived at Penn State to work on my Ph.D., I was firmly convinced that I wanted nothing to do with research in the long term. I wanted to focus on becoming a good teacher of American government and state and local politics. I would do some research for my dissertation, but that would be enough. Now, Penn State has a quantitatively-oriented faculty and, thus, the program is similarly weighted towards quantitative methods. I did alright in my stats courses and even decided to do my first minor field in methods. My first immersion into the murky waters of teaching research came when I was offered the position of graduate research methods preceptor in my fourth year. It’s basically a Teaching Assistant position but has an advanced Ph.D. student supporting the three-course statistics sequence required in the first two years of the Ph.D. program. The stats sequence had changed a lot since I took the courses. This came with a change in faculty. The faculty I worked for in the fall semester took a more deeply quantitative approach, using a stats textbook with ample calculus problems. I had never had calculus. Not only did I need to help students one-on-one with homework issues, I had to lead a weekly recitation where they could ask me math questions on the spot. I got by, but boy was it stressful. I was definitely learning to teach. In the spring semester, I had the opportunity to teach about heteroscedasticity in one linear regression course, but that was my only true in-class teaching experience. It was daunting. I fumbled a lot and read many of my notes, but I got through it. At the same time, though, I began to really enjoy my dissertation research. A second year as a methods preceptor helped boost my confidence in teaching methods, but I still had never taught my own methods course. I in no way considered myself a “methodologist” when I was applying for jobs, but I was also not shy about saying I could teach methods. I was sure I could figure it out when the time came. That time was delayed, as my first position at Stockton University (a regional comprehensive) did not require me to teach research methods (whew!). I got my chance, though, at

4

D. J. MALLINSON ET AL.

Julia:

my current position. When I interviewed at Penn State Harrisburg (a branch campus of an R1), I was directly asked about teaching methods. In fact, it would become 2/3rds of my teaching load. Of course, I agreed to do it, but I did not fully appreciate how involved I would be in methods. The program decided to split their current Masters in Public Administration methods course into a two-course sequence (design and stats) and I was asked to develop the second part. Has that ever been a pedagogical and bureaucratic journey. I designed the course, which had to be approved through Penn State’s byzantine process and also needed to be developed for our online MPA degree. In addition, the program decided that the course should be taught using Excel for statistics. That was not in my wheelhouse. I typically use R for all of my work. My first experience teaching the old version of the course was very intimidating. Though I had learned all of the basic design and stats material in grad school, I never taught it and had not reviewed a lot of it since my preceptor experience. Also, I was teaching MPA students, nearly all of whom wanted nothing to do with research and statistics. Even better, the course was cross-listed with Health Administration, and there are always more MHA students than MPA, but the textbook for the course is for public administration. I quickly noted that all of my examples were also either public administration or political science (ideology and polarization were weird example choices for this class). Then, in my second year, we started having PhD students take the course. The course became pretty unwieldy in terms of the wide gaps in expectations across my three audiences (MHA, MPA, and Ph.D.). Like so many in this book, though, I’ve adapted, failed, learned, and grown. Teaching research has been a humbling experience in so many ways, but it has also made me a better researcher and teacher. You really have to break down the concepts and take a “less is more” approach to the subject, otherwise students get nothing out of the class. I have learned so much from reading the chapters of this book and I hope you will take as much away as I have. As an immigrant to the United States, I often felt uncertain about my writing. This feeling was confirmed (over and over

1

INTRODUCTION TO TEACHING THE RESEARCH PROCESS

5

again) in graduate school as my committee members regularly would return pages of my dissertation with red marks all over them. Similarly, when I started graduate school, I felt woefully under-prepared when on the first day of one of my classes the professor started off the class by ensuring that all students knew how to run a regression analysis. I did not. My undergraduate training at a small directional public institution had required a course in the Statistics Department and an interdisciplinary social science research course, but these courses were not well-integrated into the political science curriculum nor did they teach us any related skills. Imagine my concern when, as a newly hired assistant professor, I was asked to teach the department’s “Writing and Research” course. While I felt confident in my ability to use methods and design research, I had never imagined that I would be asked to teach a course on research—surely this would be left to someone whose graduate training emphasized methodology. There I was, less than a month out from defending my dissertation, teaching undergraduate students about research and writing. What I felt I lacked in topical expertise, I made up for in both passion and significant experience in teaching. By the time I started my tenure-track position at University of South Dakota, I had been a teaching assistant for several courses and had independently taught nine separate courses. I love teaching and was already then well-known for my active learning pedagogical style and creative class assignments. What I was not as (at least mentally) prepared for, was teaching courses outside of my areas of study, including research and skills-based courses. I found myself grateful for having had the opportunity to meet the retiring lecturer who taught the course before me and to receive her notes, even if, ultimately, they primarily helped me realize the ways in which I did not want to approach the class. In my previous teaching experience, I regularly asked upperdivision students to write research papers as their final assignments. I sometimes would divide the class into four different groups depending on their past experiences and future goals to tailor the final assignment better. Students with very limited research experience and/or who primarily had ambitions to

6

D. J. MALLINSON ET AL.

enter the workforce directly were encouraged to write stronger descriptive pieces, while the pre-law students were encouraged to focus on the strength of their theoretical arguments, finally, students with more methods training and with ambitions for graduate school would write empirical papers with a testable hypothesis and some basic research design. Although this approach worked very well, I realized it was because I was able to provide a lot of individual attention as I was only teaching one course and able to pour all of my teaching energy into this class. In addition, in a course focused on content, I felt freer to leave the skills to the students. Enter the Writing and Research class, and I realized that this approach would not be sustainable. I was suddenly teaching three courses at once, two of which were outside my comfort zone. Furthermore, in a course designed to teach students skills rather than content I realized I would not be able to teach four different sections of students at once—I had to adjust. Simultaneously, as with many ambitious new professors, I welcomed the idea of teaching a rigorous empirical research paper to students. I did not want my students to feel the shame and embarrassment I did when I started graduate school and so I was determined to teach them how to write a proper empirical research paper. I decided to start at square one and not assume they knew anything about writing or research. More than that, based on my peripheral observations of English language teaching and students’ preparedness for writing at the college level, I realized that students either do not cover some basic writing concepts in earlier grades or do not recognize how this material still applies to collegelevel writing. In addition, beyond writing, students have yet to recognize that writing skills are cumulative on the one hand, but that there is a variety of writing styles and contexts where they will apply these skills. With these objectives in mind, I designed my class in three uneven units for the semester, starting with foundations of writing. The first class covers parts of speech, we then graduate to sentences, including punctuation, and end the unit with paragraphs. Students often start the semester, and the unit, with frustration over covering such banal concepts, though

1

INTRODUCTION TO TEACHING THE RESEARCH PROCESS

7

they quickly realize that they were lacking these foundations. I regularly have students tell me they were unfamiliar with concepts such as subjects and predicates, fused sentences, or independent and dependent clauses. Having taken multiple foreign language courses, I decided to approach teaching these basic English concepts as I learned them in French or German class. Although I was eager to teach the empirical research paper, I also recognized that few of our students go on to graduate study (although I had not quite realized how few in that first semester) and that as a general education course, I was bound by South Dakota Board of Regents specified assigned goals that required me to teach students to “write persuasively, with a variety of rhetorical strategies (e.g., expository, argumentative, descriptive).” I decided to teach four different styles of writing with an incremental degree of formal writing styles: the personal statement, an opinion editorial, a policy brief, and a book review. I wanted assignments that were relatable to students’ future ambitions, though this has caused me on more than one occasion the need to defend my decision against the SDBOR’s “e.g.,” noting that these are mere examples. Finally, I spend the entire second half of the semester teaching the empirical research paper using Lisa Baglione’s (who is a contributing author to this Handbook) Writing a Research Paper in Political Science: A Practical Guide to Inquiry, Structure, and Methods. Using her book, I realized my frustration with earlier undergraduate research papers I had read—they tended to be extended research “reports” not empirical research papers. I also realized our fault as faculty in not teaching students the difference between the writing they had done in high school or in English courses and the papers we hoped to see in social sciences courses. Relying on the book and on the previous unit of the semester where we spend much time acknowledging variety in writing objectives and writing styles, I start the unit discussing the differences between research reports and the type of paper I was expecting them to write. The results have surpassed my expectations and this course has quickly become one of my favorites to teach. Students

8

D. J. MALLINSON ET AL.

Eric:

admittedly struggle through the research paper writing process but are motivated by the fact that they choose their own topics and develop their own research designs. Ultimately, they build a lot of confidence as students, writers, and researchers over time as they work to produce and finally deliver an empirical research paper. This class was by far the most intimidating course for me to teach when I started teaching in 2016. Now, it’s a course I take a substantial amount of pride in. Students who have taken the course report that at the beginning of the course they may have felt confident in their writing and wondered why they’d need to (re-)learn those basic concepts, then felt somewhat torn down after realizing that they had not previously mastered those skills. Finally, they tend to say that Writing and Research was one of their most useful classes and that they feel much more confident as writers and as students as a result of taking my course. I have found that the old adage that teaching is the best form of learning has held true for me. Being put in that uncomfortable and intimidating situation of teaching the Writing and Research course, I learned a lot about myself. I am a stronger and more confident writer and researcher now precisely because I had to teach this course. I have also realized that I held many similar assumptions as my students about writing, not realizing its true utility and need for adaptation of style. I am especially excited to edit this Handbook because I often find that writing is not generally part of the conversation when it comes to teaching research methods. I believe that we have created a comprehensive volume that speaks to all parts of the research process. Like many graduate students, I completed numerous courses in research methods during my Ph.D. program, though it was not one of my examination fields. Also like many graduate students, I took advantage of occasional opportunities for professional development in pedagogy, but teaching was not a significant point of emphasis during my graduate training. Finally, like many freshly-minted Ph.D.s, I accepted a university position that included regular sections of our upperdivision research methods course. At the time, though, in August 2015, the ink was barely dry on my doctorate, moving

1

INTRODUCTION TO TEACHING THE RESEARCH PROCESS

9

boxes lined my office walls, and I had to get a methods course together. Stat. Step 1: what textbook should I use? I wasn’t sure. So I did what many new Ph.D.’s do: I called my advisor to ask for guidance. With a book recommendation in hand, I placed an order with the bookstore, and sat down to lay out the course, letting the textbook’s table of contents dictate much of the course substance. It was not a bad class, yet comparing that syllabus to the one I use now, I marvel at how much has changed. That first term was unforgettable, sometimes in not-so-nice ways! I leaned heavily on some concepts—like inferential statistics—that, in retrospect, were perhaps not the right priority for that particular course. As a result, time dedicated to other concepts that deserved more attention, like measurement, was truncated. Some activities that sounded great in theory did not work out so well in practice, such as a coin-flipping exercise to demonstrate the central limit theorem. (Note to self: next time, just have students do it using an app! It takes way too long with actual coins!) Some pedagogical decisions I made based on my graduate experience—such as assigning a couple of extensive homework problem sets instead of smaller, more frequent assessments—would have worked out better had I approached them differently. The students were generous and patient, though, and, ultimately, we were all reasonably satisfied. Many things went well, but I remember walks back to my office reflecting on how they could have gone better. Indeed, on many days it felt like I was hitting singles rather than home runs. Sometimes I simply struck out altogether. One of my most memorable teaching experiences came that first semester teaching research methods. The students were completing their midterm. It was my first exam in my new faculty position. With only fifteen minutes left in a seventy-five-minute period, no student had yet completed their test. This was unusual. In my (limited) experience, a few extremely high (or low!) performers usually complete their exams well before the one-hour mark. I polled the room and discovered only a few students were even three-quarters done. Those last few minutes were dreadful. I could feel the tension mounting in

10

D. J. MALLINSON ET AL.

the room. Heads swiveled with greater frequency to check the clock on the back wall. Writing implements began to move faster. Paper whooshed as students flipped from one question to another and back again. Although some students did turn in their exams before the end of the period, about half did not. Many in the latter category hastily dashed off an apology for not completing the exam in the margins of their booklets, noting that they simply ran out of time. It was agonizing, for it was I who owed them an apology. I had not properly aligned my expectations with either the substance of the material or the logistics of the course. Fortunately, a few statistical adjustments while grading exams resolved the Methods Midterm Crisis of 2015, but that course taught me about calibrating standards as much as it (hopefully!) taught students about research design and hypothesis testing. That experience, among others, fueled an ongoing dialogue I’ve had with myself and, increasingly, with others over the years. What is an undergraduate research methods course all about? What are the appropriate learning objectives? For that matter, should we distinguish between “research” and “methods”? When I started teaching this course, I focused much more on the latter. The course prioritized statistics and inference more than it did design and the basics of social inquiry. My methods students today still enjoy (endure?) a healthy dose of quantitative social science, and even conduct and analyze their own experiments. However, I have increasingly focused on the fundamentals of what research itself is all about, and how to navigate its many perils. I also completely flipped the script on course design: Rather than starting with a suitable textbook, I saved the textbook decision until the very end, after I worked out course goals and objectives. Indeed, I eventually opted to ditch a textbook altogether. Ultimately, I am more concerned that students leave my course appreciating the complexities of measurement, the logic of control, and the spirit of scientific significance than I am outfitting them with a statistical tool belt laden with narrow instruments of inquiry that many will not likely use again. The resolution I reached on these critical questions represents only one of countless perspectives methods instructors might adopt based on personal values, student abilities, course

1

INTRODUCTION TO TEACHING THE RESEARCH PROCESS

11

objectives, and institutional standards. Moreover, for most of us, these reflections are ongoing based on evolutions in the field, such as the increasing availability of user-friendly methodological tools like SurveyMonkey and Google Forms, as well as emerging learning priorities based on pressing developments in the political world, such as pedagogical efforts to promote data literacy in the age of fake news. This book represents an extension of the conversation that political science research methods instructors routinely have with themselves and their colleagues concerning the proper scope and nature of research methods courses in our discipline. Our goal with this text is to encourage and facilitate this dialogue more broadly. Having shared a bit of our stories, we now present the mission and aims of the book. This is followed by a brief survey of the state of the field of political science and how our contributors represent a diverse crosssection of the discipline. We next explain the strategy and approach of the book, including the deliberate approach to its organization. Finally, we discuss our contributors and preview what can be found in the rest of the book.

Mission and Aims A great many texts convey the substance and practices of political science research methods: measuring concepts, estimating models, rejecting or accepting hypotheses. This book is not about these skills, crucial as they are to social scientists. This book is about how we define the scope and nature of research methods courses in the first place. What role do they have in political science education? What are the ultimate objectives of research methods courses, particularly for the vast majority of students who will not pursue a doctoral degree that requires their regular application? Simply put, what is the point of research methods, and how do we teach it? These questions often go overlooked, even though research methods courses are common in political science curricula—over 80% of political science departments offer at least one methods course.1 Ironically, the ubiquity of research in our professional lives may be one reason why we do not always give these questions the attention they deserve. Our graduate training rightly focuses on developing methods skills. What it does not typically do is prepare us to teach these skills to others or even to

12

D. J. MALLINSON ET AL.

determine which skills are worth teaching. True, some departments have dedicated political methodologists, but these individuals comprise a small fraction of the discipline.2 They also typically work at research-intensive institutions and may not have the opportunity to work frequently with undergraduate students. The goal of this book is to assist methods instructors on their pedagogical journey by sharing the stories of fellow travelers who have been thinking about critical issues they are addressing in their departments and in their classrooms. It addresses three crucial questions: 1. Who are research methods courses for? One obvious answer is future graduate students, but most of us teaching research methods in the discipline are not at R1 institutions. If political science departments offer/require courses in research methods to all students, it is imperative to reflect on the larger aims and purposes of the course beyond conducting academic research. 2. What should research methods courses cover? If most students are not future political scientists, we must reflect more deliberately on the scope of methods curricula: Do we aim to prepare students to be able to interpret rigorous research in their upper-division courses? Alternatively, should instructors focus more on civic skills, like information literacy and data visualization, so students can correctly interpret common political information like election polls? Perhaps research methods courses should cultivate their students’ general skill sets (e.g., formal/technical writing, data management) that will serve them in future non-academic careers? Instructors, departments, and the discipline itself will benefit from a more deliberate reflection on questions of scope and coverage in our methods courses. 3. How should we teach research methods? Political science departments vary considerably in their resources, personnel, course offerings, and learning objectives. Individuals across the field have encountered numerous challenges—some common, others idiosyncratic—and devised innovative approaches to address them. To better position our discipline to serve its students and our common mission, we should document, organize, and disseminate the many approaches instructors take in the research methods classroom.

1

INTRODUCTION TO TEACHING THE RESEARCH PROCESS

13

The goal of this book is to encourage a formal conversation in our discipline about these questions, and the myriad corollaries that inevitably follow. Our contributors have reflected extensively on both the perils and possibilities related to teaching research methods. Their personal stories showcase the evolution of their pedagogy through experience, and each offers best practices that can serve the wider community of teachers. Ultimately, in this text we focus less on the technical substance of the research process, and more on the experiences that have guided instructors’ philosophies and practices related to teaching it. It is our hope that their journeys and pedagogical life lessons will be valuable to other instructors on the same teaching journey.

Strategy and Approach This book seeks to spark a conversation about how we prepare for, adapt to, and succeed in teaching research methods and writing by bringing together authors’ own experiences in the classroom. Our book is unique in that it brings together political science teacher-scholars and librarians who work in all aspects of the research process, including information literacy, the research design, statistics or traditional research methods, and the research writing process. In an effort to illuminate how to effectively teach the political science research process, especially when few of us are extensively trained how to do so, the contributors profile their own experiences in developing their teaching practices, reflect on their pedagogical approaches, and share their learning objectives. In particular, the auto-biographical narratives provide us a glimpse at how the pedagogical philosophy surrounding teaching the research process has evolved over time both individually and broadly speaking across the discipline. One of those apparent changes has been to embrace a more holistic view of research methods. We purposefully broadened our scope to the whole research process instead of focusing only on empirical political methodology. Students should be learning not only statistics, but also the analytical, writing, and oral presentation skills necessary to consume and produce rigorous research. Indeed, many political science programs require a methods sequence that incorporates two or more of these objectives, such as one course on statistics and another on writing. Other programs offer courses where the instructor includes several of these units in one class. Indeed, the section on developing a research design is the largest section of our Handbook with 13 chapters.

14

D. J. MALLINSON ET AL.

As we considered the various parts of the research process, we structured this Handbook around four core elements of the political science research process: information literacy, research design, methods and statistics, and research writing. The pedagogy literature informs us that one of the more effective means to consider learning objectives is to use active verbs relating to skills we hope for students to achieve, rather than solely focusing on memorizing vocabulary for the sake of gaining knowledge (Adams 2015). The four-part research process involves four different activities, consuming research, developing research, producing research, and delivering research. Section 1: Information Literacy. Information literacy is defined by the American Library Association as “a set of abilities requiring individuals to ‘recognize when information is needed and have the ability to locate, evaluate, and use effectively the information’” (https://alair.ala.org/han dle/11213/7668#f1). In other words, information literacy seeks to teach students to become savvy consumers of information. In today’s environment of rapid, digital, and profit-driven media, students are bombarded with a plethora of information each and every day, and, unfortunately, a substantial portion of this information is misguided, unclear, or even deceitful. While information literacy is an important foundation in any academic subject, we see this as particularly important in the realm of politics and political science where the fair distribution of accurate information should be a cornerstone of our democracy. Section 2: Research Design. Some instructors and programs have moved away from research courses that focus solely on statistics, allowing students to focus more broadly on developing a comprehensive reserach project that incorporates a multitude of valuable activities. In these courses, students are often tasked with developing an empirical research project with at least some hypotheses, research design, and interpretation of results, and often a full research paper. The foundational argument here has been twofold: to connect research methods to the bigger picture that is the research process and to provide an opportunity to see the application of methods to social science questions. In some cases, instructors have used this type of pedagogy to replace traditional statistics-heavy courses. In other cases, these courses are part of a broader sequence of the research methods curriculum. And, in yet other cases, these are assignments embedded in substantive and/or capstone courses. Section 3: Research Methods. Traditionally, research methods courses have been taught as a seminar in statistics, with the aim of teaching

1

INTRODUCTION TO TEACHING THE RESEARCH PROCESS

15

students to produce some statistical analysis. This approach remains common at the undergraduate level, either as a part of a multi-course methods curriculum or as a primary feature in a required research methods course. Producing research also becomes central to graduate (master’s and doctoral) research methods teaching. These courses are typically heavy on quantitative methods and seek to teach students both descriptive statistics, including basic probability theory and the Central Limit Theorem, along with some foundational inferential statistics, typically including confidence intervals, z- and t-scores, and perhaps regression analysis. In addition, these courses are often taught in computer labs allowing students to learn in an applied setting using statistical programs, such as Excel, SPSS, STATA, or R. Section 4: Research Writing. Research writing is the last part of the research process and focuses on delivering the research to an audience. As with the development of research, research writing has been implemented in a variety of ways, such as a stand-alone course, an embedded assignment in a research methods course, a substantive course, and/or a capstone course. In some cases, research writing is strongly tied to research development as students are asked to write an empirical research paper that includes their own methodology and interpretation of results. These research papers are typically comprehensive, asking students to write a full paper with sections on introduction, literature review, theory, data and methods, results, and discussion and conclusion (Baglione 2019). In other cases, assignments focus on one section of a typical research paper, most commonly on writing a literature review. The organization of this book is loosely tied around the order in which these skills are generally, or should be, mastered, with literacy being a foundational step, designing the research next, and learning to produce results and delivering a written work generally coming later. That said, and as many of the chapters affirm, we acknowledge that the research process is fluid; for some instructors, these activities may happen simultaneously. In the end, though, this volume speaks comprehensively to the many constituent parts of a quality research methods learning experience. Indeed, the most effective learning emphasizes the relationship between and among all parts of the research process.

Contributors We are intentional about elevating a multiplicy of voices in this text. Our contributors represent both individual and institutional diversity so that the book is accessible and relevant to many audiences. It also reflects

16

D. J. MALLINSON ET AL.

the true diversity that exists in political methodology teaching across the discipline. While the research enterprise on political methodology has historically suffered from a strong bias toward white male researchers (Hidalgo et al. 2018), we contend that those engaging in methodological work as well as teaching methodology and the research process are today more diverse. This Handbook and our contributors purposefully reflect that contention. In terms of institutional diversity, it is important to think about how the research process is taught differently and tackles different learning objectives depending on whether it is taught at a community college, liberal arts college, comprehensive university, or a research-intensive university. These academic environments differ considerably in a variety of ways, from the types of students they serve to the resources to which they have access, such as data management software or survey design platforms. The differences in learning objectives are perhaps starkest when comparing graduate and undergraduate courses. Thus, we include voices from all types of schools and both graduate and undergraduate instructors. Upwards of 85% of political science programs offer at least one course in research methods.3 Further, most political science programs have a methods requirement in their degree program. However, these courses vary widely in their mission, aim, and scope. To capture more precisely the various approaches instructors use, we surveyed contributors of this book about their programs, and we present some highlights here. The assembled contributors are broad and diverse in their backgrounds, approaches, institutions, and degree programs. While not precisely a random sample of instructors, the contributors represent a broad cross-section of the discipline. Figures 1.1 and 1.2 summarize the nature of programs represented by the contributors. Authors come from numerous types of institutions and serve a variety of degree programs. Most of the programs represented by the authors do not offer credentials beyond the Bachelor’s degree, meaning undergraduate political science majors make up the primary audience of our authors. But graduate faculty are an important part of this conversation, too. Contributors’ home institutions vary in terms of the number of research methods courses available to students, as well as the number of courses required of them to complete a political science degree. Figures 1.3 and 1.4 present a summary of these data. Ninety-five

1

INTRODUCTION TO TEACHING THE RESEARCH PROCESS

17

Fig. 1.1 Distribution of contributors by institution type

Fig. 1.2 Highest degree offered at contributors’ institutions

percent of contributor institutions offer at least one methods course (Figure 1.3), and approximately two-thirds of programs offer between one and two courses. Fewer than a quarter of programs offer more than three methods courses. Figure 1.4 illustrates that availability of courses correlates strongly with program requirements. Approximately 13% of programs require no methods courses, 68% require one or two methods courses, and 18% require three or more. Overall, these data underscore that research methods courses are a common feature in political science curricula, and most students in our programs are required to engage with them to some degree.

18

D. J. MALLINSON ET AL.

Fig. 1.3 Number of methods courses available

Fig. 1.4 Number of methods courses required (undergraduate)

At the same time, there are substantial differences in how instructors approach their courses. In some ways, programs have similar approaches. For instance, roughly nine out of ten programs require a capstone experience of some kind. Similarly, 93% of programs mandate a writing-intensive methods course in their sequence. Yet in other ways, differences are evident. Approximately 32% of programs emphasize statistics in their

1

INTRODUCTION TO TEACHING THE RESEARCH PROCESS

19

Fig. 1.5 Contributors’ program’s main priorities

methods courses. Forty-six percent of programs emphasize information literacy. We asked contributors to indicate the main goal of their program’s methods sequence. Figure 1.5 summarizes these priorities. Although research design is the most common objective, many programs also emphasize other learning outcomes, such as statistical acumen and writing skills. Some contributors report that their program does not focus on a specific objective, while others indicate that their departments pursue multiple goals simultaneously. Other individuals indicate that program goals in their department are instructor dependent. These approaches and emphases each have their advantages and drawbacks. More importantly, instructors in these programs all have a story about their pedagogical evolution based on the varied contexts in which they honed their teaching craft. As peers, we stand to benefit from hearing these perspectives and reflecting on our own approaches in the research methods classroom.

Chapter Summaries This Handbook is structured to follow the research process. The first section addresses information literacy, the second research design, the third research methodology, and the fourth research writing. Although many of our authors teach multiple aspects of the research process in their courses, the authors of each section enter their pedagogical reflections on

20

D. J. MALLINSON ET AL.

that section’s theme. Within each unit, readers will find stories that vary widely in scope: some chapters discuss broader pedagogical questions, while others focus on narrower teaching techniques.

Information Literacy Emily Reed begins this section with a librarian’s perspective on information literacy. Reed’s chapter offers an overview of the standards of information literacy, as well as suggestions and recommendations for faculty to best use the resources libraries and librarians provide. Chelsea Kaufman then discusses the integrated relationship between consuming (information literacy) and producing (statistics) research. In particular, Kaufman discusses how to relate classroom learning to “real world” applications in communicating research to the general public. Jennifer Epley Sanders illustrates how deliberately integrating student identity into conventional statistics-based methods courses can promote information literacy in undergraduate research methods courses. Kristina Mitchell’s background in both political science and K-12 science education illuminates how a broader understanding of science literacy can help faculty better connect with students. Doing so allows faculty teaching social science research to speak the language that students acquire in their early education. Tavishi Bhasin asserts that methods instructors should focus not only on information literacy, but more fundamentally on scientific history, when they reorganize courses to prepare students to consume political science rather than produce it. Sally Friedman and Trudi Jacobson discuss their faculty-librarian partnership and how to use the meta-literacy framework to teach research literacy. Jonathan Ring advocates for an approach to methods teaching that prioritizes students as consumers of political information and emphasizes a few fundamental learning objectives related to information literacy rather than an extensive set of skills related to conducting research. Finally, Verónica Reyna brings a community college perspective to the Handbook and discusses how to teach methods and the social science research process in a curriculum without a methods course. Reyna points out that teaching methods is about methodological and systematic thinking. These skills can be imparted through means that empower students when taught alongside civic skills.

Research Design Jessica Hejny begins the section by presenting her central aim to train students in how to think like social scientists. She presents a course

1

INTRODUCTION TO TEACHING THE RESEARCH PROCESS

21

where students are introduced to a range of political science methods and encouraged to build their own research design proposal using the approaches taught in class. Christina Fattore demonstrates how to be authentic in the classroom when teaching students about research design. Fattore’s narrative is refreshing, engaging, and honest. It demonstrates how authenticity is possible and makes us better teachers. Kelly Bauer explores the value of imagining research methods as professional development that prepares students for career paths that do not necessarily include academic scholarship. Bauer emphasizes the broad value of knowledge about research methodology rather than specific statistical applications. Jarrod Kelly then discusses tools for navigating teaching an interdisciplinary research methods course (political science and sociology). Kelly’s approach includes focusing on the transdisciplinary nature of the research process and having students pick campus problems to study for their practical exercise. Jennie Sweet-Cushman argues for a “less is more” approach to teaching methods that allows students to gain mastery over fewer methods, rather than having incomplete or surfacelevel knowledge about many methods that will not serve them well in capstones or careers. Robert Postic reviews the arc of his methods teaching trajectory, from adjunct to full professor, and how he has evolved to embrace teaching methods. This includes further developing his own skills in statistics and online teaching. Postic encourages readers to leverage anxiety over teaching methods to better themselves as an instructor. Debra Leiter encourages instructors to focus on the research process over research outcomes and suggests replacing dedicated statistical analysis software with widely available online data analysis platforms. Research projects that prioritize sound theories and design over specific data findings help promote student interest and success. Amanda Bittner describes a pedagogical approach that evolved to center principally on encouraging creativity and emphasizing collaboration. This chapter describes these two pillars in detail and explores their application in Bittner’s Gender and Politics Lab. Michalene Cox takes the reader on a journey through an arts-based approach to teaching research. Cox notes that the transdisciplinary Arts-Based Research approach bridges a connection between science and the humanities allowing for teaching both creativity and analytical skills. She uses the example of a photo essay assignment to illustrate this pedagogical approach. Cherie Strachan argues that positivist political science is inherently normative, not a removed objective study. Strachan incorporates critical theory, particularly feminism, into

22

D. J. MALLINSON ET AL.

teaching students about political science research. Nadia Brown, Jasmine Jackson, Aayana Ingram, India Lenear, and Ariel D. Smith describe how Brown used the Sistah Circle concept to form a supportive Black Lady Classroom for graduate students at Purdue (a supermajority white institution). The model recognizes the intersectional marginalization of Black women in political science and offers one method for advancing equity and inclusion in graduate methods training. Erik Cleven then describes the challenge of moving past solely conveying technical knowledge to instead cultivating in students the ability to conduct their own research. Cleven offers a series of examples of how teaching practices can emphasize a “learning by doing” approach. Finally, Neil Chaturvedi and Mario Guerrero discuss how their department has integrated rigorous and frequent program assessment to collectively reshape the curriculum, with methods courses featuring prominently as the “connective tissue.” They illustrate the department-level discussions and buy-in necessary for a cogent methods curriculum.

Research Methods John Garcia opens the section by sharing his journey as an “accidental academic.” His story includes extensive involvement with the ICPSR seminar on Methodological Issues in Quantitative Research on Race and Ethnicity and the development of ICPSR’s Resource Center for Minority Data. Lisa Bryant then shares her experience transitioning to Excel, a more ubiquitous and accessible tool than many in common use in academic research, as a statistics software platform. Bryant describes how to utilize Excel to develop a wide variety of data management and analysis skills that serve students in professional settings outside of academia. Andre Audette shares how research methods skills are integrated into elective classes in a department that neither requires nor offers a methods course in its program sequence. Audette demonstrates how to “sneak in statistics” so that research projects can increase the value and relevance of substantive course material, even among students who suffer from math anxiety. Jennifer Bachner emphasizes the need for research methods courses to use practical examples and build transferable research skills. Students in public service careers increasingly need skills like data visualization, data interpretation, quantitative reasoning, and effective writing. These should be part of any methods curriculum. Chris Zorn shares

1

INTRODUCTION TO TEACHING THE RESEARCH PROCESS

23

his journey of becoming an “accidental methodologist,” along with his perspective on the place of methodology, and methodological teaching, in the discipline. Andreas Sobisch relies on extensive experience in teaching the research process to describe how much teaching research has changed. Sobisch notes that methods teaching has transformed through the incorporation of such courses into mainstream curricula and through practical changes such as user-friendly software and data availability. Dongkyu Kim discusses teaching introductory statistics courses in a highly diverse context where students are struggling to balance school with work and family obligations. Kim reflects on the changes he has made in his pedagogy to meet the needs of his students and to make the course more engaging for them. Finally, Whitney Ross Manzo describes the transition from learning methods as a graduate student at a statistics-intensive R1 program to teaching at an undergraduate-only women’s college. The evolution is punctuated by unique challenges and opportunities, such as combating gender stereotypes surrounding STEM and integrating data from the on-campus polling center in her methods courses.

Research Writing William O’Brochta begins the writing section by reflecting on the evolution of his philosophy on writing assignments. O’Brochta highlights how innovative research writing pedagogy increases the relevance of political science itself, cultivates transferable writing skills, and promotes inclusion and collaboration in the classroom. Martin Edwards then presents some of the challenges of teaching a capstone course with independent research projects to students with a heterogeneous skillset. Edwards offers a typology of individual approaches for instructors to consider students’ needs in order to be effective and responsive in coaching students. Lisa Baglione presents her tried and true approach of teaching students research methodology through research writing (Baglione 2019). Baglione argues that doing so helps students understand the skills they bring into the process from previous courses, while helping to hone and draw together those skills for their own independent research. This approach can give students at smaller schools a competitive advantage over students at more prestigious institutions that do not teach these skills. Colin Brown comments on the challenges of helping students unlearn “good” writing, particularly the five-paragraph format. Students

24

D. J. MALLINSON ET AL.

are often unaware of their writing problems because they are using a tool they were previously taught that lends itself to knowledge declaration, not argument and knowledge production. Emily Farris explains how she centers research writing, and the writing process, in her upper-level Survey Research course. Farris describes how the course has evolved in the five years she has taught it. Lauren Grimes evaluates how differences in political science course offerings across multiple institutions within the same system—a community college and associated university—are a source of student frustration upon enrolling in more advanced courses. Grimes offers insights and suggestions on course content and sequencing, with an eye toward maximizing student success when they reach the university level. Darrell Lovell discusses the challenges of teaching research writing and methods in the increasingly ubiquitous online setting. As more institutions shift courses online, Lovell debunks myths and offers suggestions for effective online teaching that take advantage of technology. Finally, Jessica Rich describes how to teach transferable writing skills in writing-intensive substantive courses. Using topical concepts, Rich seeks to demystify the research process and convey to students that research is a process of small tasks rather than one intimidating project. The stories and data from this chapter make one thing abundantly clear: The philosophies, environments, opportunities, and constraints of teaching the research process vary widely across individuals as well as institutions. We trust readers will agree that the myriad perspectives, stories, and best practices carefully developed by the contributors above set the stage for a rich conversation within the political science community about how we approach the research courses that most students are required to take. We are confident that everyone will discover new ideas and strategies to bolster their pedagogical prowess. Enjoy these stories, and please continue this important conversation in your departments and across the discipline.

Notes 1. www.apsanet.org/Portals/54/APSA%20Files/Data%20Reports/Enroll ment%20Data/APSA%20Departmental%20Survey_Enrollment%20and% 20Curriculum_FINAL.pdf?ver=2019-05-21-113745-243; data also cited in https://preprints.apsanet.org/engage/api-gateway/apsa/assets/orp/ resource/item/5cf54411a0144d0011d9c5d6/original/rethinking-the-und ergraduate-political-science-major-the-wahlke-report-revisited.pdf.

1

INTRODUCTION TO TEACHING THE RESEARCH PROCESS

25

2. Recent surveys by the American Political Science Association reveal that in recent years political methodologists comprise less than two percent of job market candidates (https://www.apsanet.org/portals/54/Files/DSP%20D ata/Data%20on%20the%20Profession/SixYearsofPSDoctoralStudentPlacem ent_022315.pdf), and methods specialists secured only about one percent of academic placements in political science. https://preprints.apsanet.org/ engage/api-gateway/apsa/assets/orp/resource/item/5d2dd689f4cf650 01aa0744b/original/apsa-graduate-placement-report-analysis-of-politicalscience-placements-for-2017-2018.pdf. 3. https://www.apsanet.org/portals/54/Files/DSP%20Data/Data%20on% 20the%20Profession/CurriculumReport%20201213DeptSurvey%20FINA L21814.pdf.

References Adams, N. 2015. Bloom’s Taxonomy of Cognitive Learning Objectives. Journal of the Medical Library Association 103 (3): 152–153. https://www.ncbi.nlm. nih.gov/pmc/articles/PMC4511057/. Baglione, L. 2019. Writing a Research Paper in Political Science: A Practical Guide to Inquiry, Structure, and Methods. Newbury Park, CA: Sage. Hidalgo, F. Daniel, Suzanna Linn, Margaret Roberts, Betsy Sinclair, and Rocio Titiunk. 2018. Report on Diversity and Inclusion in the Society for Political Methodology. February 21. https://www.cambridge.org/core/membership/ services/aop-file-manager/file/5a7065fec871636539c5c70a/diversity-rep ort-SPM.pdf. Accessed October 23, 2020.

PART I

Teaching Information Literacy

CHAPTER 2

Information Literacy Development of Undergraduate Political Science Student Researchers: The Instruction Librarian’s Role Emily Reed

Training for Academic Librarians There is no such thing as a typical path to librarianship. Librarians come from a variety of educational backgrounds and professional experiences. My own path to academic librarianship began with my pursuit of a bachelor’s degree in music education. When I graduated, I entered a professional world at a time when local schools were not hiring new teachers of any subject or grade level. I found part time work at my local public library. A year later, I found full time employment in a school library, and a year after that, I began my Master of Library and Information Science (MLIS) program. My work in public libraries taught me to value the public service that libraries provide their patrons. My work in the school library taught me a love of purchasing materials and interacting

E. Reed (B) University Libraries, Penn State Harrisburg, Middletown, PA, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_2

29

30

E. REED

with students and teachers. Once I graduated with my MLIS, I was armed with my undergraduate degree in an educational field and experience working in two different library types. I landed my first professional position as an Instruction and Reference Librarian at a small, private college in my local area. I have also taught credit-bearing first-year seminar courses and an undergraduate capstone research course. I began my current job as a Reference and Instruction Librarian for Penn State University Libraries at the Harrisburg campus in September of 2019. To work at an academic library in the U.S., job requirements nearly always include the terminal degree, an MLIS (or MLS) from an American Library Association (ALA) accredited institution. Many academic librarians have additional master’s or other advanced degrees. While most academic librarians harbor a love for learning and are service-oriented, only some are specifically interested in teaching. The library is usually expected to provide instruction and reference services in some capacity to the institution.

Information Literacy in Higher Education ACRL Information Literacy Standards and Framework In my first professional position, I provided library instruction for the college. My initial course of action was to investigate best practices in librarianship for teaching library instruction. In my exploration, I intensely studied the formal documents that guide academic libraries to have a successful information literacy (IL) program. My journey to becoming an IL teacher begins with ALA. Libraries in the U.S. and around the world look to ALA for guidelines, best practices, research, professional development, and support. The Association of College and Research Libraries (ACRL) is a division of the ALA and focuses on providing support and resources specifically for academic libraries. ACRL developed The Information Literacy Competency Standards for Higher Education (IL Standards) and formally adopted them in 2000. The purpose of developing overarching IL Standards is to help guide academic libraries and institutions of higher education to develop independent undergraduate researchers. The IL Standards detailed a list of six standards including performance indicators and measurable outcomes.1 While ACRL officially rescinded the IL Standards document in 2016

2

INFORMATION LITERACY DEVELOPMENT OF UNDERGRADUATE …

31

in favor of a more flexible framework, many institutions still actively use the Standards for planning and assessment purposes. The Association of American Colleges and Universities uses the IL Standards as the foundation of their Information Literacy VALUE Rubric (2009). In what is still considered a highly controversial decision in 2016, the ACRL Board rescinded the IL Standards because they were deemed too prescriptive, not reflective of the nuances surrounding information processes, and no longer accurate regarding born-digital information. The new Framework for Information Literacy for Higher Education (IL Framework) redefines IL as “the set of integrated abilities encompassing the reflective discovery of information, the understanding of how information is produced and valued, and the use of information in creating new knowledge and participating ethically in communities of learning” (2016). Additionally, the Framework distinguishes six interconnected concepts that together serve as a flexible framework for IL skill development for undergraduate college students. The IL Framework builds on research about backward design (Wiggins and McTighe 2004), threshold concepts (Meyer and Land 2003), and metaliteracy theory (Mackey and Jacobson 2014). The six frames are: Authority Is Constructed and Contextual—“Information resources reflect their creators’ expertise and credibility, and are evaluated based on the information need and the context in which the information will be used. Authority is constructed in that various communities may recognize different types of authority. It is contextual in that the information need may help to determine the level of authority required.” Information Creation as a Process—“Information in any format is produced to convey a message and is shared via a selected delivery method. The iterative processes of researching, creating, revising, and disseminating information vary, and the resulting product reflects these differences.” Information Has Value—“Information possesses several dimensions of value, including as a commodity, as a means of education, as a means to influence, and as a means of negotiating and understanding the world. Legal and socio-economic interests influence information production and dissemination.”

32

E. REED

Research as Inquiry—“Research is iterative and depends upon asking increasingly complex or new questions whose answers in turn develop additional questions or lines of inquiry in any field.” Scholarship as Conversation—“Communities of scholars, researchers, or professionals engage in sustained discourse with new insights and discoveries occurring over time as a result of varied perspectives and interpretations.” Searching as Strategic Exploration—“Searching for information is often nonlinear and iterative, requiring the evaluation of a range of information sources and the mental flexibility to pursue alternate avenues as new understanding develops.” Each frame has an expanded definition as well as knowledge practices and dispositions.2 Because the IL Framework was intended to be flexible and not prescriptive, the knowledge practices and dispositions are merely examples of how instructors can implement and students can inhabit each frame. ACRL urges each institution to develop appropriate IL learning objectives and outcomes using the IL Framework as a guide; institutions can tailor each frame to its own unique needs and context. I use the IL Framework to identify which IL concepts are most critical to student success in the programs with which I liaise and to develop lesson plans. Information Literacy in the Political Science Curriculum ACRL offers many committees, sections, interest groups, and discussion groups for members. One section that I actively work with is the Politics, Policy and International Relations Section (PPIRS), previously known as the Law and Political Sciences Section (LPSS). PPIRS “serves as an educational forum and information exchange for librarians with an interest or subject expertise in politics, policy and international relations” (American Library Association 2016). PPIRS offers its members opportunities for professional development, discussion, and guidance based on research and best practices in political science librarianship. In 2008, LPSS developed a set of guidelines specifically for political science research competency for undergraduate students based on ACRL’s IL Standards. The Political Science Research Competency Guidelines (PSRCG) adapts the IL Standards along with performance indicators and outcomes; it also features

2

INFORMATION LITERACY DEVELOPMENT OF UNDERGRADUATE …

33

specific examples as applicable in political science programs. The document recommends whether the course instructor or the librarian should have primary responsibility for each outcome or share the responsibility together. When interacting with research methods courses, I usually focus on teaching students to identify appropriate informational resources and develop search strategies. These informational resources can be qualitative or quantitative. Because my teaching focuses on the early stages of the research process, I usually select performance indicators and outcomes from the first two PSRCG Standards which focus on determining and accessing information. The lesson plan I develop depends largely on discussions with the course instructor. One lesson that I frequently teach is how to access different types of information (journal articles, polls, datasets) and develop research strategies, usually in anticipation of a research assignment. I start the lesson with a class discussion about the research assignment and ask the students what their research topics are. This naturally leads into a conversation where students discuss what type of information they think they need to locate in order to successfully research their topic (PSRCG Standard 1, Performance Indicators 2 and 3; IL Frames “Authority is Constructed and Contextual,” “Information Creation as a Process,” and “Scholarship as Conversation”), such as: ● Do they need statistics or data? Primary sources? Secondary sources? ● Do they think they need scholarly materials? Why or why not? ● How recent does the literature need to be, and what is the justification? ● What types of sources are available if they are searching for an extremely current topic (and the formal research-publication process is too slow)? After this conversation, I lead a demonstration showing students how to identify which library databases and other online resources can fulfill their information needs. Students learn how to access resources, craft search queries in one or two of the resources using Boolean operators (AND, OR, and NOT), evaluate results by previewing the record and reading the abstract, refine their results using filters, and save sources (PSRCG Standard 2, Performance Indicators 1, 2, 3, 4, and 5; PSRCG

34

E. REED

Standard 3, Performance Indicator 2; IL Frames “Research as Inquiry” and “Searching as Strategic Exploration”). I model these behaviors and have students follow along on their own devices; sometimes I provide a worksheet to help them record notes about the strategies learned in class. The PSRCG also recommends specific outcomes for which the ILfocused course instructor can take responsibility, such as teaching students how to develop a research question or thesis statement (Standard 1, Performance Indicator 1), how to test theories with discipline-appropriate techniques (Standard 3, Performance Indicator 4), and how to select the most optimal format to present research (Standard 4, Performance Indicator 3). Take-Aways Program and course coordinators who are responsible for developing the political science curriculum and/or research methods courses should collaborate with a librarian to: 1. Review the PSRCG (or the IL Framework) to identify which standards, performance indicators, and outcomes are most critical for students to be successful in the undergraduate political science program. 2. Map the selected standards, performance indicators, and outcomes throughout the political science curriculum in a way that scaffolds student development in IL. 3. Develop instructional activities that assess measurable understanding and successful demonstration of the outcomes.

Library Instruction for Information Literacy One-Shots The most common pedagogical format I use to teach students is the “one-shot” library instruction session, which is considered the signature pedagogy of instruction librarians. These sessions are usually at the invitation of a course instructor who wants a librarian to guest teach a library or research-related topic, such as searching library databases, evaluating sources, source integration, among others. My process for preparing a one-shot involves learning about the course objectives, the need or

2

INFORMATION LITERACY DEVELOPMENT OF UNDERGRADUATE …

35

problem the instructor wants the librarian to address, and logistics for delivering the library session. I then develop a lesson plan that includes learning outcomes and activities; I also make sure to map the lesson to the IL Framework, the PSRCG, and institutional competencies. I have found one-shot library instruction sessions to be a conflicting experience. I have taught well over 200 one-shots, and there are many variables involved that contribute to its success including classroom layout, available technology, class dynamic, and student buy-in of the guest lecturer and lesson content. In my experience, the most influential element of a successful one-shot is the course instructor. If the instructor speaks positively about the librarian’s visit during class before the library session takes place, students are more likely to attend and engage. I prefer for an instructor to stay in the classroom because students are more likely to stay focused and the instructor can speak up during the library lesson to punctuate or contextualize certain points. The instructor can also publicly acknowledge the authority of the librarian which can help break students of any “stuffy librarian” stereotype and recognize that the librarian has expertise from which they can benefit. The other variable worth mentioning is the timing of the lesson. Students are most likely to engage with a library lesson when they recognize that they are at their point of need. If the lesson focuses on searching informational resources, then the instructor and librarian should work together to schedule the lesson after the instructor has introduced a research assignment with an upcoming deadline, but before the students have done a significant amount of research. One-shots also help build rapport with students; when students see me more than once, they begin feeling more comfortable with my teaching style and begin to speak up more. Students are also more likely to seek me out for assistance after I visit their class. In a research methods course, the librarian’s lesson plan can be determined in conjunction with the course instructor. Examples of topics relevant to a research methods course include searching political science databases, searching for Census data, locating polls, organizing information with citation managers, developing a data management plan, information ethics, and copyright and fair use. My lesson planning process utilizes the Backward Design process (Wiggins and McTighe 2004). For any political science course, I first identify which outcomes I want to use from the PSRCG. Next, I decide

36

E. REED

what evidence will be acceptable that proves that the students have successfully achieved the outcomes, and lastly, I design my instructional activities. My most frequently requested one-shot is bibliographic instruction, which primarily consists of database and search strategy demonstrations. I prefer to host the session in the library’s computer lab so that students can model my information seeking behavior. I always try to impress upon the students that many search strategies are transferable between informational databases. Library databases and other online resources I most frequently demonstrate to political science student researchers include Worldwide Political Science Abstracts, Congressional, CQ Electronic Library, and Inter-university Consortium for Political and Social Research. Because most undergraduate students do not initially consider bibliographic instruction to be an exciting experience, student engagement is a major consideration when I plan my lessons. To help students become more engaged, I will often ask them at the beginning of the class for their own research topics to stimulate interaction and so that I can demonstrate search strategies that are directly relevant to their research. I invite additional interaction by asking the students questions throughout the lesson as formative assessment so that I know if I need to clarify anything. I sometimes also bring worksheets so that students can write down which databases they believe will be most useful for their project as well as search strategies, keywords, and synonyms. The worksheet helps them to stay focused on the lesson and gives them a tangible reminder of what they learned and how they can apply that information when they recreate the research process later. Take-Aways When asking a librarian to provide a library instruction session for your students: ● Begin planning for the session before the semester begins and work with the librarian to find a date (or range of dates) so it can be listed on the syllabus. The best timing to schedule bibliographic instruction is usually after a research project has been assigned, but before students have done too much work on their own. ● Tell the librarian as much as you can about your course. What assignments will your students be working on? What are your

2

INFORMATION LITERACY DEVELOPMENT OF UNDERGRADUATE …

37

course objectives? What common problems are you noticing in your students’ work? ● Send the librarian relevant course documents, such as your working syllabus and assignment instructions. ● Talk up the library lesson before it occurs. Encourage students to attend and assure them that they will learn skills that will be critical to their success in the course. ● Be open to new ideas. The librarian may suggest a different topic for instruction based on your conversation or even a different mode of instruction other than the one-shot depending on the topic and the class logistics. Other Instructional Modes While the one-shot is the most common instructional mode, there are other modalities that I employ to meet the unique needs of a class based on content, logistics, and preference of both the instructor and me. Since I interact with many courses, it is critical that I create flexible and sustainable learning opportunities (lesson plans and learning aids) so that I can reuse them with minimal modification. Live Webinars and Pre-Recorded Lectures My webinars are similar to my one-shot sessions but delivered synchronously online through a video-conferencing platform. I have found that webinars work best when scheduled in remote synchronous courses, though I like to offer them as an option to asynchronous course instructors too. When I offer a webinar, I always record it so that students in the course can view the webinar later. I also honor requests for pre-recorded lectures for asynchronous courses. Both webinars and pre-recorded lectures have different advantages and challenges such as amount of prep time, control over the learning environment, flexibility to adapt the content on the fly, measurable student engagement, scalability, and sustainability. Both webinars and prerecorded lectures are conducive to many IL topics. In the past, I have offered webinars and pre-recorded videos that the instructor can post in the learning management system (LMS) that focus on topics such as using the advanced search option in Google Scholar, downloading citations from library databases, and finding print materials using the library catalog.

38

E. REED

Digital Learning Objects Digital learning objects (DLOs) are learning artifacts that support instruction. DLOs can include videos (discussed above), online modules, digital badges, and research guides. They can be implemented into both inperson and online courses via the LMS. Online modules and digital badges are ways that library instruction can be directly embedded in the LMS. Both can provide information about how to conduct research, information ethics, how to cite sources, and more. They offer students an asynchronous option to advance through the content at their own pace and can use a variety of text, images, audio, video, and interactive activities throughout the lesson. Online modules and digital badges do require a good amount of up-front work to develop, but once created, they should be easily reusable. Either the instructor or the librarian can monitor student engagement with online modules. My current institution developed homegrown digital badges. The badges have built-in assessments that require a librarian to review and grade in order for students to obtain the badge. I currently evaluate IL digital badges for students enrolled in a variety of courses at my institution. Online research guides are content-focused web pages that librarians control. These research guides can target a specific subject, course, or topic to provide subject-specific database links, websites, books, search strategies, and librarian contact information. Research guides can be embedded or linked in the LMS. Embedded Librarianship At some institutions, it is possible to embed a librarian into a course via the LMS. An embedded librarian is more deeply involved in the course, so this mode requires extra time and effort on the part of the librarian. Activities that the librarian may coordinate when embedded can include discussion forums, announcements, messages to students, co-grading of assignments, and development of modules in the LMS. I have had the opportunity to be an embedded librarian for several different courses. As an embedded librarian, I often moderate a “Library Q&A” discussion forum which provides students an outlet to ask questions publicly so that the answers are also publicly displayed to the advantage of all students in the course. I post announcements that address common student challenges I see related to research, reminders of upcoming library events, and other timely information such as links to

2

INFORMATION LITERACY DEVELOPMENT OF UNDERGRADUATE …

39

citation style guides. In this modality, the librarian has more of a presence in the course, but it is up to the students to reach out when they need help. Embedded librarianship is an excellent option for researchintensive courses. Students can connect more easily with a librarian, and students often feel the benefit of developing a working relationship with the librarian that can last beyond the course. I have had many students contact me after their course is over to reach out for additional help. Take-Aways When investigating possible IL instruction alternative to the library oneshot: ● Explore your library’s website as well as the website for your institution’s Center for Teaching Excellence (or equivalent). Both sites may have a section devoted to instruction support and may link to teaching resources. ● Contact your librarian to discuss possible options. They will be able to make specific recommendations based on what skills you are trying to develop in your students, institutional resources, and the librarian’s own teaching style preferences. Faculty Support for Information Literacy Librarians have deep expertise in teaching and assessing IL. In a previous position, I mapped IL outcomes from the general education curriculum to institutional competencies and the IL Framework. I worked with faculty to integrate IL development into courses, both in the general education program and throughout individual programs. I also assisted with an assessment project to assess IL in a research-intensive capstone course. Many librarians also have training in instructional design practices and as such, librarians often consult with course designers and instructional designers on overall course design in tandem with course objectives, especially for research-intensive courses such as research methods courses. I have assisted faculty with selecting IL learning objects, developing student activities (such as research assignments), and have provided feedback about rubrics that aim to assess IL.

40

E. REED

Librarians are usually happy to provide training to faculty in a teachthe-teacher model, which is arguably the most scalable and efficient form of instruction. Librarians can train faculty how to incorporate IL into their courses, how to search and maximize use of library databases, how to effectively use citation managers, and more. Instructors can learn these skills either by contacting their librarian for an individual consultation or by attending formal faculty training that may be offered by the librarian.

Conclusion This chapter has focused on the library’s role in IL development for students. The IL documents presented (IL Standards, IL Framework, and PSRCG) have shaped my teaching as a librarian by framing IL as necessary knowledge behaviors and dispositions for students to successfully research independently. The teaching methods presented are all based on my own experiences as an instruction librarian. My most important piece of advice to political science research methods instructors would be this: seek out your librarian. Discuss with them the IL skills your students will need to be successful in your course. Be open to new ideas presented by your librarian. We are on the same team—invested in the academic and lifelong success of our students.

Notes 1. See (American Library Association 2000) for a full list of standards, performance indicators, and measurable outcomes. 2. See (American Library Association 2015) for expanded definitions, knowledge practices, and dispositions.

References American Library Association. 2000. Information Literacy Competency Standards for Higher Education. https://alair.ala.org/handle/11213/7668. Accessed on July 23, 2020. American Library Association. 2008. Political Science Research Competency Guidelines. http://www.ala.org/acrl/sites/ala.org.acrl/files/content/standa rds/PoliSciGuide.pdf. Accessed on July 23, 2020. American Library Association. 2015. Framework for Information Literacy for Higher Education. http://www.ala.org/acrl/standards/ilframework. Accessed on July 23, 2020.

2

INFORMATION LITERACY DEVELOPMENT OF UNDERGRADUATE …

41

American Library Association. 2016. Politics, Policy and International Relations Section. http://www.ala.org/acrl/aboutacrl/directoryofleadership/sections/ ppirs/acr-ppirsec. Accessed on July 23, 2020. Association of American Colleges and Universities. 2009. Information Literacy VALUE Rubric. https://www.aacu.org/value/rubrics/information-literacy. Accessed on July 29, 2020. Mackey, T. P., and T. E. Jacobson. 2014. Metaliteracy: Reinventing Information Literacy to Empower Learners. Chicago, IL: ALA Neal-Schuman. Accessed on July 23, 2020. Meyer, J., and R. Land. 2003. Threshold Concepts and Troublesome Knowledge: Linkages to Ways of Thinking and Practicing within the Disciplines. http:// www.etl.tla.ed.ac.uk/docs/ETLreport4.pdf. Accessed on July 23, 2020. Wiggins, G., and J.McTighe. 2004. Understanding by Design. Alexandria, VA: Association for Supervision and Curriculum Development. Accessed on July 23, 2020.

CHAPTER 3

Should Research Methods Teach Information Literacy or Statistics? Why not Both? Chelsea N. Kaufman

From Facing Fears to “Fake News:” The Evolution of My Teaching Philosophy My passion for research methods began when I was an undergraduate student. My coursework, a professor who encouraged me to further pursue the research I had begun in her class, and an internship where I gained experience using SPSS led me to be well-prepared when I enrolled in my first research methods seminar in graduate school. I still remember my excitement when I received the assigned textbook for that class in the mail and knew that I would soon be analyzing data in Stata. I was thrilled when I was assigned the next year to be a teaching assistant in our undergraduate research methods course. Given this background, I am sure that you can imagine my surprise when I reported for this assignment and learned that this subject had a reputation for being difficult and intimidating to students. I suspect

C. N. Kaufman (B) Department of Political Science, Wingate University, Wingate, NC, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_3

43

44

C. N. KAUFMAN

this gap between my expectations and reality came from my early exposure to research, which made the importance of learning these tools clear, while many students saw the course as a hurdle to graduation that would provide little to no direct benefit to their future. With each new experience I have had as a methods instructor, my reflections have led me away from this point and to a point where I now teach the course in such a way that makes research methods meaningful and useful to students while also instilling them with the knowledge that they can succeed. When I began my TA assignment, I was fortunate that the professor had designed the course in a way that enabled us teaching assistants to walk many of the students through the material successfully. Her mentorship was crucial to my success the first time that I taught the course on my own. That course was focused primarily on teaching students to calculate statistics and carry out and interpret the associated analyses in SPSS, ending with ordinary least squares multiple regression analysis (OLS). As use of SPSS was highly intimidating to most students, I began with lessons about use of the program early on. Students also had assignments on basic topics, such as understanding rows and columns in a spreadsheet (many of them had never used any software for data analysis, including Excel). They were largely able to succeed because we proceeded toward the final goal in very tiny steps. The next time that I taught the course, I began to strike out in my own direction. I had noticed that although the design of my course was conducive to helping students to succeed despite their fears, I had not done much to actually ease their fears. The first change that I made was that on day one, I asked students to work through a series of what appeared to be simple math problems. At the end, I informed them that they had just completed the most difficult calculation that I would ask them to perform on an examination: the standard deviation. By starting the class in this manner rather than placing an intimidating looking formula on the board weeks later, I started to break down this barrier. I also tried to find ways to make the course more engaging to the students, such as using data about the 2016 presidential election. At that time, however, I still felt that the most important goal of the class was to impart students with the knowledge of how to carry out and interpret OLS using SPSS. Beyond using data from the “real-world,” students were not shown clear connections between this material and its relevance to their own lives. Additionally, I realized that by designing the course in response to what was happening in the classroom—for example,

3

SHOULD RESEARCH METHODS TEACH INFORMATION …

45

if students seemed off-put by calculating the standard deviation, creating an assignment to address that obstacle—I was ignoring my own methodological and pedagogical training. I began to intentionally update the course using backward design, included additional content on qualitative methods, and incorporated high-impact teaching practices such as active learning exercises. My other research and teaching interests also began to shape my approach to this course. I noticed while teaching American government that many students lacked the information literacy to evaluate the political news and information that they were encountering. Furthermore, a coauthor and I came across a finding that led us to realize the importance of incorporating this topic into our classrooms: approximately one-third of the students surveyed in the Youth Participatory Politics Survey Project indicated that they had “never” had a class covering the trustworthiness of news and information that they find online or discussed this topic with an educator in the last six months (Cohen and Kahne 2016, 2018). These lessons are necessary for any young adult entering the electorate. The average adult encountered 1–3 stories which were “intentionally and verifiably false” in the 2016 election, but factors such as education could make an individual less likely to believe these (Allcott and Gentzkow 2017). If students are not prepared to evaluate the political news and information they come across, they may become disillusioned with the news coverage and potentially disengaged from political and civic life (Ceron 2015; Marchi 2012; Mihaildis and Viotty 2017). The development that led to the greatest change in my philosophy about teaching this course, however, was being asked to teach a new version of the course. This course was intended to be introductory and provide students with a survey of approaches—ranging from archival research to big data—to studying politics scientifically. The main goal of the course was to introduce students to these approaches so that they could answer important questions about politics and evaluate political information, rather than to teach students how to carry out the analyses needed to conduct research on their own. This course was a stark contrast to my typical experience of teaching upper-level Political Science majors in a computer lab, and led me to completely re-evaluate my approach to teaching research methods. My philosophy going forward would be to find a way to engage students in the course material regardless of their background or career goals, so that they could successfully become either consumer or producers of research.

46

C. N. KAUFMAN

Introducing Information Literacy When I began planning this new introductory course, I first considered my pedagogical training, and realized that I should design my lessons, activities, and assignments around the objectives for each topic in the class while also engaging the students using high-impact practices. Outside of the computer lab, we tackled additional questions that I soon realized were crucial topics. These included how statistics and research should be communicated to the public and how students could critically evaluate the information they encountered in the news or elsewhere in the future. I learned that best practices for teaching students to evaluate data and information, especially if they had been exposed to misinformation, included lessons that promote discussion, reflection, and real-life application; embrace students’ media preferences; and encourage critical thinking about the information (and misinformation) we may encounter (Bowyer and Kahne 2017; Cook et al. 2015; Crocco et al. 2018; Kahne and Bowyer 2016; Millet McCartney 2019; Mihaildis and Viotty 2017; Neundorf and Niemi 2016). Once I began teaching this new course, the realization that I would also need to teach this information in future upper-level methods courses came quickly. A former student who had previously taken the upper-level quantitative methods course enrolled and found that they were struggling with fundamental concepts. At first, I was highly surprised by this—after all, shouldn’t they have learned this information the first time? However, I then realized that the course had encouraged understanding of how to use a statistical software package and interpret the output, rather than a holistic understanding of what the results meant in terms of providing evidence for a hypothesis or establishing causality. How would this knowledge be beneficial in the future, where they were more likely to encounter a misleading chart that provided “evidence” in support of an argument on social media than to encounter SPSS output? A final reason to teach these skills is that they are valued by employers in several fields. Many employers are looking for graduates with skills in critical thinking, analytical reasoning, and the ability to find, organize, and evaluate information from many sources (AACU 2018). Teaching information literacy to students in our methods courses should help them to gain these skills. However, this does not mean that statistical analyses should be ignored, as over half are also looking for those who are able to work with numbers and statistics. In data analysis positions specifically,

3

SHOULD RESEARCH METHODS TEACH INFORMATION …

47

the vast majority of employers require a moderate level of skill, and over half require advanced skills (Society for Human Resource Management 2016). These positions require employees to be able to interpret and communicate data analysis results and use spreadsheets or simple analysis tools. Nearly half require skill with statistical software such as R or SPSS. A course that teaches students all these skills will allow them to pursue a variety of career opportunities.

Balancing the Competing Goals My current teaching philosophy is that we should be preparing students to be both consumers of and producers of research. This philosophy stems from the past experiences described so far, as well as discussions inspired by a presentation that I made on teaching research methods during an era of misinformation at an American Political Science Association Centennial Center workshop. Most recently, I applied this philosophy as the instructor for a relatively new research methods course at Wingate University. Wingate is a private, comprehensive university with an undergraduate enrollment of approximately 2,700. Our university’s Political Science major was first established in 2012.1 When I was brought into the department in 2018, they had just recently added research methods to the course catalog and made this a required course for the Political Science major. The first offering of the course was scheduled for Spring 2020.2 According to the course description, I needed to include both qualitative and quantitative methods to study politics scientifically. Otherwise, as the first instructor of the course, I had some freedom to structure its content. I knew that familiarity with statistical software and interpretation of various statistics would be important to any of our students planning to attend graduate school—but also realized that this was not the path for all of our students and wanted to ensure that I would also prepare students for other careers or civic life in general. I designed the new course based on our needs, our resources, and my own teaching philosophy. Additionally, to ensure that I was engaging students in the material and providing equitable access to those materials, I planned to include active learning exercises, discussions, and use resources available to students without additional costs (RStudio, an Open Educational Resource (OER) textbook, and other free online materials).

48

C. N. KAUFMAN

Although the two goals are not necessarily at odds, including both information literacy and statistics in a single course poses an obstacle. Most semesters are only about 15 weeks long, but many of us teaching this course could probably spend that entire semester on a topic we have been forced to compress into a single 50-minute lecture. Furthermore, I needed to cover both qualitative and quantitative analysis in my course. I was forced to think carefully and creatively about my priorities and how the course would be structured. My goals were to expose students briefly to several research methods, allow them to engage in the material, and help them to tie what they were doing to the creation or evaluation of research and political information. Cutting some content was necessary to achieve these goals efficiently and effectively. Ultimately, I decided that the main piece of content that I would cut was forcing students to learn detailed code. They would still be required to run code I had given them for RStudio, and make their own minor edits, but there were no assignments that focused primarily on the goal of having them understand how the code worked or requiring major manipulations of the data. I also tried to integrate earlier lessons into later ones to re-emphasize key themes. For example, for each of the methods we discussed the question of establishing causality—and whether that should in fact be the goal of the research process—was reviewed. I also combined some topics, such as simultaneously introducing the ideas of research design and research ethics by having students consider how they would ethically design research projects. Similarly, I taught students about evaluating data visualizations while also teaching them how to create them. There are several assignments and activities that I use in the course to achieve the goals of promoting information literacy while also teaching students the fundamental methodological tools they need to carry out research on their own. Here, I will highlight a few examples of assignments3 that led students to understand how to seek information on their own and how to evaluate information published by others. Students also completed more typical assignments focused on carrying out and interpreting the results of statistical analyses. The final project required students to propose a research project using any methodology we had discussed in the course and present this proposal to the class. This gave them the freedom to explore topics and methodologies that interested them while allowing for a comprehensive assessment of their ability to apply the information we had learned.

3

SHOULD RESEARCH METHODS TEACH INFORMATION …

49

The following assignments helped students not only understand how to find information on their own, but also how to evaluate political news and information: ● Reading examples of research using qualitative interviewing4 and then carrying out brief interviews on their own. ● Listening to a podcast in which the authors described the process of seeking out evidence through interviews and archival research and considered questions such as why it was necessary to seek out this evidence and whether it was sufficient to support the conclusions.5 ● Using an online tool provided by the National Archives to learn about the woman suffrage movement using primary sources.6 Students were specifically asked to sequence the documents, identify arguments made in the documents, and explain how those arguments evolved over time. ● Conducting a basic survey about American politics. Students proposed a survey experiment to conduct and additional questions to include, then distributed it to their friends and family on social media. We used the data for several assignments, explicitly comparing the results generated using our convenience sample to results generated using data from the 2018 Cooperative Congressional Election Study when possible. ● Reading about examples of misleading visualizations of political information as well as accounts of how information could be misinterpreted or misrepresented.7 They then applied the knowledge learned from these readings in their assignments on creating data visualizations. ● Finding an example of political research or original data (such as a survey) covered in the media and then comparing the media coverage to the original source, assuming it could be found. They then discussed whether they usually critically evaluate coverage of political news and information and what effect this may have on political knowledge and understanding in society. As an illustration of how these assignments development students’ information literacy skills and how these were assessed, consider the final example. This lesson was especially eye-opening for the students, as many of them realized that they did not usually do any critical evaluation of

50

C. N. KAUFMAN

the political news and information that they consume. Students were assessed on their ability to identify the research question, evaluate the evidence of the authors’ argument, locate the original source information, compare that to the media coverage, and reflect on the broader implications of poor information literacy for politics and society. Prior assignments involving collection of original data, building evidence-based arguments, and reviewing examples of misinterpreted or misrepresented information prepared them to take on this task and reflect on these points.

Discussion Instructors who would like to incorporate this content into their classes should consider that my approach requires substantial work. I could not use pre-packaged datasets and their corresponding assignments, which required me to design the assignments (including most of the code) from scratch and provide them with clean datasets; I had to design and program an original online survey; and I had to guide students through drafting their individual research proposals. This approach may not work for an instructor with less preparatory time or a larger class size. However, given that both information literacy and data analysis will prepare students not only for their careers but also to engage in political and civic life, I encourage instructors to consider including at least some content targeted at teaching both skills. I believe that my approach was mostly successful in meeting the goals that I had set. At the end of the course, I asked students to reflect on the most interesting thing they learned in the class and how it relates to information about politics they may encounter in the future. While not every student agreed that I made the course engaging and relevant in their situation (for example, a student in an anonymous evaluation said: “It’s only useful for those who want to go into research.”), several students commented that they would be more likely to critically evaluate news and information they saw in the media, online, or in their future courses. A partial comment by one of the students in response to this prompt perfectly captures what I wanted students to take away from the course: I definitely think my understanding of politics and information has changed for the better. I was excited to use my background in statistics and apply them to my major. Even though I am on the pre-law track and I wouldn’t be expected to use systems of data analysis like RStudio like a political

3

SHOULD RESEARCH METHODS TEACH INFORMATION …

51

scientist would, I learned to evaluate data and look deeper into how data and statistics are made. I think that this skill can help me so much in the future to not only be an informed citizen but also become the best corporate lawyer I can become.

In the future, I plan to continue to approach the course in a similar manner. Given that students were able to not only complete assignments, which required statistical analyses and interpretations but also cited their new critical thinking skills when asked what they had learned, most students appeared to come away from the course having achieved both goals. Additional improvements could be made to the course; however, especially considering that at least one student still felt it was only suited to training for conducting research. For example, I could allow students more agency in their assignments such as having more input in designing the survey, or choosing which archives, articles, or podcasts to review for an assignment. My goal is not only to have a course which aligns with my teaching philosophy, but which also makes the relevance of the course evident to the students and helps them to realize their own potential.

Notes 1. Ellis (2020) includes a discussion of the creation of our university’s Political Science major. 2. Minor changes to the course were made due to coronavirus pandemic, which led our university to cancel in-person instruction after March 13, 2020, with online instruction resuming March 23, 2020. Students had been following along with my instruction on the use of RStudio in a university computer lab. However, several students were then unable to access RStudio at home. I posted videos where I recorded my screen while I performed these analyses which students had to follow along with and interpret for their assignments. They were offered an extra credit opportunity if they could replicate the assignments in RStudio Cloud. 3. Kaufman (2020) includes additional assignment and activity examples that I have used in methods courses. 4. Students read an excerpt from The Politics of Resentment (Cramer 2016). Other examples using qualitative interviewing or participant observation as a methodology would be appropriate. 5. Students listed to the podcast “The Who and the What” from the NPR series White Lies (Beck and Brantley 2019). This series specifically investigates a murder, and other podcasts focusing on an investigation would be appropriate.

52

C. N. KAUFMAN

6. Students completed the activity “Extending Suffrage to Women: Finding a Sequence” (DOCSTeach). Other primary sources archived online would be appropriate. I have also previously taken students in-person to an archive to complete a similar activity. 7. Students were asked to read a chapter of How Charts Lie (Cairo 2019) and “The Real Story of 2016” (Silver 2016). Other “real life” examples of misleading, misrepresented, or misinterpreted political news and information would be appropriate.

References Allcott, H., and M. Gentzkow. 2017. Social Media and Fake News in the 2016 Election. Journal of Economic Perspectives 31 (2): 211–236. Association of American Colleges & Universities. 2018. Fulfilling the American Dream: Liberal Education and the Future of Work. https://www.aacu.org/ sites/default/files/files/LEAP/2018EmployerResearchReport.pdf. Beck Grace, A., and C. Brantley. 2019. The Who and the What. NPR White Lies, podcast audio, May 21. https://www.npr.org/2019/04/30/718661 791/the-who-and-the-what. Bowyer, B., and J. Kahne. 2017. Facing Facts in an Era of Political Polarization: Young People’s Learning and Knowledge about Economic Inequality. PS: Political Science and Politics, October, 1056–1061. Cairo, A. 2019. How Charts Lie: Getting Smarter About Visual Information. W.W. Norton & Company. Ceron, A. 2015. Internet, News, and Oolitical Trust: The Difference Between Social Media and Online Media Outlets. Journal of Computer-Mediated Communication 20: 487–503. Cohen, C.J., and J. Kahne. 2016. Youth Participatory Politics Survey Project. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2016–10–13. https://doi.org/10.3886/ICPSR36514.v1. Cohen, C.J., and J. Kahne. 2018. Youth Participatory Politics Survey Project, United States, 2013 and 2015 Panel Data. Ann Arbor, MI: Inter-university Consortium for Political and Social Research [distributor], 2018–12–03. https://doi.org/10.3886/ICPSR37188.v1. Cook, J., U. Ecker, and Lewandowsky. 2015. Misinformation and How to Correct It. In Emerging Trends in the Social and Behavioral Sciences: An Interdisciplinary, Searchable, and Linkable Resource, Robert Scott and Stephan Kosslyn, ed. John Wiley & Sons, Inc. Cramer, K. 2016. The Politics of Resentment: Rural Consciousness in Wisconsin and the Rise of Scott Walker. University of Chicago Press.

3

SHOULD RESEARCH METHODS TEACH INFORMATION …

53

Crocco, M., et al. 2018. Deliberating Public Policy Issues with Adolescents: Classroom Dynamics and Sociocultural Considerations. Democracy & Education 26 (1). DOCSTeach. 2020. Extending Suffrage to Women: Finding a Sequence. The National Archives, Accessed June 5. https://www.docsteach.org/activities/ student/extending-suffrage-to-women. Ellis, J.M. 2020. Ten Years In: Reflections on Creating a Political Science Major from Scratch. Presented at the American Political Science Association Teaching and Learning Conference, Albuquerque, NM. February 2020. Preprint: https://doi.org/10.33774/apsa-2020-8thq1. Kahne, J., and B. Bowyer. 2016. Educating for Democracy in a Partisan Age: Confronting the Challenges of Motivated Reasoning and Misinformation. American Educational Research Journal 54 (1): 3–34. Kaufman, C.N. 2020. Civic Education in a Fake News Era: Lessons for the Methods Classroom. Journal of Political Science Education. https://doi.org/ 10.1080/15512169.2020.1764366. Marchi, Regina. 2012. With Facebook, Blogs, and Fake News, Teens Reject Journalistic “Objectivity.” Journal of Communication Inquiry 36 (3): 246– 262. Mihailidis, P., and S. Viotty. 2017. Spreadable Spectacle in Digital Culture: Civic Expression, Fake News, and the Role of Media Literacies in “Post-Fact” Society. American Behavioral Scientist 61 (4): 441–454. Millet McCartney, A.R. 2019. The Rise of Populism and Teaching for Democracy: Our Professional Obligations. European Political Science October. https://doi.org/10.1057/s41304-019-00225-1. Neundorf, A., R.G. Niemi, and K. Smets. 2016. The Compensation Effect of Civic Education on Political Engagement: How Civics Classes Make Up for Missing Parental Socialization. Political Behavior 38: 921–949. Silver, N. 2017. The Real Story of 2016. FiveThirtyEight. January 19. https:// fivethirtyeight.com/features/the-real-story-of-2016/. Society for Human Resource Management. 2016. Jobs of the Future: Data Analysis Skills. Research and Surveys. November 4. https://www.shrm.org/ hr-today/trends-and-forecasting/research-and-surveys/pages/data-analysisskills.aspx.

CHAPTER 4

The Politics of Identity and Teaching Information Literacy in Political Science Jennifer Epley Sanders

My husband jokingly calls me “Google Monster” because of how often I use the search engine. My colleagues and students are also not surprised whenever I send them follow-up information after a discussion or meeting. A student once asked me in class, “Do you like to read?” Her classmate replied before I did: “Of course she likes to read. That’s all she does. She’s a professor.” I share so much information capital with students that I earned the moniker “The Queen of Handouts.” Because of my reputation of being open and honest with information, my email inbox, office hours, and social media accounts include communications with diverse groups, including those who have never taken a class with me. What can I say? I truly appreciate data because information can be empowering.

J. Epley Sanders (B) Department of Political Science, Social Sciences, Texas A&M University-Corpus Christi, Corpus Christi, TX, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_4

55

56

J. EPLEY SANDERS

For context, I teach undergraduate political science classes in the core curriculum and the major at a medium-sized public regional university in Texas that has the federal designation of a Hispanic-Serving Institution. Most of my students are first-generation college students and from working-poor, working-class, or middle-class families. I teach “non-traditional” students, too, in terms of age, childcare and eldercare responsibilities, and/or status as active military or veteran students. Almost all of my students have a part-time or full-time job while pursuing a full course load. Fold in a rich and varied set of racial, ethnic, cultural, gender, religious, and nationality backgrounds along with the unique features and politics of South Texas, teaching and learning political science are interesting and challenging for all involved. It helps that I personally relate to and empathize with my students on certain identity issues. I myself was a first-generation college student, experienced poverty, know the ups and downs of a military family, have a mixed religious background, held part-time jobs throughout college, lived and worked overseas, engaged in eldercare for my mother who had early Alzheimer’s, and am married with a child. In addition, I recognize the beauty, complications, and politics of race, ethnicity, culture, gender, and nationality in the U.S. and abroad as a woman who is biracial and multicultural. Empathy with my students early in my career led me to think critically about my teaching pedagogy and continues to inform my practices. With the politics of identity in mind and practice for my students and myself, my teaching pedagogy evolved to increasingly address “identity” and incorporate “what’s missing,” meaning what are students not learning or getting elsewhere that I can provide to facilitate their professional and personal development. What was frequently missing from my own educational and career paths and is commonly missing from my students’ prior exposure or training is information literacy. I find this omission to be intentional and unintentional depending on my students’ past experiences with the politics of identity concerning family, cultural, or social norms; influences from domestic and international politics; and students’ former circumstances in the K-12 system and college. For the latter, much depends on who taught them what content, how, why, and under what conditions. My students report wide variation in public or private school settings, amounts and types of individual and group resources, homogeneous or heterogeneous communities, historical legacies, and current events. Although I cannot address all of the complexities for the politics of identity and all facets of information literacy in a single semester, I can

4

THE POLITICS OF IDENTITY AND TEACHING INFORMATION …

57

equip my students with some basic tools to improve how they navigate the study of politics and the doing of politics. To assist students in offsetting the effects of prejudice, stereotypes, and discrimination and help them better address identity-related issues from a strength-based position, I incorporate “information literacy” into my undergraduate courses. “Information literacy” refers to discerning different types of information and their corresponding value; multiple tools for seeking, evaluating, using, creating, and communicating information; and knowledge about the politics of information. For students and faculty needing a formal detailed outline for information literacy, the Association of College and Research Libraries provides a useful, flexible framework that consists of six interconnected concepts: ● ● ● ● ● ●

Authority Is Constructed and Contextual Information Creation as a Process Information Has Value Research as Inquiry Scholarship as Conversation Searching as Strategic Exploration (ACRL 2015)

Each of these concepts has an associated set of knowledge practices and dispositions as well.1 My lower-level core curriculum courses focus on U.S. government and politics every semester, whereas my upper-level major courses alternate semesters between Comparative Politics, International Relations, and Religion and Politics. Our undergraduate political science program requires one semester of research methods concentrating on statistics. I rarely teach that course, so I embed research methods, specifically information literacy, into the courses I do teach. Interestingly, my upper-level students often comment that they initially signed up for an international class but ended up “really” being in a research methods class. My teaching pedagogy for those courses is profoundly shaped by my value system and empathy arising from past experiences as a student. The logistics of my instructional and assessment strategies are based on a combination of personal experience, formal training, social science research, and lessons from my students over the years. During my first couple years of teaching, I followed the advice of senior faculty and modeled a lot of my teaching pedagogy on what I

58

J. EPLEY SANDERS

observed other faculty at my institution were doing. I noticed, however, that there was lots of room for improvement because the politics of identity were deeply affecting students’ perceptions of me, the course content and assessments, themselves, and other students. I started researching best practices in earnest around my third year of teaching. I sought out informal mentorships with women faculty and faculty of color at my university and during conferences. I attended professional development workshops and began reviewing literature on the Scholarship of Teaching and Learning and teaching-related social science research. Over time, I figured out how to adopt features of “mainstream” teaching pedagogies where appropriate, but also shift away from such thinking and practices when the politics of identity call me to do so. For example, I emphasize the “science” in political science using Robert K. Merton’s “ethos of science” (i.e., universalism, “communism,” disinterestedness, and organized skepticism) to facilitate critical thinking and manage class discussions. I have written previously on how this approach has been successful for student development and class management, especially given my identity as a woman of color and the need to legitimize my authority and expertise through a more powerful, mainstream ideology (Merton 1973; Epley 2015). At the same time, I incorporate a “what’s missing” and “not that way” sort of philosophy into my teaching pedagogy after learning what not to do as a teacher during my studies and career. My attempts to balance “mainstream” and “unconventional” teaching approaches are informed by old memories. As early as elementary school, I learned about the detriments of ignoring aspects of a student’s identity, receiving mixed messages about merit and worth, and the domino effects of unequal treatment. I vividly remember working on a family tree in which the teacher actively encouraged me to only focus on my British and German heritage, completely disregarding my Asian and Pacific Islander family. As a result, I missed out on learning about my family as a whole, diverse cultures, and multiple historical narratives. That would continue to be the case with incomplete and inaccurate textbooks or syllabi and educator biases in future classes as a student. As a professor today, I agree with the mainstream tradition of discussing and learning about popular topics like families, but I incorporate an unconventional twist by being much more inclusive of identity and weaving in information literacy. At the beginning of the semester, my students engage in a powerful activity that I call “The History of

4

THE POLITICS OF IDENTITY AND TEACHING INFORMATION …

59

Your Name.” Small groups discuss the origins and meanings of their names and then report to the whole class. I share my name’s backstory, too. Inevitably there is a rich, complex history behind everyone’s names involving deep-rooted and contemporary politics. There is laughter, confusion, and a-ha moments in class, and numerous students have told me that they go on to have interesting conversations with their family members later. Besides getting to know each other, my students quickly learn that even if they initially think “Politics is not my thing,” they cannot avoid it because by virtue of being named, one is a political creature. Moreover, starting a semester with this kind of activity introduces my students early on to information literacy. For instance, how do they know the history of their names? Did their parents tell them, or did they just quickly look their names up on their phone? If a student does not know how or why they were named, is there a reason for that lack of information? What quality resources can we use to find out why names were changed for or by immigrants during different historical periods? What research exists regarding marital name changes? How do we find out the legal processes for changing birth names? Which government and non-profit websites provide the most clarity on “authorized” names and identification? Related inquiries delve into the politics of information for genealogy or genetic businesses like Ancestry.com and 23andMe.com. Another old memory that informs my teaching pedagogy is from my high school days. There is the general mainstream belief in the U.S. that everyone should learn math, but stereotypes about how not all people can learn it well. As a teenager, I recall disagreeing with the way my pre-calculus teacher taught our class. He physically separated students’ desks by gender, commented that girls were not as good as boys at math, and ignored girls who raised their hands to answer questions. I knew deep down that this teacher was biased, but I did not have the language or status to lodge a complaint. Unfortunately, I internalized his ideas and lacked confidence in my math skills during college. I thought I had moved beyond such insecurities and bias problems in graduate school only to have a statistics professor reinforce similar gender stereotypes. Not only were those individuals intentionally withholding valuable educational information from girls and women, but they were also essentially teaching us that we were not able to understand specialized and advanced information or deserving of its access and use. I remember repeatedly thinking to myself that I would be generous with information and that I would

60

J. EPLEY SANDERS

not automatically stereotype a student’s aptitude, skills, talents, values, or work ethic based on gender if I ever taught one day. Consequently, while I incorporate what is considered to be mainstream math and statistics into my classes, I again take an unconventional approach by adding in the politics of identity and information literacy. Before working on replication problem sets, for instance, our class discusses their previous experiences with the intersections of identity and math. Who has math anxiety and why? Who is “good” at math but does not want to disclose their skills or perhaps does not know they are actually good at it? Who in fact does need some math basics or a math refresher? What existing critical thinking, study, and technology skills can we take advantage of to improve our math skills? How can we as a class assist each other in answering math problems? What resources are out there to help a student “catch up” or problem-solve an issue during the semester for an assignment? We then move on to discussing preconceived notions about information and numbers. Students typically think, “Numbers don’t lie,” but, oh, how they can! For example, how did an author create a quantitative table and what messages are they trying to convey and to whom? Where did the author get their original source information for their analysis? Why did the author use that data and not data from another source? How should we or other audiences evaluate that author’s information? What biases, if any, are present in the information provided? What is the value-added or a potential conflict related to the author’s identity for their research? In this way, I connect the politics of identity at the individual level (e.g., the student or the author) and group level (e.g., different social, professional, and cultural communities) with information literacy in order to teach my topical courses productively. Yet another old memory from college guides how I generally teach about definitions, concepts, and measures and how information literacy is tied to those processes. When I arrived at college, I learned the term “Affirmative Action” for the first time from other students who used it as a pejorative. They strongly disagreed with my presence at “their” elite institution. I eventually asked one of my political science professors about it and he fortunately helped me unravel the politics of the term and the resulting negative treatment by my peers and even a few professors. I learned what was missing from my prior education, in this case specialized vocabulary and how to find out their meanings, along with what not to

4

THE POLITICS OF IDENTITY AND TEACHING INFORMATION …

61

do as a classmate or teacher, based on how I was treated and felt under those circumstances. Therefore, rather than assume students’ familiarity with and proficient use of “common knowledge” and “fancy” discipline-specific languages, theories, and methods, my teaching pedagogy integrates principles of “universal access to education” where equal opportunity is simultaneously a goal and teaching approach. It was around my fifth and sixth years of teaching that I got much more in-depth learning about “universal access” after numerous discussions with staff from Disability Services at my university, participation in workshops, compiling lessons from my students, and peer reviews. “Universal access” requires being intentional in the planning and implementation of a course structure, processes, and content, as well as being open to making revisions from the recommendations of students, faculty, and staff. In practice, I now utilize guidelines and checklists that are freely available online related to Universal Design of Instruction (UDI) (Burgstahler 2020) and Universal Design for Learning (UDL) (CAST 2018) to build my syllabi, organize my courses, select content, create assessments, guide learning activities, etc. This type of pedagogy attempts to minimize barriers to learning and maximize learning for students from all backgrounds. There are also adaptations for face-to-face, hybrid, and fully online classes. When I began making UDI and UDL changes to my courses, I thought students might perceive me to be confusing, condescending, overthinking, or wasting their time, but my students have regularly told me that they appreciate my efforts. They report a need and desire for clarity, context, the exact how-tos, accessibility, and resources from start to finish because they did not learn that information before taking my class or they want to utilize that information in advanced ways. When students do express concerns about not having adequate formal preparation or frustrations during the learning process, it is a chance for me as an instructor to be honest about how the politics of identity affect professor–student dynamics. I give them details about who I am and what I know and do not know. I share my knowledge and expertise whenever possible, but I am aware of potential limitations. I tell my students that this is where partnership enters into the picture. Students come from a range of backgrounds and have their own skill sets, which may be shared with others. As partners, we can leverage our identities together to supplement or change what we know and how we know about the world around us. While admittedly there is a threshold to the collaborative dynamic

62

J. EPLEY SANDERS

because as the professor I am ultimately in control of such matters as the syllabus, examinations, and grading, I still feel that students have something to offer and that they can be good teachers, too. This is further supported by classroom dialogue guidelines. I offer a set of eight rules that incorporate respect, confidentiality, trust, and security. Students work together on revising the rules and compromise on the final version. For example, two of the rules state, “Challenge the idea and not the person” and “Speak your discomfort.” One popular additional clause for these two existing rules is “Do not interrupt someone who is speaking.” This kind of class setup was missing from my education even as teachers talked about democracies, the marketplace of ideas, and fair playing fields. The guidelines permit students to openly ask and answer questions and give or defend their opinions, which allow for critical thinking, information literacy, and communication skills to develop without too much fear and negativity from other students or the instructor. The guidelines structure student–student and instructor–student engagement in a professional manner and reduce negative perceptions and behaviors linked to identities. The list below provides further logistical examples of how “universal access to education” is implemented in my classes, which sets up teaching about the politics of identity and information literacy more efficiently and effectively: ● Syllabus: Emphasis on clarity, details, and explaining terms and steps that may be unfamiliar. In my early days of teaching, my syllabi were less than eight pages, but now they are usually approaching fifteen pages, single-spaced, and include a detailed course schedule, explanations about responsibilities for all parties involved, extra handouts about assessments, dialogue guidelines, university policies, and a syllabus contract. The latter has to be signed and submitted before the start of the semester. The syllabi are a great lesson on cause and effect, contract law, reading the fine print, and professionalism. The lengthier syllabi function like a glossary and starter workbook or manual for students for their benefit and mine. I refer to specific components of the syllabi regularly during the semester as well. ● Course structure: I model my upper-level classes on think tanks where teaching and learning are inclusive, collaborative, and applied instead of amplifying individualism or separation. Students frequently work in pairs or threes, and then those small groups work

4

THE POLITICS OF IDENTITY AND TEACHING INFORMATION …

63

in slightly larger groups, and then the whole class works together. I encourage everyone to share resources but submit their own original work. I support students in crowdsourcing useful tips and tricks for reading, writing, research methods, technology, presentations, etc. via wiki-documents in Blackboard (a learning management system), too. ● Open Educational Resources: During the past few years, I have gradually shifted to using free and freely licensed educational materials when they are available to help students’ budgets and accessibility. Since last year in particular when textbook costs were astronomically high, I actively teach students how to locate and use credible and accurate open educational resources for content learning and their assessments. We work together on identifying when and how such resources are evidence-based versus persuasive or only opinions, demonstrate critical thinking or logical fallacies, and show suitability and precision for a given problem of interest. ● Course Content: I find that students appreciate a foundation or framework that they can work from and with over the entire semester. For my classes, I highlight what I regard as key political science questions on their own and ones that incorporate the politics of identity and information literacy. Embedded in all six of the following questions is “Why?”: – What is the “puzzle?” (e.g., conflict, confusion, concern, surprise, etc.) – What do we think is happening versus what do we know is happening? – Who are the actors? (e.g., Who is present and who is missing from a narrative, who has a political voice, and who “counts?”) – What are their interests? (i.e., What do they need and/or want?) – How do they get their interests? (e.g., legally/illegally, ethically/unethically, morally/immorally, voting, petitions, protests, lawyers and court cases, boycotts, force, etc.) – What are the consequences of them obtaining their interests? (e.g., “winners” and “losers” in a political competition) ● Assessments: It took me more than six years, but I have now changed most of my individual and group assessments to “open book, open notes, and open internet.” Contrary to popular belief or expectation, my students learn more and perform better, though not perfectly

64

J. EPLEY SANDERS

because the assignments are difficult. Cheating might still occur, but I do not have the same problems as I did in my early days of teaching. Because I encourage and make time for lots of class debriefings, peer reviews of drafts, tutor time, and self- and peerevaluations after submissions, I think it is harder to cheat and there is more social pressure to be truthful and put in one’s best efforts. I give class feedback memos in addition to individual comments to foster a more collaborative learning and assessment environment as well. ● Handouts: Emphasis on reaching students from a wide variety of backgrounds and preparations. The handouts may be for the course or resources available at the university and in the community. Hard copy and digital handouts are an easy and reliable way to connect what students are learning to their current and future jobs, too. ● Office hours: Emphasis on accessibility, availability, and productivity. During in-person discussions, phone calls, online chats, video calls, and email, my students and I mix professional and personal development topics tied to the course content, their current and future goals, the politics of identity, and information literacy. Throughout my upbringing and schooling, there were countless times when I missed out on important information or was given incorrect or incomplete information. Things were difficult because of the politics of identity and the politics of information. When certain teachers, schools, and communities believe that some groups of children and adults deserve more and the best of everything over other groups, they give their time, energy, and resources to the favored “accepted” groups and hinder the lives, work, and progress of any “outsiders.” Each time I experienced that unfairness or observed it for my family and friends, I made a mental note. I discovered over time that I did not have to repeat what previous teachers did; I could leverage my differences and choose a different paradigm in which to be and teach. Combining mainstream and unconventional pedagogical approaches to teaching information literacy while incorporating the politics of identity in the process helps my students “work smarter, not harder”; gives them applied practice with course content and professional skills; exposes them to different types of identities, leadership styles, and teamwork dynamics; and offers psychological and emotional support when challenges arise. For me, one of the “best practices” for teaching

4

THE POLITICS OF IDENTITY AND TEACHING INFORMATION …

65

with empathy involves asking the tough questions of “What is missing?” and “What should I not do?” before, during, and after a course.

Note 1. According to the ACRL, “The Framework opens the way for librarians, faculty, and other institutional partners to redesign instruction sessions, assignments, courses, and even curricula; to connect information literacy with student success initiatives; to collaborate on pedagogical research and involve students themselves in that research; and to create wider conversations about student learning, the scholarship of teaching and learning, and the assessment of learning on local campuses and beyond” (Introduction, para. 5).

References Association of College and Research Libraries. 2015. Framework for Information Literacy for Higher Education. American Library Association. Accessed February 2, 2015. http://www.ala.org/acrl/standards/ilframework. Burgstahler, S. 2020. Equal Access: Universal Design of Instruction, DO-IT (Disabilities, Opportunities, Internetworking, and Technology) Center, The Center for Universal Design in Education, University of Washington. https:// www.washington.edu/doit/equal-access-universal-design-instruction. CAST. 2018. Universal Design for Learning Guidelines, Version 2.2. http://udl guidelines.cast.org. Epley, J. 2015. Emphasizing the ‘Science’ in Political Science: Reflections from a Multiracial Woman Professor. Professional Resource Center—International Studies Association. http://www.isanet.org/Programs/PRC/Articles/ ID/4856/. Merton, R.K. 1973. The Sociology of Science: Theoretical and Empirical Investigations, 267–278. Chicago: The University of Chicago Press.

CHAPTER 5

Using K12 Foundations to Teach Scientific Literacy in College Research Methods Kristina M. W. Mitchell

In April 2013, the final draft of the Next Generation Science Standards (NGSS) was released. The goal of creating a new set of standards was to ensure that K-12 students in the United States left their school experience with a working knowledge and greater interest in science. In particular, the framework seeks to help students understand the scientific process and method so that they can evaluate scientific evidence in college and in their careers and lives as citizens. Since then, 40 states have adopted the standards and use curriculum that supports students in achieving them. I didn’t know anything about the NGSS standards when I started teaching political science research methods in 2013. After finishing graduate school in 2012, I started teaching at Texas Tech University as a Visiting Assistant Professor, and I was assigned the daunting task of teaching undergraduate research methods. Like many political science PhDs, I taught my first research methods course as

K. M. W. Mitchell (B) Department of Political Science, San Jose State University, San Jose, CA, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_5

67

68

K. M. W. MITCHELL

though I were training budding political scientists. I drilled them with formulas and Stata commands, we memorized Type I and Type II errors, and we learned the notation for null and alternative hypothesis. The class was dry and technical, but successful. Or so I thought. In 2018, I left Texas Tech for private industry. I took a job working for a publishing company that primarily sells science curriculum to K-12 schools and school districts, and I was suddenly thrust into the world of the NGSS. I was in front of rooms of middle school science teachers trying to demonstrate a lab experiment with methylene blue and microscope slides, all the while thinking, this is not what I was trained to do. But I also continued to teach research methods, now as a Lecturer at San Jose State University in Silicon Valley. The more time I spend in a world juxtaposed between middle school science curriculum and political science undergraduate research methods, the more I realize that the biggest issue with my teaching style was that my research methods course bore no connection to what the students were learning under the new K12 science standards before they got to college. And perhaps even worse, my course had no connection to their careers and lives as informed citizens and scientific thinkers once they graduated as it turns out, I wasn’t trained to teach scientific research methods to undergraduates either. As political science graduate students, we delve into the literature on political behavior and institutions, we put ourselves into bins of American politics, comparative politics, or international relations, and we learn complex statistical methodology to answer esoteric questions like, “Why do developing countries participate in the WTO Dispute Settlement Mechanism at lower rates?” I did take a philosophy of science course in my graduate work, but I was taught no pedagogical theory (Manzo and Mitchell 2018). I was given many tools for answering research questions with multiple variables, but no tools for teaching undergraduate students. Like most of us, I was given zero training or preparation before I taught as a graduate teaching assistant. The first course I taught was Game Theory. It was a disaster. Oh, I was able to teach the mechanics of mixed strategy Nash equilibrium and evolutionary stable strategies, but had no framework to tie these skills to anything they might need in their other classes or after college. I had plenty of tools for formal modeling, so I passed these tools along to my students. But they didn’t need them. When we are thrust into research methods classrooms, we often fail to recognize the fact that most of our undergraduate political science

5

USING K12 FOUNDATIONS …

69

majors are not planning to go on to graduate school, nor will they work in careers that truly require them to act as political scientists. My first job after my undergraduate degree was selling pharmaceutical products for hemophiliacs, not exactly something that required any knowledge of regression analysis or hypothesis testing. The majority of our political science students leave our programs with no need for the game theory or the statistical tools I was teaching them. I was failing the students who were leaving my classroom at the end of the semester without any skills that were useful. But I was failing the students that were entering my classroom at the beginning of the semester, too! Students came into my classroom with a vague idea that their major was called “Political Science,” instead of just “Politics,” but they saw no real connection between their political science classes (which were all mostly substantive up to that point) and the “real” science classes they’ve been taking since kindergarten. Science, they had been told, was about doing lab experiments and wafting fumes from test tubes and dissecting frogs. Politics was something entirely different. Upon having this realization, I decided to revamp my research methods classroom completely, taking the tools I was learning from my work in K-12 science education and reusing them in my political science curriculum. I wanted to help my students connect the basic NGSS science concepts from their K-12 education to their political science coursework and to the world more broadly: to help them realize that science goes beyond test tubes and frogs. Science, I wanted to show them, is a process and a way of understanding the world, and it can be used to study all sorts of questions, including political ones.

The Next Generation Science Standards Created in 2013, the Next Generation Science Standards (NGSS) were intended to formalize science education across the country, and to encourage students to think about science as a way to question and investigate the world around them. Instead of memorizing facts about Newton’s laws or frog biology, the NGSS encourages students to think about science as a process. As of 2019, 40 states have either adopted NGSS formally, or created standards based on the NGSS framework (Thompson 2019). Most of us who are teaching political science research methods aren’t well versed in the kind of science education that our students have experienced in K-12, but tying our coursework to this

70

K. M. W. MITCHELL

framework might help our students understand the “science” component of their political science degree. The NGSS standards center on three pillars: disciplinary core ideas, cross-cutting concepts, and science & engineering practices.

Disciplinary Core Ideas (DCIs) Disciplinary Core Ideas are “the fundamental ideas that are necessary for understanding a given science discipline” (Next Gen Science 2020). We begin with the idea that each discipline in science has its own set of principles, facts, and findings that guide investigations. In life sciences, DCIs focus on structures and processes (molecules, organisms), heredity and variation of traits, and evolution. Earth and Space Science DCIs include the systems on planet Earth and Earth’s place in the universe. But DCIs are found in all scientific disciplines, including social sciences. DCIs in political science might be the processes of democratization, the causes of international conflict and cooperation, and the explanations for voter behavior. There is a set of findings in the field that we use to start our investigations, and we can build on or challenge the consensus in the DCIs.

Cross-Cutting Concepts (CCCs) Crosscutting Concepts “provide students with connections and intellectual tools that are related across the differing areas of disciplinary content” (Next Gen Science 2020). These concepts can help students enrich their understanding of the disciplinary core ideas and broaden their ability to see connections across disciplines. In short, CCCs show students that all disciplines use many of the same concepts to explain phenomena. The NGSS specifies seven concepts that apply across all disciplines of science. These concepts are: 1. Patterns 2. Cause & effect: mechanism & explanation 3. Scale, proportion, & quantity 4. Systems & system models 5. Energy & matter 6. Structure & function 7. Stability & change

5

USING K12 FOUNDATIONS …

71

While at first, some of the CCCs may not seem to be applicable to political science (we don’t spend much time discussion conservation of matter), in truth, most of these concepts can be applied to the social sciences as well, and making these connections explicit can introduce students to the idea that politics can be studied in a scientific way. After all, sometimes it seems like 90% of political science is identifying causal mechanisms and providing explanations. Framing the study of politics by using CCCs like patterns, systems, and measurement can give students a touchstone for how to think scientifically about political questions.

Science & Engineering Practices (SEPs) The Science & Engineering Practices are the final pillar of the NGSS, and likely the one that we are most familiar with in teaching our introductory research methods courses. SEPs are taught because educators “must take into account that students cannot fully understand scientific and engineering ideas without engaging in the practices of inquiry and the discourses by which such ideas are developed and refined” (Next Gen Science 2020). Essentially, it is not enough to teach students about science, we must teach them to do science. The eight practices as defined by the NGSS framework read very much like the scientific method that we all learned for the Science Fair in 3rd grade: 1. Asking questions 2. Developing & using models 3. Planning & carrying out investigations 4. Analyzing & interpreting data 5. Using mathematics & computational thinking 6. Construction explanations 7. Engaging in argument from evidence 8. Obtaining, evaluating, & communicating information In retrospect, my early attempts at teaching political science research methods focused only on the SEPs. I dived in on Day One with asking testable research questions (still a skill I would love my students to master), I taught data collection, and then we delved into descriptive statistics, correlation coefficients, and regression. Finally, students

72

K. M. W. MITCHELL

wrote research papers that explained their research findings, supported by evidence. Fantastic! I have been teaching SEPs all along! But the key takeaway from the NGSS framework is that teaching SEPs alone is not sufficient. Without context, provided in the form of CCCs and DCIs, the practices we teach our students are deemed unimportant and soon forgotten.

A Revamped Version of Political Science Research Methods So, how could I take my newfound knowledge and understanding of the NGSS framework to create an undergraduate research methods course that both (1) tied itself to their previous coursework in science, and (2) provided them with information and skills that would be useful to them as scientific thinkers in the real world? I decided to break my course down so that it matched the three pillars of the NGSS framework: in the first segment of the course, I focus on DCIs, asking my students to consider, “How do we know what we know about political science?” In the second, I use datasets and broad patterns of data interpretation to help them see the CCCs and answer, “What broad concepts help us confirm what we know?” And finally, in the third, we focus on the scientific method by considering, “How can the scientific method help us understand what we observe in the political world?”. This might not seem like a dramatic shift from what you already teach in your research methods class, but the key difference is that I let go of the idea that I was training students to be political scientists. Instead, I was focusing on broadening their understanding of science. My students had learned how to use science, through DCIs and CCCs and SEPS, through the lens of “traditional” scientific fields like chemistry or biology in their K-12 education. Now I could teach them to apply those skills to other topics, like politics. Or the economy. Or psychology. Or marketing. Or whatever career they chose to take on after their undergraduate degree. I focused on teaching them to think about the world as scientists and ensuring they are equipped with the practices to do so. So what, in practice, does this format look like? And how is it different than how I taught before I learned about the NGSS framework?

5

USING K12 FOUNDATIONS …

73

Segment One: The DCIs The first segment of my research methods course, I focus on DCIs: the core ideas in political science. I frame the course with the question, “How do we know what we know about political science?” I start with a broad discussion of how we know anything. What does it mean to know something? Particularly in recent years, the emergence of fake news, misinformation, and conspiracy theory has made it critical to begin with a discussion of where core ideas come from, and what can be considered reliable. We discuss the difference between empirical and normative, and how to look for empirical evidence before forming opinions. Throughout the first segment of the course, I focus on the combination of philosophy of science and the importance of literature review. In teaching students the answer to the question, “How do we know what we know?” Starting with the framework of what it means to “know” something helps us establish that there are multiple ways of knowing: ways that aren’t scientific. While faith, for example, can be an important part of a student’s life, and an important way to know something about the world and their place in it, in a scientific setting, we must set aside this way of knowing. Distinguishing scientific knowledge from other types of knowledge (like faith or philosophy) is key in getting students to think like scientists. At this point, I emphasize the characteristics of scientific knowledge, and make the distinction between science and pseudoscience. Pseudoscientific knowledge can be a treacherous way of knowing: it mimics reason and empirical investigation, but presents false or incomplete conclusions. One activity that has been highly popular to conclude this segment is one on Astrology. Students often know their astrological sign, and there is a body of “knowledge” surrounding the characteristics of individuals born under each sign. Is this a form of knowing? First, we test the characteristics of each star sign: students view a list of traits and assign traits to themselves that they think best fit their own personalities. We compare the traits they have self-assigned to those that their star sign predicts. Unsurprisingly, the traits do not match what their sign predicts. We also look at horoscopes from different sources to see if there are major differences in what different astrologers predict. There nearly always is! This helps frame the idea that not all forms of “knowing” are the same. We discuss the ways in which the “knowing” that comes from Astrology is

74

K. M. W. MITCHELL

not scientific. Astrologers are often wrong, and when their predictions fail, there is little for them to do when they fail other than explain failure away. Scientific knowledge, on the other hand, is characterized by learning from failure. When scientific predictions fail, we can use that failure to improve our explanations or predictions next time. In addition, successful scientific predictions can be replicated, where pseudoscientific findings cannot. “What’s the harm in astrology, though?” my students often ask. “It’s just for fun!” This is where we return to the concept of DCIs. If we are looking for foundational ideas that make up the body of what we know, it is key that those foundations are solid. Once we accept pseudoscience as an acceptable way of knowing, it can quickly move from a belief in astrology to a belief in anti-vaccination conspiracies, which present a material harm to the population. In science, our disciplinary core ideas are based exclusively on scientific knowledge, and the students leave this segment of the course with a good understanding of what that means. Segment Two: The CCCs In the second segment, we begin to examine the ways in which science generates those explanations and predictions. I help them identify the elements of scientific research that we use in political science by asking, “What broad concepts help us confirm what we know?” This is the segment in which I introduce the ideas of data, measurement, and variables. Considering that our students often enter their classroom perceiving physics and biology as “real” science, this is the opportunity to show them that all disciplines of science use the same concepts to explore ideas and come to conclusions. We start with definitions and vocabulary. To give students a basic understanding of what they’re about to encounter, I define dependent and independent variable, causation, and measurement. But, I found that my students often can’t really make the connections between the definitions and the operation until they see data in action. Thus, the activity that drives this segment of the course is having students build and manipulate their own dataset. While physicists might build datasets on the movements of planetary bodies,1 we build datasets on social science topics. I require them to create a dataset with four variables and at least 25 observations in Excel, using straightforward and easy-to-find data like sex education and teen pregnancy rates or literacy

5

USING K12 FOUNDATIONS …

75

rate and poverty rate. Since we have already covered ways of knowing and disciplinary core ideas at this point, students can tie those ideas to data collection: they can review literature for ideas for topics and know a reputable source for their data. Building a dataset in Excel (which students can download for free through the university’s website) also helps them generate some basic workplace skills, as familiarity with Excel and Microsoft Office is most definitely a marketable skill in any career. My students often came to my classroom without even knowing how to enter data into a cell (or even what a cell was), so learning to add, subtract, and manipulate data provides a huge value to a political science research methods class that may not otherwise seem useful to students who aren’t pursuing a career as political scientists. After collecting and reviewing the data, we go through calculating the descriptive statistics of each variable and talk about the ways in which each variable has been measured. They can go back a few pages in their notes to find the definitions of dependent and independent variables, or nominal, ordinal, interval, and ratio measurement, and then apply those broad concepts to the dataset in front of them. We also go through what it would mean for one of their variables to cause a change in another, to revisit the ideas of causation versus correlation. We look for patterns in the data, and think about systems and what variables in the system might be missing from their dataset. Finally, we discuss the limitations of social science research when compared to laboratory science research. Measurement is difficult when you’re looking at humans and human behavior! But this difficulty does not mean that political science isn’t a science. In fact, it means that political scientists must be that much better at their research design strategies to overcome the challenges. Overall, students conclude this segment with an understanding of the concepts that connect all disciplines of science, and how those concepts are used in social science research. Segment Three: The SEPs Finally, I focus on the SEPs and frame my final segment of the course with the question, “How can the scientific method help us understand what we observe in the political world?” In this segment, I teach students to ask a testable research question and then test their hypothesis.

76

K. M. W. MITCHELL

This may seem very backward from the way research methods in political science is often taught. In my early years of teaching this course, I nearly always led with research questions, feeling that students couldn’t understand data or even literature review without understanding how to ask a research question first. Over time, I saw that my students were struggling to make the connection: they learned the characteristics of a research question, but without the framework of understanding what data is, they couldn’t figure out what testable meant. Placing the scientific method in their hands after they learned about DCIs and CCCs helps them situate their knowledge. Now that students have a good understanding of what scientific knowledge is, and they are familiar enough with their datasets to understand what they have measured and how, we focus on what research questions we can answer with the data we have. This is key! Of course, while it would be wonderful if all political scientists could get tens of thousands of dollars in grants to gather all of the data we want, we are very often limited to using data that already exists. I tell students this! In an ideal world, we would create our research questions first, based on theory, and then gather the data that helped us answer it, but in the real world, we must often decide a broad topic that interests us, and find the data that already exists to help us craft our research questions. We spend time discussing what makes a good research question, revisiting the idea of empirical versus normative and emphasizing that we don’t want to go into our dataset hoping for one particular answer. We propose a hypothesis and examine causal mechanisms, and then we delve into the scientific practice of testing our hypotheses. Students feel much more comfortable thinking about using statistics to answer their questions after having the chance to work with their datasets on their own. I teach the scientific method, and we go through each step, from asking a question to analyzing data to engaging in argument from the evidence. In this segment of the course, I have taught correlation coefficients, ANOVA, and OLS regressions. My recommendation is to focus on one of these methods, as teaching all of them tends to just get confusing for students. My most successful semesters have focused on correlation coefficients, as ANOVA often isn’t appropriate for their data, and analyzing regression results is often overwhelming. However, you may find a different method works for your course, and fortunately, Excel does have the capability to do all three, as well as most others you may want to teach.

5

USING K12 FOUNDATIONS …

77

The final step in the third segment is to compose a research report that presents the findings. This is a key science practice, and it ties what they have done in the third segment back to the first: their findings contribute to what we know about our discipline. They have used science & engineering practices to generate disciplinary core ideas.

Conclusion As more academics take time to consider how their course offerings fit into a broader curriculum and into their students’ career ambitions, it is critical that we reevaluate the way we teach our research methods courses so that we can encourage our students to leave programs with literacy: in political science, in science more broadly, and in their future careers. In combining my experience as a political scientist with my experience in K-12 science curriculum, I have been able to create a methods course that provides this literacy, tying to my students’ previous experience in the science classroom and to their future lives as engaged citizens and scientific thinkers. It certainly isn’t the case that every political science professor needs to incorporate the NGSS framework into their course in this same way: other faculty might feel more comfortable tying their course to students’ substantive classes, or to other subjects that students are familiar with from their K-12 experience. But pulling our research methods courses out of the silo that has held them for so long (and terrorized so many math-averse students!) can help make our courses more accessible, more useful, and more successful for both our students and ourselves.

Note 1. I assume. Not being a physicist myself, I am not entirely certain of what their datasets contain.

References Manzo, W.R., and K.M.W. Mitchell. 2018. We Need to Rethink Training for Ph.Ds. Inside Higher Ed. Accessed September 11, 2018. https://www.ins idehighered.com/advice/2018/09/11/academic-training-phds-needs-focusmore-teaching-opinion. Next Gen Science. 2020. https://www.nextgenscience.org/.

78

K. M. W. MITCHELL

Thompson, G. 2019. Adoption of the Next Generation Science Standards. Victory. Accessed May 6, 2019. https://victoryprd.com/blog/update-onnext-generation-science-standards-ngss/.

CHAPTER 6

Designing a Research Methods Course for a Skeptical Classroom Tavishi Bhasin

Inauspicious Beginnings I came to research methods the way most of my students do: kicking and screaming. Given these auspicious beginnings, it was no small miracle that I graduated from an R1 Political Science graduate program with a minor concentration in research methods. I had the good fortune of having excellent methodologists teach a series of research methods courses, including the standard fundamentals of research design, another covering basic statistics, and then advanced courses in Game Theory, Statistics and Qualitative Methods. I took them all, excited to learn about the best ways to get answers to the questions that sparked my curiosity in Political Science. In a social science field filled with nuance, I loved this rare area with some clarity. By the time I finished my graduate degree, there was no longer a quantitative–qualitative chasm in the discipline.

T. Bhasin (B) Department of Political Science and International Affairs, Kennesaw State University, Kennesaw, GA, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_6

79

80

T. BHASIN

King, Keohane and Verba was an assigned textbook in most graduate programs and there was a move to use mixed methods, recognizing that each approach, used systematically, had its role to play in the larger scientific endeavor of seeking answers (1994). I did my part, readers. I drank the whole jug of Kool-Aid, then immediately started a neighborhood stand, making and selling glasses of it. I am, not ashamed to say, an unabashed rigorous research design evangelist. As an Assistant Professor, I started with an undeserved level of confidence. I looked forward to taking on a challenging course and converting students who associated methods with dread to those ready to take advanced methods courses and pursue graduate school. I had some experience assisting a senior faculty member to teach the undergraduate methods course for years at Emory University, an exclusive private, liberal arts university where I got my graduate degree. The upper-level course was built around a broad social science methods text with a STATAfocused lab manual. Students learned the software while also learning the foundations of research design. The senior Professor lectured on the foundational concepts of research design and I had them for lab time where they learned basic data manipulation and analysis. It taught junior and senior students basic STATA and analysis skills, culminating in them writing complete research papers with a brief analysis using survey data. It offered students hands-on experience practicing and writing professional research, from a literature review to conducting statistical tests on survey data. The students at my alma mater were extremely privileged, coming from some of the best schools around the country. They were traditional students, attending college full-time, used to getting high grades with every resource available to them. The vast majority had their own computer with few distractions outside of those offered by a vibrant college campus. Many of them came to the course with some research methods training from high school such as AP Statistics or had, at the very least, been introduced to some of the basics of research design in other courses. The sections I worked with were small and I enjoyed helping students acquire and practice these skills. They didn’t have to be sold on the importance of learning research methods. They came to college already prepared with the basics of the scientific process. They could identify independent and dependent variables, write simple hypotheses, and understood the importance of reliability and validity of findings.

6

DESIGNING A RESEARCH METHODS …

81

I modeled my first few methods courses on the one I had helped teach at Emory. I left convinced I was hitting the ground running at my new home, a large suburban, public university with a diverse body of students, many of whom were non-traditional students. I was hired to teach Research Methods and it was one of the two courses I was asked to teach my very first semester as a new hire. Teaching two courses while finishing up a dissertation is never easy—I still think of that semester as the most exhausting yet. I chose a new textbook that seemed to be written with political scientists in mind and that came with an accompanying lab manual like I had used before. The course was set up in one of the computer labs in a beautiful new building housing the Political Science department. I was excited to help 35 new students develop the same excitement for the research process that I experienced.

Reality Bites Reality hit as soon as my five-feet-high-self entered the classroom, a computer lab with every student hiding behind a giant screen. Additionally, my students ranged from freshmen who just left high school to seniors about to graduate. They came from a variety of majors. Most worked at least part-time to support themselves. Many were young parents, juggling demands of the workplace, home and making time for college classes when possible. Many were first-generation college students. While this has changed rapidly with the growth of the university, this was the average student when I started in 2007. Now my public university has more traditional students, although many still work at least part-time and have little to no previous experience in statistics or research methods. My initial approach to teaching research methods, built on what I had done as a Teaching Assistant at my alma mater, focused primarily on building SPSS skills in data manipulation and analysis. The first few weeks were spent on basic research design concepts. A week each on defining and understanding the role of theory, differentiating theory from hypotheses, a week on how to conduct academic research (focused on both using the university library as well as citation styles), a week on operationalization and measurement, a couple more on experimental design and operationalization and measurement. The vast majority of the course, however, was spent providing students hands-on experience with data analysis using SPSS, the data program provided by the university. Students learned all the basic modes of analysis, from descriptive cross-tabs, tests of

82

T. BHASIN

means to bivariate and multivariate analysis. There was one week devoted to qualitative methods, where I talked with students about different types of case studies and research designs used in comparative politics, my own area of interest. I taught the class twice in this manner, building a portfolio of other new courses alongside, getting more comfortable with teaching as a new instructor. I slowly developed my own classroom style, but my methods class was associated with an unsustainable practice and yielded disappointing results. Many students clearly struggled to get to the point where they could proficiently attempt the basic data manipulation needed to complete their papers inside classroom hours. To help them bridge this gap, I spent hours outside the classroom helping individual students grapple with basic SPSS skills. I kept a pillow and blanket in my office because it didn’t make sense to drive home for a few hours of sleep. A few advanced students thrived with the individual attention and mentoring, and loved the idea of putting SPSS skills on their resumes. Many others, though, didn’t seem to enjoy the class. I ended up with students surprised about the level and amount of work required, papers written to ensure they met the minimum requirements for a grade. Even worse, in spite of my spending dozens of extra hours outside of the classroom, I wasn’t making the dent I had hoped in inspiring a love for research in students or improving their appreciation or understanding of the scientific process. At the heart of this disappointment were incompatible expectations for both the students and me. This highlights the innate problem for a research methods course, how to strike the balance between rigor and accessibility for students. Time spent with the students and their final products demonstrated that most gained a decent overview of the process of conducting data analysis, given that the majority of the class focused on building SPSS skills, and they did get hands-on experience in the same. Yet many struggled with the key building blocks of research design. They had a shaky grasp of the basic concepts of dependent and independent variables, had trouble writing good hypotheses without assistance, were able to competently conduct bibliographic research but didn’t know how to successfully organize comprehensive literature reviews that provided a background for their main hypotheses. They could find resources on any subject but did not know how to evaluate these resources and weigh scientific evidence on a subject. They were practicing the scientific method at a micro level but were leaving without an appreciation of the overall process as one

6

DESIGNING A RESEARCH METHODS …

83

mostly made up of incremental change and occasional paradigmatic shifts (Kuhn 1996). They were losing the forest for the trees and I was the guide who was failing them in this quest. The approaches I had learned at my alma mater did not work for this required course at my new institution. For most students, woefully under-prepared and pulled in a million directions, the basic concepts of independent and dependent variables continued to be as elusive as the mysteries of the universe. Their goals were not the ones I imagined for them. Many of them would not go on to graduate school. Those who were considering graduate school were primarily interested in law school. I was left with an existential pedagogical question; what was the overall purpose of the methods course I was hired to teach? How could it best serve the students in my classroom?

Take Two At the same time, I was commiserating with colleagues who taught a variety of upper-level courses in the International Affairs and Political Science majors. They bemoaned the lack of information literacy with which students arrived in their courses. They didn’t need them to be proficient in SPSS. They hoped for students who could write a decent literature review, understand a basic regression table and form a simple hypothesis. In essence, they needed students to understand enough about the research process to read and understand journal articles assigned to them for these courses. These conversations and my own experience in the classroom changed my goals for the course. Making them good producers of scholarly research was a great aspirational goal but no longer the one that would define my approach to teaching research methods. Now I was going to focus on helping students become good consumers of the scholarly literature, leaving my course with at least a good understanding of the research process if not an appreciation for the scientific method. I felt a strong sense of obligation to make some big changes to my approach to teaching methods along with the content of the course to ensure it better met the needs of the students and the degree program within which they found themselves. I spoke with mentors from graduate school, colleagues at KSU, friends at other institutions whose teaching styles I admired. All generous, wonderful people who shared their takes and strategies. These conversations helped me see that I was trying to do too much in one course. I had been faced with the eternal dilemma

84

T. BHASIN

in front of most research methods instructors given the sole research methods course offered to undergraduates: to focus on statistical and data analysis skills or to focus on principles of research design. In my first redesign, I focused on a common set of concepts that would help students become savvy consumers of academic literature. I no longer focused on data analysis. Students still had some hands-on experience with SPSS over a couple of weeks where they learned to put together some basic charts and tables and ran a basic regression. I kept the examples very simple and a much larger portion of the class now focused on them understanding the building blocks of research design. For each concept, I chose a seminal article or book chapter, sampling across the subfields within Political Science, looking to give students direct access to original research, some of these taken from my own graduate courses in research methods. My goal was to frame each topic with a provocative research question, walking students through examples of accessible research articles. I presented a new method each week, illustrated by a research article from a leading political science journal that was appropriate for this course and yet challenging for students. The articles chosen also illustrated an ongoing conversation between scholars on the same topic who support the use of different methods, highlighting the advantages and disadvantages associated with each research design. I found students benefited from both seminal, innovative examples but also examples which contained sections they could critique and improve. Sometimes I would bring in journals from our department library for students to examine and had them improve article abstracts. I ensured students understood the comparative strengths and weaknesses of qualitative, quantitative and mixed-methods designs. The initial revision did a much better job at introducing students to the principles of research design with examples from across the main sub-disciplinary areas of Political Science that illustrate effective use of these designs. More students were able to correctly identify independent and dependent variables and write directional hypotheses. They left with some practice in writing a short literature review and learned about some effective ways in which reviews could be organized. Overall, the students already committed to the major or leaning toward it enjoyed reading primary literature and being introduced to the key intellectual questions and debates highlighted in the chosen literature. However, in my enthusiasm to share with students, primary literature, the very best examples of

6

DESIGNING A RESEARCH METHODS …

85

a discipline I loved, I seemed to have forgotten a key teaching principle, dear readers. New concepts are much easier absorbed from a class and readings that are engaging for everyone! Importantly, the readings that I found interesting and engaging did not naturally engage the students, many of whom continued to see the course as a required obstacle on the path to graduation. I still needed to overcome perhaps the most difficult challenge we face in making research methods accessible and useful to the vast majority of students in our classrooms: convincing students that learning the basic principles of research is both interesting and of practical use to them. What comes easily, even naturally, when teaching substantive courses to students intrigued by the subject matter, takes a lot more thought for a methods-focused course. Armed with this lofty goal–engaging more students, from freshmen to seniors, from a variety of majors–I needed to step back and think of this course anew. In order to reimagine the course, I had to dig deep into my own methods of evangelism to remember why I loved and wanted to teach research methods, a subject so many dreaded taking and so few wanted to be burdened with teaching. What made methods so important to me was this clear understanding of its importance in how we learn things about the world.

Science Literacy: Science as a Way to Understand the World Around Us I distinctly remember driving my 45 min to work one morning, listening to Walter Isaacson’s biography of Albert Einstein (a very entertaining read/listen) (Isaacson 2008). This part of the story really stuck with me. Isaacson speaks of Einstein’s views on education. Einstein thought the most important thing students need as part of a good education is learning how to see the world as a scientist. Teaching them how to think, I extrapolate, is teaching them how to think scientifically. The key it seemed was science literacy. I started looking at science literacy and skepticism. Outside of the university, there were some worrying trends, a slow but steady growth in science skepticism among the public. Several surveys showed a decline here in the United States and in much of the Western world, in public trust in scientists working toward the public good (Smith-Schoenwalder 2019; Weber 2019; Funk 2020). Studies done to understand this trend point to ideology, education and science literacy as

86

T. BHASIN

factors explaining varying levels of skepticism(ACA 2018; Rutjens et al. 2020). Science literacy depends on a good understanding of research methods. A strong research design, I desired to help students see, is fundamental in our understanding of the world around us. Good research helps us advise on more and less effective means of protest, routes to democratization or paths to economic development. The power of research methods lies not in their inherent challenge of being an engaging puzzle for some. Rather, its power lies in testing our intuitions about how the world works. In order to get students excited, I had to make them see not only why research mattered in resolving key questions of interest to political scientists but how it could be used to answer questions of interest to humanity as a whole, including everyday concerns of their lives. First, I got students into a classroom more conducive for conversation‚ moving them out of the computer lab to a regular classroom. I set out to switch from a focus on a key skill-building approach alone to one looking to cultivate a generalized understanding and if successful, an appreciation of the scientific process. At the heart of this endeavor, was giving students a new way of looking at the world. A way to structure their thinking, a way to talk about causal relationships and the lack of them in an effective way in any context. Once they acquired this view, they would never look at the world the same way again. That was the goal.

Science History: Mostly Incremental Changes with Occasional Paradigmatic Shifts Science literacy helps people understand the rigors of the scientific process, all the care taken to reduce bias in our tests and by extension, our findings. Yet, the same studies looking to understand science skepticism found another peculiar trend. Science literacy doesn’t fully explain skepticism in certain politicized areas such as climate science where the evidence is overwhelming in support of human activity influencing climate change (ACA 2018; Rutjens et al. 2020). Yet, greater science literacy was associated with more polarized views. Thus, experience with research methods was likely to make students less skeptical of scientists working toward the public good. Yet in key areas of public policy, even areas with overwhelming evidence, those with this knowledge seem to be even more polarized in their views than people without this knowledge and experience. While science literacy is important, it isn’t enough by itself to equip

6

DESIGNING A RESEARCH METHODS …

87

students to distinguish between a healthy scientific debate in a specific area of climate science from the overwhelming evidence demonstrating the impact of human activity on the climate. Science history, I found, is key to giving students an overview of how scientific discoveries are made. Mostly incremental changes with the occasional paradigmatic shifts of Kuhn’s scientific revolutions. It isn’t enough to teach them to understand the components of research design, if they can’t step back and understand how progress is made toward improving our understanding in important areas of public policy. They need to be able to see how most scientific progress depends on incremental progress, relying on evidence from a variety of sources. It’s not enough that they be able to find a relevant scholarly resource on a topic if they are only going to use it to support their particular point of view, just increasing confirmation bias. They have to be able to see how we reach scientific consensus, when there is overwhelming evidence in support of a hypothesis. This overview of science history is just as important as science literacy. I set out to collect a wide range of examples, from as many disciplines as possible, of the scientific process working to support paradigmchanging discoveries. One such example comes from a wonderful book recommended to me by Dr. Jennifer Gandhi, a mentor at Emory, from The Incomplete Guide to the Art of Discovery (Oliver 2004). Written by a professor of atmospheric sciences, the book uses the example of continental drift to explain why our world appears as it does today. The book explains how the theory of continental drift evolved, from the initial difficulty faced by Alfred Wegener when he first presented his largely rejected idea of continental drift, to the decades of work undertaken to develop a complete explanation of a process. A post-WWII world helped us develop the tools and scientific methods needed to collect evidence that supported this original “paradigm-shifting” idea with evidence based on magnetic patterns that explained the age of the seafloor and seismic data from earthquakes. The scientific process is at its heart, a collective endeavor, one based on early theories supported by a preponderance of evidence supporting that explanation. This compelling example of a real-world shift in our paradigmatic understanding of the world usually gets most students excited enough to pay some attention to my musings on the usefulness of the scientific endeavor and the preponderance of evidence needed to cause such a shift. It also presents them with a clear scientific consensus in support of the continental drift explanation, once laughed at, but now central to our understanding of the physical geography of our world.

88

T. BHASIN

Methods as a Language, Small Assignments Allowing Hands-On Experience I have also learned to prepare students better for what they can expect in this course, unlike substantive courses they take. I explain that the subject has its own unique vocabulary. I now start the first day of class, speaking of research methods as analogous to learning a new language. Much like foreign language courses using immersion that throw students into the deep end by encouraging them to use the language on Day 1, I explain that I am going to use key vocabulary such as independent and dependent variables on the first day. These might be foreign to the students at the outset but with regular conversations, they will eventually be adept at using these new concepts. In terms of content, I still use some seminal scholarly articles but, these now form a part of a much more varied range of course materials. I now include analysis articles on provocative topics from popular news sources such as the statements on gender bias by former Harvard president, Larry Summers, to help students practice hypothesis writing. We look at a range of articles on happiness studies (the BBC news and Magazine), summarizing research on the subject from a variety of disciplines including medicine, economics and psychology to illustrate different ways of operationalizing the concept of happiness (Mombardieri 2005). I bring in articles from the current news cycle and have them think about key research questions emerging from the controversy, a couple of key hypotheses and measurements for the independent and dependent variables. My revised syllabus is now built around the core skills of identifying independent and dependent variables, writing research questions, effectively organizing a stage-setting literature review, composing testable hypotheses and operationalizing concepts for measurement. Each of these core concepts is continuously reinforced through new examples each week and then tested with a series of short one-page assignments. The first assignment asks students to write a good research question and identify the dependent variable and possible independent variables. The second assignment has them write a short literature review and at least one testable hypothesis. The third asks them to operationalize the dependent variable and one possible independent variable. The final assignment asks students to choose a research design or combination of designs that would provide a suitable test for their hypothesis.

6

DESIGNING A RESEARCH METHODS …

89

What would I say to another young educator starting on this often frustrating but always exciting and challenging journey of sharing a love for research with the next generation? First, know your audience. The students you had at your last institution may differ substantially from those at the new one. Ask them for their goals from the course along with figuring out how the course fits into the larger degree. Step back to figure out the key concepts or skills you want students to build through the course. Focus on those, introduce them and then reinforce these every week with as many examples and small hands-on exercises as possible. Second, whether the students are at an R1 or a small community college, they are all more likely to be engaged if we can illustrate how good research works to benefit their lives and those of people around the world. Third, science literacy isn’t enough. Students need an introduction to science history, an understanding of how progress happened in a key area of interest. It isn’t enough to be able to find a good resource if students can’t identify the direction of scientific consensus on the subject. Finally, there is no single best way to teach any course. Look to develop your own list of essential concepts and your own individual classroom style. Whatever these might be, know it is a constantly changing world in which we endeavor toward the difficult goal of instilling in a new generation of students, a keen appreciation and understanding of the labors and outcomes of a well-designed research process.

References American Academy of Arts and Sciences. 2018. Report on Perceptions of Science in America. Accessed on November 12, 2020. https://www.amacad. org/sites/default/files/publication/downloads/PFoS-Perceptions-ScienceAmerica.pdf. Beckett, E. 2017. Trust Me, I’m a Scientist. Australian Quarterly 88 (1): 21–27. Funk, C. 2020. Key Findings About Americans’ Confidence in Science and their Views on Scientists ‘Role in Society. Pew Research. Accessed on November 12, 2020. https://www.pewresearch.org/fact-tank/2020/02/12/key-fin dings-about-americans-confidence-in-science-and-their-views-on-scientistsrole-in-society/. Isaacson, W. 2008. Einstein: His Life and Universe. New York: Simon & Schuster. King, G., R.O. Keohane, and S. Verba. 1994. Designing Social Inquiry. Scientific Inference in Qualitative Research. Princeton: Princeton University Press. Kuhn, T. 1996. Structure of Scientific Revolutions. Chicago: University of Chicago Press.

90

T. BHASIN

Mombardieri, M. 2005. Summer’s Remarks on Women Draw Fire. The Boston Globe. Accessed August 20, 2020. http://archive.boston.com/news/educat ion/higher/articles/2005/01/17/summers_remarks_on_women_draw_fire. Oliver, J.E. 2004. The Incomplete Guide to the Art of Discovery. New York: Columbia University Press. Rutjens, B. and R. Lee. 2020. Spiritual Skepticism? Heterogeneous Science Skepticism in the Netherlands. Public Understanding of Science 29 (3): 335–352. Smith-Schoenwalder, C. 2019. People Are Getting More Skeptical of Science. Poll Finds. U.S. News. Accessed on November 12, 2020. https://www.usn ews.com/news/world/articles/2019-03-22/people-are-getting-more-skepti cal-of-science-poll-finds. Weber, B. 2019. Canadians’ Trust in Science Falling, Poll Suggests. Canadian Broadcasting Corporation. Accessed November 12, 2020. https://www.cbc. ca/news/technology/science-survey-1.5291291.

CHAPTER 7

Journeys Beyond Information Literacy: Applying a Metaliteracy Framework to Political Science Sally Friedman and Trudi E. Jacobson

When you think of information literacy, what comes to mind? Lectures about how to find the most appropriate sources for your work? How to evaluate information found online? The nagging feeling that you knew the material was important but you blocked most of it out till you were forced to use it?

S. Friedman (B) Department of Political Science, University at Albany, Albany, NY, USA e-mail: [email protected] T. E. Jacobson Information Literacy Department, University at Albany, Albany, NY, USA Faculty of Education, North-West University, Potchefstroom, South Africa T. E. Jacobson e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_7

91

92

S. FRIEDMAN AND T. E. JACOBSON

Despite being a long-time professor of political science, the first author of this paper had a fairly narrow understanding of information literacy when she found herself confronted with the challenge of developing an undergraduate course on critical thinking. The chapter’s second author was an expert in the field, but her long-standing career as a researcher and teacher left her dissatisfied with existing pedagogical models. The metaliteracy framework, a broad-gauged approach to information literacy and learning, was developed in part out of her frustration. The metaliteracy model became an important component of the critical thinking class that the first author teaches, and a now four-year collaboration between the political scientist and the librarian has led to a set of productive conversations on ways the framework may be adapted to the needs of political science classes. Our journeys related to information literacy, recounted at the start of this chapter, turned into deep dives into metaliteracy. After we describe the model, we suggest some potential applications in the teaching of political science.

Sally’s Journey I have long been intrigued with methodology and research design. As an undergraduate, I double-majored in political science and mathematics. I should admit I wouldn’t have completed my math major if I hadn’t run across a course on the comprehensive history of math. I was by then pretty bored with abstract proofs. For that course I found myself writing a paper on n-dimensional spaces (I was a nerd), and I ran across a book from the 1800s by someone named Edwin Abbot called Flatland (Abbott 1992). The story recounts the (of course fictional) story of a square (a two-dimensional being) who finds himself traveling to the worlds of Lineland (1-dimensional) and Spaceland (3-dimensional space). In Lineland, people along the line had to communicate with each other by sound (think about it; they couldn’t see what was happening at the other end of the line), and they were pretty freaked out when confronted with a square, who could move freely in and out of their one-dimensional space. Subsequently taken to Spaceland, the square in his turn has to come to terms with his two-dimensional limitations and for the first time gets to look down on the Flatland from which he has come. It’s a great lesson in appreciating different perspectives!

7

JOURNEYS BEYOND INFORMATION LITERACY …

93

In graduate school, I got more of the hands-on applied research design and statistics that I wanted. Of particular interest, the statistics department had developed an amazingly innovative computer program called MIDAS. It was well ahead of its time in terms of its interactive nature and the sophistication of the statistics it could handle. (Side note: I’m visually impaired; with MIDAS and the old noisy computer terminals, you could always tell without looking when you had made a mistake; the program had more to tell you; the noisy terminal had more to type out.) As a professor, I have long focused on teaching standard courses in research design and methodology. Teaching these courses is particularly rewarding because they provide students with a broad framework for how to think about research and help students reflect on what and how they are learning. By defining concepts and hypotheses, you’re asking people to hone in on exactly what they mean and to clarify why one variable would be a good explanatory variable for another. By teaching different types of research design, you’re asking students to think about what method will give them the most bang for their buck to answer a research question, and you’re showing them that alternative strategies for doing research exist. You are also often showing students they can utilize a set of skills they think they are intimidated by as different ways to learn. Different perspectives. It’s still about moving in and out of Flatland. But a couple of years ago, I was confronted with something a little different: the challenge of teaching a new course that included general education critical thinking and information literacy requirements. The thought of teaching critical thinking, asking people to get clearer about the arguments they were making and encouraging them to see different perspectives and to develop some new skills intrigued me for the same reasons I liked teaching research design. I wasn’t so excited about the information literacy part. Yes, we should teach people more about how/where to look for sources, how to take into account sources that challenge their own viewpoints and to evaluate the quality of the sources they find. But did I need to be doing that? Aren’t the library faculty better equipped to handle the job? It has been the collaboration with librarians that introduced me to a framework that not only focused on evaluating the nature of information but also put that information into a broader context that more closely meshed with my long-held interests.

94

S. FRIEDMAN AND T. E. JACOBSON

Trudi’s Journey I am one of those library faculty members that Sally mentioned working with. I am the head of the Information Literacy Department, so my focus is on teaching, both my own courses and as a guest lecturer in other faculty member’s classes, as well as overseeing a number of learning resources, such as tutorials, that my department members create. I didn’t plan to specialize in this area of librarianship, indeed, when I took my first academic position after graduate school, I didn’t even know it was an area of specialization. I started as a reference and interlibrary loan librarian, with no thought of teaching. But within a few weeks, a colleague said he’d show me how to conduct a teaching session (“bibliographic instruction,” in the parlance of the day) to help students in a particular course with their research assignment. He filled a cart with appropriate reference books (this obviously was some time ago) and took them into the classroom. The class consisted of holding up one volume after another and explaining how each would help for the assignment. I was bored to tears within minutes, and I expect the students were also. If I were going to have to do this as a part of my position, I had to find a new way of doing it. Learner engagement, motivation, and opportunities for active learning became key to my teaching as I continued in my career. As a part of this quest, I became engaged in national information literacy initiatives, eventually becoming the chair of the primary professional organization for this discipline. Fast forward a couple of decades, when I’d been in my current position at the University at Albany for some time. I continued to have two related major frustrations in the teaching that I and other academic librarians were doing: time and scope. My story broadens a bit at this point, to provide some background about information literacy and the emergence of metaliteracy. Information literacy requires conceptual understanding that far exceeds finding sources for a research project. Our students need to be adept at using, evaluating, and creating information. They also need to understand such concepts as scholarship as conversation and authority is constructed and contextual (two out of six frames from a new understanding of information literacy) (Association of College & Research Libraries 2015). Yet in the context of any particular course, addressing the full complexity of information literacy is challenging and often not appropriate. It takes time and experience to become information literate. Yet librarian-led sessions

7

JOURNEYS BEYOND INFORMATION LITERACY …

95

that focus solely on what sources to use and how to access them still predominate and do not assist students beyond the assignment at hand, or, at most, similar assignments. The time available is of course problematic, as is the need to address course learning outcomes. However, on the part of librarians, a guiding document was also part of the issue. From 2000 to 2015, the teaching of information literacy in the United States was shaped by a set of standards that was very much skillsbased (Association of College & Research Libraries 2000). A former colleague, Tom Mackey, and I frequently spoke about how these standards missed the mark for what students need to understand about the current information environment. The Internet and social media had radically altered everyone’s roles in connection with information consumption and production. From our conversations and our deep engagement with information literacy, metaliteracy was born.

Metaliteracy’s Foundational Elements Metaliteracy is a framework designed to broaden and challenge traditional skills-based approaches to information literacy by recognizing related literacy types and incorporating emerging technologies (Mackey and Jacobson 2011, 62–63). From its inception, metaliteracy emphasized learning goals and objectives (Jacobson et al. 2018) and learning domains. As it developed, learner roles (Jacobson et al. 2018), and learner characteristics (“Metaliterate Learner Characteristics” 2019) were added. As the model became more multi-faceted, it became clear that it was coalescing around active and responsible learning, information production, and empowering learners. While the model was originally designed to point up changes in the learning environment (more online material; more use of social media), it is equally applicable to more “traditional” environments. This resource gives a snapshot of the core components of metaliteracy: https://metaliteracy.org/ml-in-practice/integrated-metaliteratelearner-figure/. The metaliterate learner is at the center, with the first ring highlighting four learning domains. The next ring provides the characteristics we want metaliterate learners to strive for, and the outer ring shows the roles that they could take on. All of these components are undergirded by four learning goals, each with a number of objectives (Jacobson et al. 2018). In this chapter, we focus on the roles and domains and in the

96

S. FRIEDMAN AND T. E. JACOBSON

next section associated activities (quests) that move the framework from theoretical to practical. The learning domains are important because they provide a holistic view of the learning process. They begin with the familiar: the cognitive (what one knows after completing a learning activity). This domain, as well as the behavioral (what one can do), were the focus of the Information Literacy Standards and often remain the focus of information literacy teaching. However, the affective (how one feels about one’s learning, or how one’s attitudes or emotions have changed following the learning activity) and the metacognitive (reflecting on one’s thinking and what still needs to be learned, which also involves recognizing one’s preconceptions) are also critical for learning, particularly in a polarized society. When learning situations highlight the affective (Feidler and Beirer 2014) and metacognitive (Hartman 2001) domains, and ask students to actively consider them, learning can be enhanced. The roles are another core component of metaliteracy, which, like the learning domains, emanate from the goals and objectives. Students may not envisage themselves taking on various roles, such as researcher, communicator, collaborator, and teacher, roles that have both responsibilities and great potential. Simply recognizing that they may do so and concomitantly bring value to others is empowering for learners. Should you want to introduce the metaliteracy framework to students in a brief but coherent manner, “Metaliteracy and Your Role as a Metaliterate Learner” is an appropriate starting point. It explains the foundational elements, provides an activity for students to put what they have learned into practice, and also includes reading and thought questions (2019). Depending on how you might adapt the application ideas below, providing this background reading to students might be of assistance. Additional metaliteracy resources that can be used as is or adjusted for a specific course will be described in the applications section below.

Applications Potential applications of the metaliteracy framework can be as varied as an instructor’s creativity, and below we offer a start on possibilities. It is worth noting that the framework can be applied to meet a variety of pedagogical goals. It can be used to focus on broader learning concepts (as the paper’s first author needed to do in the critical thinking course), including critical thinking, information literacy or life-long learning. For

7

JOURNEYS BEYOND INFORMATION LITERACY …

97

purposes of this chapter, the main goal for most instructors is to get across substantive content, and often that’s in itself an all-consuming task. Thus below we focus on ways metaliteracy can deepen student understanding of substantive course material. Further, the metaliteracy framework can be helpful, both by suggesting additional material for current class exercises and by encouraging instructors to adapt already-existing metaliteracy material developed by the Metaliteracy Learning Cooperative (Mackey et al. 2020).

Learning Domains Learning domains are at the center of the metaliteracy model. They remind us to consciously ask a number of important questions we often leave implicit. It’s easy to get caught up in the cognitive. But the other domains yield insight too. This quote, from an undergraduate senior in a course taught by the second author, provides evidence of the impact that the affective domain had on her learning. I really think it is crazy that I have been learning since I was born, and I haven’t considered how learning the things I have learned have made me feel….I like how it feels to check in with myself and get more in touch with my emotions when learning, I think this will leave me feeling more confident and proud of myself after completing new tasks and learning objectives.

Students can be asked to engage with the domains through questions such as: What do you feel about the exercise/material (affective domain)? What actions might you be motivated to take outside class (behavioral)? Questions focusing on the metacognitive domain might move the discussion beyond the substance of the material but it could be valuable for an instructor to ask students to reflect on what they discovered about their own learning and how this will impact future learning endeavors. Two examples show the relevance. Consider, as we often do in American politics, the advantages/disadvantages of different forms of political participation: voting, giving money, engaging in a protest. The cognitiveoriented discussion of pros and cons of these different activities is certainly productive. Consciously adding in an affective dimension has the potential to get students to think about what these activities really mean to them. I feel proud when I enter the voting booth. I feel nervous when I think

98

S. FRIEDMAN AND T. E. JACOBSON

about the uncertainty of going to a protest. A behavioral focus could also contribute to the discussion. What can I really do to fix a system I think isn’t working? What are additional possibilities for action? How motivating is this for me to actually get involved, and why? Similarly, a discussion of violence and terrorism could benefit from conscious incorporation of affective and behavioral domains. How does one feel when one reads about a terrorist act or, conversely, when one watches an interview with someone who has committed a terrorist act? What can be done to decrease chances of violence, and what as an individual might I be willing to do to work on this issue? There are of course any number of ways to get students thinking about these issues, but a conscious use of metaliteracy’s four domains reminds us that it is important to do so.

Roles There are a number of possibilities when asking students to consider the roles that a metaliterate learner would feel comfortable taking on. In this section, we suggest several. When asking students to engage with these roles, it may be helpful to share more information with the class. The first author found that her students were having difficulty conceptualizing some of the roles, and in response, the second author and other members of the Metaliteracy Learning Collaborative developed a set of introductory reflective questions for each role (Jacobson et al. 2018). A short introductory video describing the roles might also be an excellent resource (“The Roles of the Metaliterate Learner—YouTube”, n.d.). Below are a couple of starter examples. Different Perspectives and Small Group Activities With an understanding of the value of active-learning pedagogy, instructors often divide a class into task-focused small groups. Sometimes each student in a group is assigned a different role, with group members asked to develop a specific policy or make a decision on the basis of coming to terms with the different roles they were assigned. Alternatively, all students in a particular group might be assigned a specific interest, with group differences sorted out when the groups come together in the class as a whole. Either way, the metaliteracy roles can be used to enhance these activities.

7

JOURNEYS BEYOND INFORMATION LITERACY …

99

Thus, in a class on federalism, students might be asked to take on the perspective of a state (New York, Ohio, California), representing its particular interests. An instructor of international relations or comparative politics might develop exercises where students represent the perspectives/interests of particular countries. Nor is geography the only way to break students into groups. Any activity with the goal of thinking through the importance of different perspectives (various stakeholders on a college campus; administrative departments in a bureaucracy; bargaining among demographic groups to better understand descriptive representation) would be appropriate. In these types of exercises, the metaliteracy roles help to deepen the learning. For example, students might be asked to assume the role of researchers collecting additional information on their designated task. They could be cast in the role of teacher, not only representing the interests of their state, say in a federalism class, but using the teacher role to formalize their perspective, conveying to other students the depth of the interests of the state. Students might be asked explicitly to collaborate with each other (collaborator role)—i.e., in a federalism exercise, what would happen if students representing different states were asked not just to compete for a valued good but to explicitly develop a win/win scenario for several states? Focus on Specific Roles Another example of a context where metaliteracy could prove valuable would be to single out specific roles for focus. An instructor wants students to collect additional information on a topic; label it the researcher role, and students could augment their understanding of the topic at hand with a sense that they were engaged in a learning activity with larger implications. While gathering information about Topic X, they are also getting a glimpse into what it means to do research. Similarly, the role of teacher might be invoked when dealing with particularly complex topics (multi-layered philosophies of important political theorists; advantages/disadvantages of different theories of international relations). Instructors sometimes ask students to explain ideas to each other; explicitly assigning the role of teacher formalizes and augments these activities. In such an activity, students would first read the clarifying questions about the teacher’s role, which will help them to understand the full scope and responsibilities involved.

100

S. FRIEDMAN AND T. E. JACOBSON

A course with a focus on roles might have an assignment that asks students to consider how they could make that unit of content accessible to someone who is not in the course. How would they translate the content to make it understandable (translator role)? Another approach, taken by the first author in her critical thinking class, asked students which of the roles they most wanted to become more comfortable with. In a number of subsequent group activities, she asked the students to take on the roles they had designated. Quests To translate the learning domains and roles for students, the metaliteracy framework has been embodied in a resource that incorporates a number of quests (starting level) and challenges (which ask students to reflect on and synthesize learning from the quests). The full resource can be explored at https://sites.google.com/view/metaliteracy. The first author of this paper has used a number of these as staples of her critical thinking course. The Develop Content quest asks students to collate a number of sources on a specific topic and describe why they picked these particular sources. There is also the option of making the results public via an online curation tool. While clearly a key component of information literacy, the first author has also used this quest as a chance for students to put their own stamp on a particular unit we have discussed (e.g., income inequality). After engaging in a number of specified readings, the quest offers a chance to expand on what students have been “forced” to read and to focus on an aspect of the topic of particular interest to each of them. Similarly, the Expand Horizons quest asks students to put their own stamp on a topic by giving a short presentation to the class based on a diverse set of formats (short videos, controversial articles, research findings, personal accounts). The first author has used this quest in conjunction with the particular topic under discussion (the impact of generations, immigration) and also in an open-ended format so students could choose their unique focus. A couple of the most innovative studentauthored quests (author role) have covered the acknowledgment of loneliness among the millennial generation even in an age of social-media connectedness and the varied meanings of the commonly-enjoyed sport of baseball to different generations in Taiwan (this quest was selected for inclusion in the learning resource).

7

JOURNEYS BEYOND INFORMATION LITERACY …

101

The first author of this paper also adapted The Expand Horizons Quest to an upper-level course on American values. She wanted to make it a group rather than an individual project. In the context of talking about the level of division versus unity on the current political scene, groups of students picked a particular demographic (race, religion, region) and then presented to the class as a whole on the importance/character of the demographic group they were considering.

Conclusion While using the metaliteracy framework is not a necessary condition for developing some of the activities highlighted in this chapter, it does provide a cohesive organizing structure. Its interconnected components are learner-centric. It focuses on enhancing the quality of one’s learning and engagement with information in both online and in-person settings, and while we have here only scratched the surface of its components and potential applications, it is clear that the learning domains, roles, and quests have the potential to contribute in important ways to classroom activities and student learning.

References Abbott, E. 1992. Flatland: A Romance of Many Dimensions. New York: Dover Publications. Association of College & Research Libraries. 2000. Information Literacy Competency Standards for Higher Education. Association of College & Research Libraries. https://alair.ala.org/bitstream/handle/11213/7668/ ACRL%20Information%20Literacy%20Competency%20Standards%20for% 20Higher%20Education.pdf?sequence=1&isAllowed=y. Association of College & Research Libraries. 2015. Framework for Information Literacy for Higher Education. Association of College & Research Libraries. https://www.ala.org/acrl/standards/ilframework. Feidler, K., and S. Beirer. 2014. Affective and Cognitive Processes in Educational Contexts. In International Handbook of Emotions in Education, ed. Reinhard Pekrun and Lisa Linnenbrink-Garcia. Educational Psychology Handbook Series, 36–55. New York: Routledge, Taylor & Francis. Hartman, H.J., ed. 2001. Metacognition in Learning and Instruction: Theory, Research, and Practice—Neuropsychology and Cognition, vol. 19. Dordrecht and Boston: Kluwer Academic Publishers.

102

S. FRIEDMAN AND T. E. JACOBSON

Jacobson, T.E., T.P. Mackey, and K. O’Brien. 2018. Metaliterate Learner Roles. Metaliteracy. https://metaliteracy.org/ml-in-practice/metaliterate-lea rner-roles/. Jacobson, T.E., T.P. Mackey, M. Forte, and E. O’Keeffe. 2018. Metaliteracy Goals and Learning Objectives. Metaliteracy (blog). https://metaliteracy. org/learning-objectives/. Mackey, T.P., and T.E. Jacobson. 2011. Reframing Information Literacy as a Metaliteracy. College & Research Libraries 72 (1): 62–78. https://doi.org/ 10.5860/crl-76r1. Mackey, T.P., T.E. Jacobson, and K.L. O’Brien. 2020. Metaliteracy Resources for Online or Remote Teaching & Learning. Metaliteracy.Org (blog). April 7. https://metaliteracy.org/2020/04/07/metaliteracy-resources-for-onlineor-remote-teaching-learning/. “Metaliteracy and Your Role as a Metaliterate Learner.” 2019. Metaliteracy.Org (blog). September 10. https://metaliteracy.org/ml-in-practice/what-is-metali teracy/. “Metaliterate Learner Characteristics.” 2019. Metaliteracy.Org (blog). March 10. https://metaliteracy.org/ml-in-practice/metaliterate-learner-characteristics/. “The Roles of the Metaliterate Learner—YouTube.” n.d. Accessed July 11, 2020. https://www.youtube.com/watch?v=Fe5t3FW4AII.

CHAPTER 8

The Savvy Consumer of Political Science Research Jonathan Ring

My first semester teaching research methods was a disaster. I expected students to share my passion and curiosity for understanding how we know what we know. Instead, I encountered a distribution of students: 5% were the enthusiastic, methods-prone students I had imagined, 75% were capable but unenthused, and 20% were often confused and openly hostile about learning math in a political science class.1 My journey in teaching research methods has been one of transforming that distribution so that all students understand not just the content, but the value of learning research methods. This chapter will discuss how I attempt to reshape the distribution of student experiences within a research methods course. I advocate for a curriculum-wide view that positions the course as a bridge between introductory (freshmen and sophomore) and upper-level (junior and senior)

J. Ring (B) Howard Baker Jr. Center for Public Policy, University of Tennessee Knoxville, Knoxville, TN, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_8

103

104

J. RING

courses. One semester is not long enough to teach skills in information literacy, research design, statistics, and basic computer programming. Instead, the introductory course on research methods should be designed to equip all students with the knowledge to understand, as Chief Justice Roberts so eloquently called it, “sociological gobbledygook.”2 To do so, I structure the course around data literacy and the premise that research methods should be an introduction to students’ journey into research and the scientific study of politics.

My First year Teaching Research Methods Research methods has a reputation of being difficult (Earley 2014). Nonetheless, I was excited to try my hand at teaching it. I thought I could bypass all the challenges by showing my passion for the subject. I did not expect to hear, anonymously, after just one month that I was a terrible teacher through mid-semester evaluations and the infamous ratemyprofessor.com: “If you value your sanity and gpa do not take.” “He expects too much from students…Terrible all over.” I valued and incorporated the productive feedback like the suggestions to provide more frequent quizzes and slow down during lectures and discussion. However, a concerning theme began to emerge from these evaluations: a resistance to taking responsibility for one’s own learning. The students with the biggest complaints and most cutting commentary on my value as a teacher also reported that they did not read before class. Feeling bemused and a bit stumped, I drew them a picture (see Fig. 8.1) and explained that learning takes effort. Rather than resetting the classroom culture, my mid-semester effort at reform seemed to inspire even greater resentment. That semester took an emotional toll on me. It shattered my belief that I was a good teacher and that my own passion for research methods was enough to inspire passion in my students. Despite my bruised ego, I took stock and opted to try again. I am passionate about math, science, and research methods, and I wanted to prove their accessibility to political science students. A few students rose to the challenge that first semester, but it was clear to me by the end that the course was too densely packed. I covered the philosophy of science, research design, probability and statistics, with a pinch of computer programming. Although I had no control over the curriculum for the political science major, I decided to approach the course design with my idea of an ideal curriculum in mind. It is not

8

THE SAVVY CONSUMER …

105

Fig. 8.1 An inaccurate model of learning

possible to create an effective research methods course that is isolated from a broader program of study. I began by asking myself, what should an introduction to research methods look like. If students leaving an intro class could not feasibly do their own research, what could they actually take away? What skills and concepts are most valuable? If (nearly) all majors have to take this course, and most take it in “order,” how can I use the class to position them for success in their other classes? These questions prompted me to add a simple sentence to my syllabus, which has become my core guiding principle in all subsequent tinkering to the

106

J. RING

course: “At the end of the course, students will be savvy consumers of contemporary scholarly research.” To carry out this new learning objective, I turned to Innumeracy by mathematician John Allen Paulos. Paulos defines the concept of innumeracy as the inability and discomfort in dealing with numbers, analogous to the well-known concept of illiteracy (Paulos 2001, 3). The major difference between innumeracy and illiteracy is that the average person is perfectly comfortable saying “I’m not good with numbers” or “I can’t do math” but would never say “I’m not good with words” or “I can’t do English.” The core problem with innumeracy is that it leads to decision-making pathologies at the individual and society levels. Innumerates are either crippled by irrational fears or cavalier when they ought to be cautious. With a better appreciation and understanding of numbers, probability, and data, we can adopt better policies as a society and individuals can live better lives. Combatting innumeracy is not about creating a world full of sophisticated scientists; it is about establishing a baseline of scientific and mathematical literacy for all.

Back to the Beginning I improved in my second semester teaching research methods. I was fortunate to receive a small grant to translate the course into an online format, which gave me the time and support to think broadly about my course goals. The online platform also posed a new challenge that required careful considerations. Students would not have the benefit of interactive, in-person discussions, and my ability to assess their understanding of difficult concepts on the fly would be highly constrained. The struggles of my first year and the opportunity to re-develop the course for online learning gave me space to think deeply about my own experience learning research methods, especially as an undergraduate student. When I came to college, I had an undeveloped sense of how knowledge is produced. I thought science was a set of disciplines like chemistry and physics. I had no awareness of the broader culture of knowledge production that values systematized methods of inquiry and the logic of falsification. I was attracted to political science because I liked history, and I liked to debate. This aligned well with the political science courses I took as a freshman. I read Plato and Plutarch; Freud and Friedman; Montesquieu and Mill, and I considered what a good society should look like.

8

THE SAVVY CONSUMER …

107

However, things took a life-changing turn in one class. The entire course was devoted to a single (normative) question: should the U.S. keep or abolish the death penalty? The core question fit with the political theory orientation I was familiar with, but something new emerged. We read studies that asked empirical questions: Is the death penalty (and the criminal justice system) racially biased? Does the death penalty deter crime? What is the likelihood of executing an innocent person? It wasn’t just that the answers to these questions were surprising (they were). It was the questions themselves that seemed different. The answers were important for the normative debate, but they seemed to exist independently of the legal and normative debates I was used to. I started to pay attention to things like whether particular measures of racial bias and deterrence were valid or if a particular sampling method biased our estimate of wrongfully convicted death-row inmates.3 By the time I started my junior year, I was actually excited to take the dreaded research methods course. In the first week, the professor assigned On Bullshit by Harry G. Frankfurt. The main premise of the monograph is that truth has ceased to matter. Frankfurt dissects the vulgar word with a philosopher’s careful approach. A lie is formulated to obscure the truth, and therefore must take the truth as true. “Bullshit” is different. According to Frankfurt, “the bullshitter… does not reject the authority of the truth, as the liar does, and oppose himself to it. He pays no attention to it at all. By virtue of this, bullshit is a greater enemy of the truth than lies are.” I never had a full appreciation for why I found talk radio and cable news so offensive, but Frankfurt’s concept of bullshit helped me understand. Bullshit does not “genuinely [submit] to the constraints which the endeavor to provide an accurate representation of reality imposes” (Frankfurt 2005). After being exposed to this idea, a new worldview came into focus. My attempt to resist bullshit was to be scientifically rigorous and learn research methods. The culmination of my undergraduate research methods training occurred during the process of writing my senior thesis in which I defined a key variable, collected data to measure it, and applied an ordered logistic regression model to see which variables potentially affected it. I learned so much by struggling through my own project, but I was successful because I had the foundational knowledge and a scientific worldview thanks to my experience in the introductory course. I continued research methods training in graduate school. In my first semester, I was introduced to the matter-of-fact teaching philosophy of

108

J. RING

my methods professor: “old crap out, new crap in.” From there, the most difficult semester of my academic career unfolded. However, in between learning about statistical distributions, expected values, and econometric models, the professor often shared bizarre and hilarious observations about the class and the world such as, “what’s the probability we’ll be lucky?” and “OLS is a jealous god; it doesn’t like other estimators.” My memories of that time are full of positive feelings of being with my friends while overcoming difficult assignments and laughing at the absurdity of our professor’s quick wit. I continued to learn a lot over subsequent years of taking methods courses. Statistical concepts and programs became familiar, and the struggles of learning them have since began to fade. Most importantly, I discovered that methods can be fun. Unfortunately, I also forgot what it was like to learn methods for the first time, and the unfamiliarity of concepts that now felt like second nature. Remembering both the difficulty and the fun has given me deeper empathy for my students as they are experiencing these ideas for the first time.

Incorporating My Reflections into Concrete Changes Reflecting on my experiences teaching and learning research methods led me to three main lessons: (1) less is more (2) focus on the core goal of information literacy, and (3) make it fun. In this section, I discuss the concrete ways I have incorporated these insights into my current teaching. Less is More After three short years of failing, experimenting, and improving, my thought experiment of designing a broader research methods curriculum came true in an abrupt and unexpected way. I was offered a position to run student programs at the university’s policy center. One important duty is to administer the public policy analytics minor program and teach its capstone two-course sequence, giving me a chance to consider methods within a broader curriculum and teach more advanced methods with students over a longer period. This stroke of luck has helped my introductory methods class run better. I am able to focus on the indispensable foundational topics that are necessary for grasping advanced concepts later on. Every student can benefit from this solid foundation.

8

THE SAVVY CONSUMER …

109

Doing too much can actually lead to less learning, especially with less confident and more skeptical students. I save the deeper dives and more advanced content for the upperlevel courses. Students who catch the bug have the opportunity to pursue the material in subsequent classes in a structured environment, preparing them for graduate school or policy-relevant research in government, nonprofit, or business sectors. They can opt into an advanced two-course sequence at the senior level. I now have the freedom and space to tinker and enact my own vision for what an ambitious undergraduate can achieve in these courses with students who already have a baseline understanding of research methods and statistics. Focus on Information Literacy Students are heterogeneous, and making the course valuable to all is a challenge. A single classroom contains a variety of learning styles, baseline aptitudes in mathematical reasoning and verbal expression, and career goals. Only a minority of my students head to graduate school. Of those, the vast majority study law. I strive to design a methods class that is worthwhile to all of my students, even those that claim they will not “use” their degrees at all. Learning how to ward off bullshit is helpful regardless of where you come from or where you are going. For students who do use their political science degrees to target government jobs, learning research methods adequately is essential. Bachner (2020) strongly advocates for extensive training in quantitative methods. She uses case studies to demonstrate the need that government agencies have for employees who are trained in quantitative methods and identifies six “in-demand skills [that] will best prepare students to be successful when they transition from academia to their professions” (Bachner 2020). The first three skills relate to information literacy. The last three relate to mastering the technical skills of a data analyst. Mastering information literacy is a necessary (but not sufficient) condition to obtaining and succeeding in a career in government. Students who do not pursue this track also benefit from the introductory course by gaining the core civic skill of thinking for themselves. They have the ability, practice, and confidence to understand scientific research without filtering it through a bullshitter. Due to the large number of students who aspire to go to law school, I have a particular mission to affect future law students. I frequently incorporate the story of Gill v

110

J. RING

Whitford, a Supreme Court case involving the allegations of gerrymandering in Wisconsin. Chief Justice Roberts resisted the use of quantitative political science research to inform the court’s decision, calling it “sociological gobbledygook” (see Roeder (2017) for an accessible discussion of the controversy). The case itself provides a compelling example of measurement of the well-known concept of gerrymandering and points to the importance of precision in conceptualization. And, importantly, it can help convince budding lawyers that they may indeed need to consider math in their future careers. Make It Fun My own experiences made it clear that learning research methods can and should be fun. The burgeoning literature on simulations and games makes a strong case that students do better when they have fun. They focus more in class and retain the information they learn more deeply. They also experience greater motivation to persevere when the going gets tough. I have adopted and advocated for game-based learning in substantive classes (Cohen et al. 2019). There are arguably fewer opportunities for fun activities in a research methods course, but several scholars have found innovative ways to include them. For instance, following Rosen (2011, 2016), I began using the board game Zendo on the first day of class to get students thinking about theory building, observation, and hypothesis testing. However, fun is not just facilitated by playing games in class. Another strategy I use is to deliberately frame the class as fun. After that first semester, I inclined to warn students “you may have heard that this class is hard” or “I regularly hear that this is one of the least enjoyable classes you will take in the major.” These are probably true statements, but they are also self-fulfilling prophecies. In the last iteration of the class, I tried something new. I omitted any priming for students to think of the class as hard or unenjoyable. Instead, I said how much I loved the class and how much fun we could have. I used the word “fun” a dozen or more times in the first class, and I frequently used it throughout the course. I noticed a real difference in my students (and myself) when I adopted this positive attitude toward the course.

8

THE SAVVY CONSUMER …

111

Reflections on Where I Am Now In a review of literature on teaching research methods, Earley (2014) discusses a fundamental debate on whether to consider students as producers or consumers of research.4 A significant portion of articles advocates for active or experiential learning (e.g. Aguado 2009). The merits of this hands-on approach are evident: students will gain practical experience with the research process. If they internalize these lessons successfully, they will have marketable skills that employers want (e.g. Bachner 2020). Moreover, experiential learning is widely believed to promote student interest. For me, however, the introductory course should treat students of research methods as consumers. The approach that I have articulated and adopted in this chapter provides almost no hands-on research experience. I recognize that as a possible weakness, and I may change my mind over time. There will always be tradeoffs, but I think the merits of my approach outweigh the shortcomings. Anyone can be taught how to download data and use statistical programming software to fit a model to that data, but it is a mistake to immerse students in data without first conveying basic scientific principles. From my own experience, fitting both baseline knowledge and application into a single semester is problematic. If the application of methods is useful for a minority of the student population and the concepts of research methods are valuable to all, it makes sense to focus on information literacy first. I still find that most classes begin with a distribution of students in which about 20% are confused (if not hostile), 75% are capable (if unenthused), and 5% are competent (and enthusiastic). Figure 8.2(a) shows a random draw of 30 students from a standard normal distribution with cut points at the appropriate locations to illustrate this distribution. I have accepted that not every student will love learning about research methods, but I have found that with measured expectations for content coverage, a focus on what all students need to know to be savvy consumers of political science research, and a conscious effort to make the class fun, that I can shift that distribution significantly. No student should leave the class confused. All students should be capable of understanding the research articles they are bound to encounter in upper-level classes in the political science major. A significant number emerge competent in their skills and prepared to deepen their understanding through further study.5

112

J. RING

Fig. 8.2 Hypothetical distributions of students before and after a successful research methods course

Notes 1. This is largely consistent with, but somewhat more optimistic than, Earley’s literature review of teaching research methods articles, which tend to describe students as unprepared, anxious, uninterested, and with poor attitudes as well as coming to the class with “misconceptions about research” and “[failing] to see the relevance of the course to their major and their lives” (2014, 245–256). When students are actually surveyed, the image of the monolithic “research reluctant” student body does not usually hold up. For instance, among social work students, a majority held positive views about learning research methods before taking their first research methods course (Secret et al. 2003). 2. See the oral arguments for Gill v. Whitford (2017) https://www.oyez.org/ cases/2017/16-1161. 3. A recent study estimates that around 4% are innocent (Gross et al. 2014). 4. The debate about how applied research methods should be has been going on for decades. Rodgers and Manrique (1992, 236) asked “how much computers? How much statistics? How much methods?” and concluded that “students benefit more when computing and statistics are an integral part of a research methods course.” While I agree with the importance of computers and statistics in general, the question of how applied that practice should be is unsettled. 5. However, as Murtonen (2015) demonstrated through an analysis of students’ mind maps before and after teaching research methods to education masters students, confused understandings of the relationship between the concepts of “theory,” “empirical,” “qualitative,” and “quantitative” still existed in more than half of the students. Therefore, my goal should be seen as aspirational, not descriptive.

8

THE SAVVY CONSUMER …

113

References Aguado, N.A. 2009. Teaching Research Methods: Learning by Doing. Journal of Public Affairs Education 15 (2): 251–260. Bachner, J. 2020. Refocusing Methods Courses on Workplace Applications. Journal of Political Science Education 16 (2): 257–261. Cohen, A.H., J. Alden, and J.J. Ring. 2019. Gaming the System. New York: Routledge. Earley, M.A. 2014. A Synthesis of the Literature on Research Methods Education. Teaching in Higher Education 19 (3): 242–253. Frankfurt, H.G. 2005. On Bullshit. Princeton, NJ: Princeton University Press. Gross, S.R., B. O’Brien, C. Hu, and E.H. Kennedy. 2014. Rate of False Conviction of Criminal Defendants Who Are Sentenced to Death. Proceedings of the National Academy of Sciences 111 (20): 7230–7235. Murtonen, M. 2015. University Students’ Understanding of the Concepts Empirical, Theoretical, Qualitative and Quantitative Research. Teaching in Higher Education 20 (7): 684–698. Paulos, J.A. 2001. Innumeracy: Mathematical Illiteracy and Its Consequences. New York: Hill & Wang. Rodgers, P.H., and C. Manrique. 1992. The Dilemma of Teaching Political Science Research Methods: How Much Computers? How Much Statistics? How Much Methods? PS: Political Science & Politics 25 (2): 234–237. Roeder, O. 2017. The Supreme Court Is Allergic to Math. Five Thirty Eight (blog), October 17. https://fivethirtyeight.com/features/the-supreme-courtis-allergic-to-math/. Rosen, A. 2011. Playing with Blocks. Active Learning in Political Science (blog), August 1. https://activelearningps.com/2011/08/01/playing-with-blocks/. Rosen, A. 2016. Zendo Revisited: A Simple Methods Game for Large Classes. Active Learning in Political Science (blog), February 24. https://activelearni ngps.com/2016/02/24/zendo-revisited-a-simple-methods-game-for-largeclasses/. Secret, M., J. Ford, and E. Lewis Rompf. 2003. Undergraduate Research Courses. Journal of Social Work Education 39 (3): 411–422.

CHAPTER 9

Zen and the Art of Teaching Methods Without a Methods Course Verónica L. Reyna

The dissertation was defended on a Friday, and work at the community college started that Monday. I am still thankful to the graduate student union and the Center for Research on Learning and Teaching for training us R1 graduate students to apply to teaching colleges to widen our job options during the economic recession. Contrary to how I was trained in graduate school, I was socialized by my family to return home and to give back to my community. I actively sought out Hispanic-serving, teaching colleges. Armed with Paolo Friere and an R1 degree, I arrived at class on that first Monday, ready to mobilize and to amplify community power! Upon arrival, however, the department-approved textbook was placed in one hand; with the other hand, I shelved the dissertation, political science and methods, and Friere. I stepped into a box in which to teach meant to use the textbook and to test textbook information.

V. L. Reyna (B) Department of Government, Houston Community College, Houston, TX, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_9

115

116

V. L. REYNA

There are two things to know about some community colleges besides the stereotypical, heavy teaching load: many are limited in the types of courses they can offer, and many are open enrollment to be as accessible as possible. Most do not offer methods or research design courses; these courses are left to four-year universities. Because of the wide variation in student academic preparedness, standardized textbooks are considered a middle ground to control college costs, mitigate different reading levels, and other learning needs. At my college, we can only use the approved textbooks for our courses and cannot require students to buy other materials. Method courses are not an option. It took years for me to learn the fundamental benefit of educating about methods–thinking systematically about politics and government or methodological thinking. The journey to figure out how to do this began when I gained enough confidence to throw out the textbook and remember why I loved political science.

The Journey The insecure, teaching novice in me buckled under the state-approved curriculum and the ease of textbook publisher materials. For the first five years of my career, I did everything I was supposed to do and was rewarded accordingly: I started where students were developmentally; I challenged them with critical thinking through different types of assessments; I made sure to vary assignments and exercises for different learning styles; and, I was engaging them in the details of government that they were never taught before. The student evaluations were consistently great, and I was often nominated for teaching awards. Students were used to textbook-dependent learning, thus making courses predictable. This remedial civics approach to government and political behavior was an equilibrium for all involved. But that balance began to shift. I became restless with following the linear nature of textbooks. I began organizing my syllabus around policy issues but still incorporated the required textbook chapters. Students responded: learning about criminal justice or environmental policies gave meaning to the institutions, but they needed help understanding the complex nature of political problems. None of this is in the textbooks. I had to supplement with more data, history, and teaching support for students with variable math backgrounds (or none) to understand the data. When they asked “why” questions, I fell back on the methods in which I was trained: complex adaptive systems (CAS). However, I had

9

ZEN AND THE ART OF TEACHING …

117

to refine it to a cursory, theory-only overview of CAS to be palatable to students. Initially, I never told them what they were learning and only brought up the way of thinking when student questions called for CAS; it was just a way of thinking about problems in the world and not formally a part of the curriculum. Then, a year or so before the 2016 elections, students began to vocalize growing political concerns during campus speaker events: concerns about deportation, concerns about not knowing a more inclusive U.S. history, and concerns about not knowing how to fix the wicked problems speakers presented to them. “But what can we do about this?” was becoming an increasingly angry mantra. This was a call to action. Listening to students express frustration compelled me to reflect on how I was teaching my courses. Where was the disconnect? Why are they not connecting what they are learning in their government courses to real life and to their participation? We make assumptions about unspoken fundamentals in political science, and these assumptions materialize in absences in the curriculum and textbooks. One assumption is students understand power: what it is, how it looks, and why it works. Another assumption is students understand how a democracy is supposed to work. Experience educates my students in perceived powerlessness and the failure of democratic institutions. For others, participation is tantamount to a temper tantrum, complaining, and blaming the government. The assumptions are magnified in diverse classrooms and increasing numbers of first generation and international students. I realized that even after a semester of a government and going through an entire textbook, students do not connect the dots to hold the government responsible for policy outcomes. They learn about participation but do not understand the power behind why it works, and therefore, why it is vital to their lives and a high-quality democracy. With commoditization of textbooks, most are essentially the same–almost apolitical and antiseptic–making the silence of these assumptions normalized. They are exclusive of my students’ histories and political realities. Mixing up the textbook chapters with policy case studies was not enough to prevent perpetuating these social injustices. It was then that I stopped teaching and began to educate. Identifying what students needed to better understand their political power to create a democracy they wanted, I circled back to my original intent of why I wanted to be a political science professor in the first place. Students need to learn how to use their power, why democracies work the way

118

V. L. REYNA

they work, and all the different ways to implement their civic power in a democracy. Most textbooks do not offer these insights; most courses do not offer civically engaged learning. Central to this renewed effort was educating about how to think systematically about wicked, political problems. And that is the skill of methodological thinking in political science.

Career Kismet What is the point of teaching research methods and how do you do it– especially when you were not hired to teach methods or are not permitted to teach a methods course? Answers presented themselves to me over the course of the 2016–2017 academic year. After realizing the gap between what students need in order to be truly empowered and how textbooks guide us to usually teach introductory government courses, I began to reinvent my courses to embrace educating students in how to think about politics, policy, and civic engagement. I started here: if the purpose of the methods we use in political science is to systematically study political phenomena, can this purpose and way of thinking be communicated to empower students without math, proofs, interviews, focus groups, or computer modeling? While I utilized CAS and agent-based modeling for fieldwork, I had not thought about teaching even the theory of CAS in almost a decade and only dabbled in presenting concepts when needed. Intuitively, however, I knew that CAS would be a good place to start to answer some of the above questions. In the summer of 2016, I began dusting off old books and catching up on the evolution of the field. I scoured CAS research to identify how they asked questions about political problems and how they applied CAS concepts (Stroh 2015). In particular, I looked for guidance from authors applying CAS, and other concepts like intersectionality, to policymaking, campaigns, mobilization, and business. As a companion to focusing on systems thinking as a tool, I also decided to make civic engagement prominent in my courses to create opportunities for students to apply systems thinking and to develop skills to use their political power. I also started a Center for Civic Engagement for my college to address the concerns expressed by the student body writ large. Without funding or any other guidance, I was granted permission from my dean and began researching what other college centers do. I

9

ZEN AND THE ART OF TEACHING …

119

had already been organizing events and speakers for the college; a Center would make things organized and official. In the fall of 2017, we started off the school year experiencing and recovering from Hurricane Harvey. This crisis put those of us not dealing with flooding into organizing mode to assist our students and colleagues who were needing basic necessities. Because of the Center for Civic Engagement, I was approached by a colleague newly charged with a design thinking program for Early College high school students, the IDEAS Academy. He was working on two projects. One project was designing a website to coordinate post-hurricane volunteers with students in need of shelter, food, and transportation information, and I knew about local government and social welfare policies. The second project was working with the Mayor’s Office of Innovation to work on a challenge to address the city’s food deserts, and he knew I was trained in CAS and systems thinking. Once we resumed on-campus instruction, I presented the political problem to my students and incorporated my colleague’s civic effort into my existing courses. I experimented with my Honors College students first because I work with them a whole academic year. First, we tackled hurricane recovery in their state and local government course. The emergent phenomenon was college success, and students dove into learning about all the actors, interactions, and outcomes that support and inhibit college success, and how hurricane recovery now factored into the model. From this, they prioritized student needs and connected these needs with government policy and programs. This information was given to my colleague to put on the website. The next semester we focused on policy innovation to address local food deserts. For the IDEAS Academy, I guest lectured on systems thinking, walking students through exercises to think about all the actors involved with the policy problem. They created what were essentially CAS models of political actors and their interactions leading to food deserts. Then they theorized about creating negative feedback loops to decrease the amount of food deserts in our area given how actors interact. I also supported the IDEAS Academy with information about ethnographic fieldwork skills, ethics, and other qualitative methods to help them gather and analyze information. Students spent the rest semester developing policy ideas and designing products for the community to use. Overall, my role was to help cultivate the skills students needed to juggle the interacting parts of a wicked policy problem; in turn, students were able

120

V. L. REYNA

to re-envision policy solutions. Those skills were rooted in systematic, methodological thinking. This partnership with the IDEAS Academy and the Mayor’s Office of Innovation lead to more projects in subsequent years: identifying refugee student needs and creating recommendations for the college; understanding the causes of the Census undercount in our city to develop a youth-led, community-specific counter strategy; and, identifying the politics of environmental policy, focusing on policy solutions for reusing or decreasing plastics use. Through these types of projects, students learn how to systematically think about complex political problems in their communities and how to use their political power to engender policy change. This approach of merging CAS with civically engaged research (CER) in my courses provides a new, fulfilling equilibrium.

A Pedagogical Marriage I do not have all the answers. I do not control the information the students discover. The conversation that ensues in the classroom includes my learning alongside them as their systems models of the world unfold. Their attempts at policy advocacy yield an infinite array of results about which I only have existing political science research to guide me. Students often have to be encouraged in this active way of learning, supported in being in the driver’s seat of their own education about politics, policy, and their role in democracy. They rarely have had the opportunity to be the subjects of their own education. Merging education about CAS, systems thinking, and other methodological tools with civically engaged learning reflect my belief that the classroom is a location of social justice praxis. As such, my experimentation with teaching methods without teaching a formal methods course brings me full circle back to Paolo Freire’s pedagogical approach and purpose of education (Freire 1998, 1995). In my community college, students are the city’s residents. The “community” is in our classrooms and is not an abstract concept outside the walls of academia. It is important to me that my incredibly diverse classroom has an immediate and applicable purpose for learning about political science thinking. In Teaching to Transgress, bell hooks writes: When I first began college, Freire’s thought gave me the support I needed to challenge the ‘banking system’ of education, that approach to learning

9

ZEN AND THE ART OF TEACHING …

121

that is rooted in the notion that all students need to do is consume information fed to them by a professor and be able to memorize and store it. Early on, it was Freire’s insistence that education could be the practice of freedom that encouraged me to create strategies for what he called ‘conscientization’ in the classroom. Translating that term to critical awareness and engagement, I entered the classrooms with the conviction that it was crucial for me and every other student to be an active participant, not a passive consumer….Freire’s work affirmed that education can only be liberatory when everyone claims knowledge as a field in which we all labor. That notion of mutual labor was affirmed by Thich Nhat Hanh’s philosophy of engaged Buddhism, the focus on practice in conjunction with contemplation. His philosophy was similar to Freire’s emphasis on ‘praxis’--action and reflection upon the world in order to change it. (hooks 1994)

hooks describes the fundamental pedagogy I draw upon to impart methodological thinking to students for making sense of their political realities. They can simultaneously use these tools to be active participants in creating an inclusive democracy that addresses their community’s concerns. The classroom itself is more democratic as I claim no ownership over information nor pretend to be “the sage on the stage.” Education is a conversation between students’ experiential knowledge and knowledge discovered during the semester. Methodological tools or thinking are taught with the expectation that students use them beyond the classroom to teach others in their lives. The purpose of educating about methodological thinking is to better understand politics, government, power, policy, etc., and to utilize this skill to empower themselves and others. Civic engagement in the classroom is an active and applied learning opportunity to marry to students’ growing political understanding. My courses that incorporate civically engaged pedagogy also provide an opportunity for my students to internalize and to consolidate their civic power. The Association of American Colleges and Universities’ list of high-impact practices (HIPs) includes several characteristics of most civically engaged practices (Kuh 2008). The AACU’s perspective is that HIPs are more than experiences for students; rather, they are tools for deep learning. When these tools are used in tandem with one another, they are incredibly powerful learning opportunities, especially for minoritized and underserved students (Finely and McNair 2013). My students’ experiences corroborate this research. Many continue their civic engagement after their course, comfortable with their power, excited to use it, and

122

V. L. REYNA

wanting to empower others. Moreover, they are learning marketable, employable skills. The American Political Science Association (APSA) has supported civically engaged pedagogy for many years. APSA’s State of the Profession Series published Teaching Civic Engagement: From Student to Active Citizen that is a resource I use for discovering best practices for adding civic engagement to courses (McCartney et al. 2013). Recently, the APSA Taskforce on Civically Engaged Research (CER) was created to better support CER in the discipline by considering training for faculty and graduate students, tenure recommendations, and research recognition. One of the opportunities I participated in is the Institute for Civically Engaged Research which trains faculty and graduate students in conducting CER, including adding CER to courses. CER is also considered a HIP, and a perfect location for educating about methodological thinking and techniques, even if students are not learning quantitative or qualitative techniques from first principles in a full-fledged methods course. Another source I use is Campus Compact, an organization dedicated to supporting civic and community engagement in colleges and universities, and their definitive resource The Craft of Community Engaged Teaching and Learning: a Guide for Faculty Development (Welch and Plaxton-Moore 2019). When students are able to apply their methodological skills to community concerns they (or a community partner) define, there is a purpose for the methodological skills they learn: to empower themselves in our democracy and to write their own realities. Learning how to participate throughout the policymaking process educates about power and how that power influences institutions. There is a social justice element, too, when minoritized students become subjects in their own education, communities, and democracy (Adams et al. 1997). They learn they belong in spaces of decision-making. Merging methodological thinking with CER has become the fundamental pedagogy driving my course design and is integrated into all my courses, from Introduction to American Government and State and Local Politics, to Mexican American/Latino Politics. In considering what not to do, courses cannot be tied 100 percent to textbooks. They are only resources, and in reality, professors no longer control information. Instead, our job is to guide using and understanding that information. Students must apply methods in order to best learn these skills in a transformative way. Another lesson learned is the

9

ZEN AND THE ART OF TEACHING …

123

beauty and utility of backward design, focusing on the three most important concepts and not the kitchen sink (which is often the tendency when using textbooks and publisher materials for the typical introductory course). What are the critical concepts with which you would like students to leave? If you only have 20 minutes to teach a lesson, on what would you focus and why? Finally, do make sure you are well versed in the ethics of CER, which often go beyond what the Institutional Review Board recommends.

A Project Excerpt and Takeaways Students have experiential knowledge and are experts in their own communities. Their trust in their knowledge can be reinforced with the following exercise as they are introduced to key concepts in CAS and systems thinking. Following an overview of CAS concepts like agents, emergence, interaction, adaptation, and feedback loops, students can apply the concepts to a real community problem. The goal of the exercise is to brainstorm about all the political actors, institutions, levels of government, and individuals with vested interests in a community problem, and to build an intuition for how the system works and adapts. Students should build a diagram of a CAS model. There are many examples on the internet of generic, CAS models. A graded assessment could include students’ analysis of the diagram or a class presentation walking peers through the system’s characteristics and dynamics. The first step is to discuss with students what they identify as dire community concerns. Prioritize one specific issue (the more specific the better) that students can analyze for purposes of the class exercise. An alternative to full-class participation is to break up students into small groups, each working on a different issue, in order to provide more options. A hidden gem in this discussion is teaching students skills to deliberate about policy priorities or to work through differences about what is defined as a community problem. In short, they are learning to define and to express their political interests. Once an issue has been chosen, students write down all the political actors, institutions, governments, etc. that are involved in the politics of that issue. Guide students with questions if they need help discovering more actors involved. Also have students consider partisanship, ideologies, political culture, and local, domestic, or international contexts. Next, have students consider how all these variables or agents interact with one

124

V. L. REYNA

another giving rise to the emergent community issue. Guide students with questions to have them consider feedback loops–what interactions make the issue worse or better, or what makes the issue more entrenched? As students add interactions to their diagram, it begins to look like a football playbook. Continue guiding students with questions to better learn the dynamics of the system. What will happen if this actor does X? How will Y react? What does this do to other actors, the system, and the community issue? Do not forget to discuss concepts like the power of different actors and subsequent variation of influence on the system. One tip is to do this on a dry erase board or devices that allow you to draw and erase easily without losing an original, base diagram. Finally, move students to think about negative and positive feedback loops with the goal of brainstorming possible policy solutions. What ideas might disrupt feedback loops to decrease the community problem? How would the system and actors react or adapt? Would many policies be necessary to address the issue? Would they need to be implemented linearly or simultaneously? This part of the exercise introduces students to possible policy ideas to address their community issue. Encourage thinking outside the box to innovate on new policy ideas! An important civic skill is to identify problems and work around them with creativity. A longer-term project would be to take the information from this exercise and tie it to policy advocacy and political participation opportunities. This is where civic engagement and CER can be introduced. If a policy idea is identified as beneficial to the community issue, students can examine participation strategies to get it passed into law, or research what other communities have done to ameliorate similar situations. Consider how much the policy would cost and how it would be funded. Anticipate political opponents and devise strategies to deliberate or compromise or challenge. If a policy already exists but needs to be enforced, what are ways students could participate to influence this enforcement? Civic engagement opportunities and CER can take on a number of variations in a course. Consider feasibility, costs, access, and transportation issues for students, too. The takeaway from this exercise in methodological thinking is to impart the way of thinking about a complex political phenomenon that truly impacts students’ communities. These are tools to understand their worlds. They are subjects in their learning by prioritizing their concerns and their knowledge. At the same time, they are learning how to democratically deliberate, how to define their political interests, and how to

9

ZEN AND THE ART OF TEACHING …

125

view other actors’ political behavior. When civic engagement or CER is added to methodological skills, students are empowered with how to champion their interests and influence government actors throughout the policymaking process. Overall, students learn their own power through skills necessary for policy innovation and advocacy to create their own twenty-first century democracy.

References Adams, M.L., A. Bell, and P. Griffin, eds. 1997. Teaching for Diversity and Social Justice: A Sourcebook. New York: Routledge. Finely, A., and T. McNair. 2013. Assessing Underserved Students’ Engagement in High-Impact Practices. Washington, DC: Association of American Colleges and Universities. Freire, P. 1995. Pedagogy of Hope: Reliving Pedagogy of the Oppressed. New York: Continuum. Freire, P. 1998. Education for Critical Consciousness. New York: Continuum. hooks, b. 1994. Teaching to Transgress: Education as the Practice of Freedom, 14. New York: Routledge. Kuh, G. 2008. High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter. Washington, DC: Association of American Colleges and Universities. Rios Millett McCartney, A.E., A. Bennion, and D. Simpson. eds. 2013. Teaching Civic Engagement: From Student to Active Citizen. Washington, DC: American Political Science Association. Stroh, D.P. 2015. Systems Thinking for Social Change: A Practical Guide to Solving Complex Problems, Avoiding Unintended Consequences, and Achieving Lasting Results. White River Junction: Chelsea Green Publishing. Welch, M., and S. Plaxton-Moore. 2019. The Craft of Community-Engaged Teaching and Learning: A Guide for Faculty Development. Boston: Campus Compact. Verschelden, C. 2017. Bandwidth Recovery: Helping Students Reclaim Cognitive Resources Lost to Poverty, Racism, and Social Marginalization. Sterling: Stylus Publishing.

PART II

Teaching Research Design

CHAPTER 10

Building Qualitative Methods Skills Through Research Design Jessica Hejny

Introduction I was not trained as a methodologist, nor was I hired to teach political science methods, but during my first year as an assistant professor of political science at Middle Tennessee State University (MTSU), I stepped in to fill a teaching need in undergraduate research methods. My methodological training as a graduate student in political science was, I think, atypical. Though I took an obligatory two-semester sequence of statistics, my department skewed heavily towards qualitative methods, including historical analysis, discourse analysis, and ethnography. Though graduate seminars dedicated to these methods were rarely offered, mentorship and field seminars focused on qualitative research. Alongside this training in qualitative methods, I developed an interest in the meta-methodological debates (e.g. positivism vs. interpretivism) that underpin methodological divisions within political science. Thus, my interest in research methods

J. Hejny (B) Albuquerque, NM, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_10

129

130

J. HEJNY

emerged not at the level of particular methods, but rather around questions of how different methodological approaches produce knowledge. When I began to develop my undergraduate research methods course, I knew I wanted to center qualitative and interpretive methods, but I felt like I was in uncharted territory given that most undergraduate methods courses are introductions to statistical analysis, a subject that I neither felt qualified nor desired to teach. Thus, in creating my research methods course, I immediately encountered two quandaries. First, was it possible to introduce the spectrum of political science methods to students who had not had an ounce of prior methods training? Was I going to throw them into the deep end of the pool without a life jacket? I also knew that I wanted to teach students skills that they could carry into other courses and into their careers. But how would I be able to teach them both the spectrum of political science methods and research skills in one semester? Could I fit both substantive knowledge and skill-building into one course? After weeks of research on how other instructors teach political science research methods along these lines, I came up with few models to work from. So, I started from scratch. There is one additional context to note before I describe how I created my course—I had to take into account the student population of my university. Our students come to MTSU with a wide range of academic preparation. Some struggle with basic reading comprehension and writing, while others need only modest guidance and hit the ground running. Most have never encountered an academic journal article, and if they have, have not been instructed on how to understand it. In our department, research methods is regarded among students as an intimidating course. Indeed, if we did not require it for our major, I’m not sure that any students would voluntarily take it given its reputation. So, on top of the inherent challenges of teaching research methods at the undergraduate level, I was dealing with a student population with varied training who dreaded taking my course. This context intensified my existing anxieties about developing the course—I had to think about teaching this complex material in an accessible and fun way in order to get student buy-in and keep them engaged. In designing the course, I knew I wanted to focus on teaching students how to think like an academic researcher. This includes the abilities to formulate interesting research questions and to select appropriate methods to answer these questions. Rather than centering the course on developing in-depth skills in one particular method, I chose to have

10

BUILDING QUALITATIVE METHODS SKILLS …

131

students learn the steps of the research process, while at the same time becoming more comfortable reading and engaging with scholarly literature. I introduce students to different ways of thinking about knowledge production through a comparison of the methods that we cover in the course—statistical analysis, single case studies, comparative case studies, historical methods, and ethnography. We read texts that introduce the basics of political science research (Baglione, Writing a Research Paper in Political Science) and methods (Howard, Thinking Like a Political Scientist ), as well as peer-reviewed scholarship. This learning process culminates in the production of a research proposal for which students develop their own research questions, locate relevant literature, produce a literature review, and create a research design. I chose to focus on these two essential research skills—writing a literature review and creating a research design—because they are foundational skills in academic research and their component skills have a wide application in professions that students might pursue.

Building the Course When I was putting together the course, I had a clear vision of what I wanted the course to accomplish—build basic research skills, teach students how to think like social scientists, and introduce a range of political science methods with a focus on qualitative methods. The first practical challenge I encountered was finding a textbook that matched my aims. The vast majority of undergraduate political science research methods textbooks are introductions to statistics. Luckily, a colleague recommended Christopher Howard’s Thinking Like A Political Scientist, which I used to structure the first half of the course. My first goal in designing the course was to provide students with basic research skills. We begin the course with a primer on locating, evaluating, and citing scholarly sources. This primer includes an introduction to our library system (conducted by our subject librarian), an explanation of peer-review, and a discussion of plagiarism and proper citation. In moving from these general research skills to the specific assignment in our course, I introduce the guidelines for our research proposal project and have students read two example proposals from former students in the course. Beyond being able to find relevant literature, students must have the ability to comprehend academic texts. We spend a day learning to decode academic articles by breaking down articles together in class.1 This

132

J. HEJNY

is a crucial class period because it demystifies scholarly work by revealing the common structure of journal articles. Once students can identify this structure, they feel much more confident embarking on their literature search. My second goal was to teach students how to think like social scientists. This is where Howard’s book was particularly useful. He frames his book using a construction metaphor. Rather than aiming to produce expert “home builders” (a tall order indeed for a one-semester undergraduate methods course), Howard crafts a guide to “careful home inspecting.” He introduces central research concepts and practices, like operationalization of variables, case selection, and the distinctions between correlation and causation, normative and non-normative claims, and descriptive and causal claims. Howard’s book provides students with essential tools to both evaluate scholarly literature and develop their own research proposals. My third goal was to expose students to a range of political science methods. Most undergraduate political science methods courses focus on basic statistics.2 In my experience of taking statistics, one semester is not sufficient to impart usable skills, particularly if you do not immediately apply the skills in other courses. Fear of statistics also seems to be the primary driver of student trepidation around research methods in my department. However, given the prevalence of statistical methods in political science, I did see a need to teach students the basic logic of quantitative analysis and how to comprehend academic texts that use statistics. So, I positioned quantitative analysis as one of several methods we examine in the course. Before introducing students to specific methods, we first explore political science research design. Using Howard’s chapter on research design as our text, we examine large-n, small-n, and experimental designs, and compare their strengths and weaknesses with particular attention to internal and external validity (all of which can helpfully be put into a chart and given to students as a handout). Subsequent classes are devoted to particular research methods. We spend at least two classes on each method. The first day provides the basics of the method, and on subsequent days we read academic articles that exemplify this method in action. We begin our survey of methods with quantitative analysis, using Howard’s chapter (“Using Numbers As Evidence”) as our foundational text. We then spend three classes breaking down academic articles that use

10

BUILDING QUALITATIVE METHODS SKILLS …

133

statistical analysis. We start with a relatively simple quantitative analysis so that students can focus on the key elements in constructing a statistical analysis and interpreting a regression table.3 We carefully walk through the methods section, the findings, the table results, and the conclusion of each article, paying particular attention to how the analysis is structured (population/sample, variable operationalization, hypotheses) and how the regression results are interpreted by the author(s). Once students have gotten comfortable with these basics, we tackle more challenging articles.4 Here, students have to contend with multiple hypotheses, multiple ways of displaying data analysis, and multiple regression models. By the end of these four classes, students have gained the basic skills of “inspecting” statistical analysis and setting up a quantitative research design. In introducing students to qualitative research, I showcase the diversity of methods that fall under this approach, including case study analysis, historical analysis, and ethnography. We begin with small-n case studies after reading Howard’s chapter on case selection (“Choosing Cases”). We then move on to historical analysis and ethnography. Here I had to look beyond Howard’s book given his limited discussion of qualitative methods. Ethnography is particularly challenging to teach in an undergraduate research methods course because it requires explaining the distinction between interpretivism and positivism. I teach ethnography last for this reason; by this point in the class, the students have had ten weeks of methods training and have a foundation to understand this distinction. Including discussion of ethnography and historical analysis is fundamental to my goal of providing students with an expansive view of political science methods. In examining single and comparative case study analysis, we focus on case selection and methods of data collection and analysis. We attend to the ways in which case selection is more complicated and critical in comparative case studies. Comprehending the logic of case selection is one of the more challenging parts of the course for students. Working through two different comparative case study articles in class helps them to understand why particular cases are chosen and what analytic purchase is gained from these comparisons.5 The example articles introduce students to the qualitative evidence gathering and evaluation techniques of interviews, surveys, textual analysis, and discourse analysis. For historical methods, we read an article that demonstrates how to utilize archival research to collect evidence.6 I take the opportunity during

134

J. HEJNY

this part of the course to explain my own research approach (American Political Development) to give students a firsthand perspective on historical research. I highlight the crucial roles that process tracing and triangulation of historical evidence play in historical analyses. For our discussion of ethnography, we read a text that explains how ethnography differs from other qualitative methods in political science, and, in the next class, focus on a reflection of an ethnographer on their research process.7 To help students understand the unique features of ethnographic practice, I highlight the participatory aspect of ethnography, the lack of distance between researcher and research subject, and the importance of the researcher’s reflection on their positionality. After beginning our study of methods with quantitative analysis and its rigid parameters, the shift to ethnographic thinking can be challenging for some students. But others are attracted to the immersive, interpersonal character of ethnographic research. Throughout our exploration of methods, I emphasize that these methods are not mutually exclusive. A researcher may choose to utilize both quantitative and qualitative analysis to gain more analytical purchase, or they might incorporate participant observation with more traditional qualitative methods, like interviews, in a case study analysis. After working through the spectrum of methods and accompanying example articles, students better understand how to select methods appropriate to specific research questions. This skill both informs their own selection of a method for their proposal and makes them more adept inspectors of scholarly literature. The research proposal brings all these elements of the course together. This assignment allows students to apply their research skills, to think like political scientists in developing a research question and evaluating relevant scholarly literature, and to design a study using one of the methods that we covered in the course. While it would be more enriching to have students also conduct primary research and write up their analyses, one semester does not provide enough time to both teach students the necessary skills and have them complete a full research project. The proposal is sufficiently challenging given that this is likely the first project utilizing academic research skills that students will have encountered. To ease students into this new form of writing and research, the proposal is broken down into elements that build on one another. Students first submit a research question, and, once approved by me, they find relevant scholarly sources and write an annotated bibliography. The

10

BUILDING QUALITATIVE METHODS SKILLS …

135

annotated bibliography gives me a chance to evaluate whether students are finding adequate and pertinent scholarly sources, as well as their ability to comprehend these texts. If students are struggling with either of these skills, I can intervene and assist them. Students then produce a draft of their literature review, followed by a draft of their research design. I provide comments on both of these drafts with the expectation that they will be significantly revised for the final version of the proposal. The final draft is graded with an eye towards the improvements that students have made in their revisions. In the final proposal, I look for evidence of proficiency in the three skills I outlined above. Students should be able to demonstrate: (1) basic research skills by locating relevant scholarly literature, writing up their research in an organized structure, and following proper citation conventions, (2) ability to think like a political scientist by selecting a feasible and interesting research question and evaluating and engaging with existing literature on their topic, and (3) their knowledge of political science methods by selecting a method appropriate to their research question and generating a practicable research design. I find that the advantages of structuring the course around these three aims are manifold. First, students gain transferable research skills— including evaluation of sources, engaging with scholarly literature, and project planning—that will help them to succeed in other courses and in their future careers. Second, pushing students to think like social scientists empowers them beyond the classroom. Practicing critical reading and engagement with academic arguments develops requisite skills of informed citizenship, like information literacy. Third, students leave the political science major with a deeper understanding of the discipline and of academic research, which aids them in determining whether they are interested in pursuing a graduate degree. One final note on course design, there are two additional elements that I cover at the end of the course: research ethics and political theory. Surprisingly, research ethics does not figure prominently in most political science research methods textbooks, including Howard’s, but it is an essential part of methods pedagogy. I ask students to address in their proposals any ethical issues that they foresee arising in their proposed research.8 Finally, political theory may seem like an odd addition to a research methods course, but I want students to understand how argumentation in political theory differs from empirical political science. I assign a short political theory piece and ask students to summarize the author’s argument and identify the methodology.9 This task stumps

136

J. HEJNY

students, but opens a discussion of the difference between normative and non-normative arguments and the methodology of normative claimmaking that augments their understanding of the place of political theory in the discipline.

Reflections on Practice In this section, I reflect on some of the specific pedagogical practices that I employ in my research methods course. First, my course is discussionbased, which is only possible because my department caps the size of the course. The small size allows me to provide students with individual attention. Twice in the term I have mandatory one-on-one meetings with each student—first to discuss their research question and then their research design. I provide extensive feedback on drafts of literature reviews and research designs, which improves the quality of the final proposals. These hands-on practices may not be possible with a larger class size, but I have found them to be essential in keeping students engaged and on track. Students are required to complete the assigned reading before class to ensure that we have a productive class discussion. To enforce this requirement, I devised a mechanism that ended up yielding a double dividend. Instead of quizzing students or assigning reading questions, I require them to underline or highlight the readings and make notes in the margins as they read. I teach students how to annotate academic texts and provide an example of an article that I have annotated. Every class period I check students’ readings for these annotations. Not only does the graded nature of this requirement induce students to come to class prepared, but it also improves their reading comprehension and notetaking skills. I have actually had students thank me for teaching them to read so carefully! For the research proposal, students select both their topic and their method. This allows students to research something that excites them and will keep them interested in their projects, and it also compels students to do the hard work of choosing and refining a research question. They come to appreciate how challenging this process is—the question has to be interesting, feasible, not too broad and not too narrow, and not already fully answered. Freedom to select a method helps students to feel less boxed in if they are intimated by a particular method, and it also forces them to carefully think through the relationship between choice of

10

BUILDING QUALITATIVE METHODS SKILLS …

137

method and research question. They find that not all methods are appropriate for their research question, and indeed their question may be best suited to only one method.

Takeaways My first observation is perhaps an obvious one: enthusiasm is crucial. Research methods is just plain challenging. The course demands that students rapidly build and employ new skills. Keeping students engaged and feeling positive about their abilities requires modeling one’s own enthusiasm for the topic. I have found that being honest with students about the challenging nature of the material while simultaneously expressing encouragement is the key to a rewarding experience for both my students and me. My second piece of advice is to build into your course multiple opportunities for students to practice the skills you want them to learn. The more they practice critical reading, annotating academic texts, writing, paraphrasing, engaging with literature, and crafting a research design, the more their skills will improve and the easier the final proposal will be. Given the rapid accumulation of skills that this course demands, scaffolding assignments helps to prevent students from feeling overwhelmed and makes the full proposal much less intimidating. Additional hands-on research exercises that can be employed include taking students to one’s university collection to practice archival research and having students write a short ethnography of a campus event. Finally, I encourage all empirical political scientists to teach research methods if they have the opportunity. Research methods stands out among my courses because it is primarily skills-based. For this reason, I really enjoy the change of pace that this course offers. In addition, developing and teaching this course has made me a better researcher in ways that I had not anticipated. Having to explain political science methods in a way that is accessible to undergraduates forced me to reapproach methodology from the ground up. The thoughtful engagement with methods that designing this course required has lent a new clarity to the way that I articulate my own methodological approach and has also made me a more diligent critic.

138

J. HEJNY

Notes 1. I use Setzler and Yanus (2018) and Thelen (2018). 2. Turner and Thies (2009) surveyed instructors of political science research methods courses and found that 77% of a sample of 106 taught statistical software. The gap in qualitative methods training manifests in doctoral programs as well. In a survey of the top 25 political science doctoral programs, Emmons and Moravcsik (2020) found that 40% of programs offered no dedicated qualitative methods training despite the ubiquity of qualitative scholarship in the discipline. 3. Here we reread Setzler and Yanus. 4. I use Rosenthal (2020) and Fearon and Laitin (2003). 5. We compare Thelen (2018) and Bae (2011). We read Woodwell (2005) as our example of single case study analysis, which offers a rich contrast to Fearon and Laitin. 6. Here we read Weaver (2007). 7. For an introduction to ethnographic methods in political science, we read De Volo and Schatz (2004). For our example of ethnographic methods in action, we read Shehata (2006), which is both excellent and accessible. 8. For this class, we read Chapter 8 of Clark et al. (2019). 9. I use Tronto (1995).

References Bae, S. 2011. International Norms, Domestic Politics, and the Death Penalty: Comparing Japan, South Korea, and Taiwan. Comparative Politics 44 (1): 41–58. Baglione, L.A. 2018. Writing a Research Paper in Political Science: A Practical Guide to Inquiry, Structure, and Methods. Washington, DC: CQ Press. Clark, T., Foster, L., and A. Bryman. 2019. How to Do Your Social Research Project or Dissertation. New York, NY: Oxford University Press. De Volo, L.B., and E. Schatz. 2004. From the Inside Out: Ethnographic Methods in Political Research. PS: Political Science and Politics 37 (2): 267–271. Emmons, C.V., and A.M. Moravcsik. 2020. Graduate Qualitative Methods Training in Political Science: A Disciplinary Crisis. PS: Political Science & Politics 53 (2): 258–264. Fearon, J.D., and D.D. Laitin. 2003. Ethnicity, Insurgency, and Civil War. American Political Science Review: 75–90. Howard, C. 2017. Thinking Like a Political Scientist: A Practical Guide to Research Methods. Chicago, IL: University of Chicago Press.

10

BUILDING QUALITATIVE METHODS SKILLS …

139

Rosenthal, A. 2020. Submerged for Some? Government Visibility, Race, and American Political Trust. Perspectives on Politics: 1–17. Setzler, M., and A.B. Yanus. 2018. Why Did Women Vote for Donald Trump? PS: Political Science & Politics 51 (3): 523–527. Shehata, S. 2006. Ethnography, Identity, and the Production of Knowledge. In Interpretation and Method: Empirical Research Methods and the Interpretive Turn, ed. Dvora Yanow and Peregrine Schwartz-Shea, 244–263. Armonk, NY: M.E. Sharpe. Thelen, K. 2018. Regulating Uber: The Politics of the Platform Economy in Europe and the United States. Perspectives on Politics 16 (4): 938–953. Tronto, J.C. 1995. Care as a Basis for Radical Political Judgments. Hypatia 10 (2): 141–149. Turner, C.C. and C.G. Thies. 2009. What We Mean by Scope and Methods: A Survey of Undergraduate Scope and Methods Courses. PS: Political Science and Politics: 367–373. Weaver, V.M. 2007. Frontlash: Race and the Development of Punitive Crime Policy. Studies in American Political Development 21 (2): 230–265. Woodwell, D. 2005. The “Troubles” of Northern Ireland. In Understanding Civil War: Evidence and Analysis, ed. Paul Collier and Nicholas Sambanis, 161–189. Washington, DC: World Bank.

CHAPTER 11

Teaching Research Design with Authenticity Christina Fattore

Political scientists’ tendency to stress the importance of quantitative analytic skills in “good” research has led methods instructors to misplace their focus in their courses. This overemphasis on quantitative skills, instead of the process of designing research that logically proceeds to that need, confuses undergraduate students. Many of them were attracted to the major because of their own interest in some aspect of the political world: whether it is the American political process, global climate issues, or a personal interest in a certain section of the world. While, as scholars, we “know” there is a difference between political science as a field and politics, we could do a better job of communicating this to students. In my chapter, I discuss how my approach to teaching methods has changed over time. Instead of sustaining the idea that methods should be the hardest course and should focus mainly on quantitative analysis, I have recently designed a course where my role is a guide for students through

C. Fattore (B) Department of Political Science, West Virginia University, Morgantown, WV, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_11

141

142

C. FATTORE

the research process. Encouragement and authenticity are the main pedagogical tools in my class, which allow students to take their own interests and shape research around them.

Carrying the Mantle of What a Methods Class “Should” Be When I first taught undergraduate methods, I was still figuring out who I was as a teacher. I was just past my comprehensive exams and was dealing with high levels of imposter syndrome. If I didn’t trust myself to be an expert, how would my students believe it either? I was also the product of a graduate program that stressed the importance of research over teaching, which shaped my bias about the time I should devote to the classroom. So, I donned my armor of expertise: that is, I went into the classroom like I was going into battle. Research methods already had the reputation of being the hardest class within the political science major. I felt that it was up to me to uphold that by challenging my students, throwing various types of statistical analysis at them, and explaining it in a way that was way over their heads. They hated it, I hated it, and it was a miserable six-week summer session. I remember my department chair, who was also my dissertation advisor, telling me that my teaching evaluation scores were so low that I was technically on teaching probation with the dean’s office, but that was the norm for grad students teaching methods for the first time. Basically, he didn’t make a big deal of it, so I didn’t either and swore I’d never teach research methods again. Fast-forward a decade, and I was now a tenured associate professor at West Virginia University. I was asked to develop a specific course on research methods for our international studies (IS) undergraduate majors. The major is housed in the political science department, which means many of those IS majors end up in political science classes, specifically the research methods class. The problem was that so many of our students were failing that particular class and had to repeat it, much to their detriment. Through exit surveys, we gathered that IS students were not getting much value out of that approach. Therefore, our IS director asked me to design a class that was more of a mixed methods approach. I use a variety of methods in my own research, and IS is an interdisciplinary field. It had easily been 15 years since that disaster of my first research methods class. If I spent the time to build my own class from scratch,

11

TEACHING RESEARCH DESIGN WITH AUTHENTICITY

143

rather than relying on what previous instructors had done, I’d be able to stress the importance of other bits of the research process, not just statistical analysis.

Revealing the Authentic Me as a Pedagogical Tool In those 15 years, I had grown as a teacher and a scholar. I realized that I would never be that “ideal” of what a professor should be, and, perhaps most importantly, I recognized that ideal was shaped by decades of gender bias and privilege. I mentioned my “armor of expertise” earlier. It took time, but piece by piece, I began to trust my knowledge and the fact that I did belong in the classroom as an expert. But I did it on my terms as well. I am a bit scatterbrained and disorganized. Instead of trying to hide that, I tell my students about my ADHD. I use my young children as examples in the class when trying to explain a concept, rather than trying to cover up the fact that I am a mother. In other words, I have learned that authenticity is the antidote to imposter syndrome. I had been teaching undergrads long enough to know they did not have a clue about the research process. So, to paraphrase one of my favorite songs from The Sound of Music, I thought we should start at the very beginning, which is a very good place to start. The first day of class, I introduced myself and explained what I saw as my role in this class: I would be a guide through the research process and encourage the students to explore their nascent ability to do their own original research. I stressed that I knew they had never done anything like this before, nor had they ever taken a research methods class before. I put aside my immediate need to establish my expertise, and instead, shared my own journey as a researcher. My first goal was to get students to realize that learning is a life-long process. Every semester in every class, I tell my students the same thing: that if they were experts, they wouldn’t need to be sitting in my classroom. But also, my expertise is in a very narrow piece of the world. My writing has improved over time through feedback and practice. I stressed how long I had been in the academic world (currently, it has been almost twenty years since my first year in grad school!) and how they are in an undergraduate research methods class, the first they may have ever taken. As I mentioned earlier, the main pedagogical tool that I use in my research methods classroom is to be as authentic as possible. As women, we are socialized not to share “too much” with our students. I throw

144

C. FATTORE

that rule straight out the window. On the first day, I tell my students that I received a B- in my undergraduate research methods class and graduated with honors by the skin of my teeth. I tell them what I didn’t like about that class: that we spent way too much time learning about data and not the research process. Most undergraduates have no idea what the research process looks like in the social sciences. They have taken topical classes that reinforce “research” but they have never walked through the process themselves. While other professors may overemphasize the end of the research process by focusing on data analysis and the presentation of results, I emphasize the beginning of the process to my students in my research methods class. I created a specially designed course focused on international studies and the various methods available to students that would allow them to test hypotheses derived from their research questions. I very specifically chose to slow the research process down for undergraduates, allowing them to experience and digest each step of research design.

Teaching Research Design as a Process We spend the first few weeks of the class talking about what we think we know (epistemology) and how we gather knowledge. In this era of fake news and the death of expertise, I ask our social science librarian to start this conversation with a discussion of how you find reliable sources. He gives the students an in-class activity where they do an internet search of different sources and then need to explain why they may be biased and/or unreliable. I follow up this discussion with what “empirical” means (observable, not quantitative) as well as a discussion on intersectionality and how our background and identity affects how we interact and observe our world (Crenshaw 1989). I liken intersectionality to a lens through which we filter events and information, and how our various backgrounds and identities lead us to ask different questions and draw different conclusions than our peers. Since my main teaching goal was to have students feel confident about their own research abilities, I decided that the final assignment would be a research design rather than a research paper with complete results and discussion. The student’s final grade would be derived from the different parts of the research design: the research question, the literature review, the hypothesis, the research design, and the presentation. The class is organized around these various components. I would spend a few class

11

TEACHING RESEARCH DESIGN WITH AUTHENTICITY

145

periods discussing a topic (for example, how to come up with a research question). Then, the students would draft their research question and present it in class. We would spend the entire period workshopping each student’s research question, from which they would receive feedback. I would send them my more in-depth feedback (and preliminary grade) privately via email, and then they would have ten days to revise using those comments for a better grade. The revision portion of each assignment was designed to reflect the real-life research process. Scholars almost never have a paper accepted out right. Instead, we go through a process of revising and resubmitting for another review. I use my own working papers and publications as illustrations. At first, I was a little apprehensive about this. I know students can be critical of faculty using their own materials (especially textbooks) in class, but I was not asking my students to purchase anything. Also, there is so much about our jobs that our students do not understand, especially our first-generation students. It’s part of the hidden curriculum (Calarco 2020), and as a first generation student myself, I have an obligation to pull back the curtain on academic life (Smith 2013). I was pleasantly surprised by how my students reacted to this discussion. We went through the peer review process from beginning to end. I showed them a draft that I submitted to a journal, the reviews I received (and how some of them can be overly critical/mean), and how I revised my manuscript for resubmission. I also showed them emails I received requesting my review of a journal submission and what goes into that side of peer review. I then showed my students my curriculum vita, which was a big deal considering my imposter syndrome. I was amazed when one of my students said, “Dr. Fattore, you published all that?!?” It felt wonderful to be able to share that with my students. My vita should be a source of pride, not shame, and it took a student to say that to me to change my attitude about what I research and publish with students. After my students developed some working hypotheses, we then turned to a discussion of various types of research design. This is purposeful with two goals in mind. First, I want my students to recognize that the default research design should not be a large-N statistical analysis. Second, instead, I want them to spend time considering what design would best fit their question and best test their hypotheses. Our undergraduate students have already been shaped to think “numbers/stats equals strong evidence” and I wanted to challenge that bias. The strength of quantitative design was drilled into me through my undergraduate and

146

C. FATTORE

graduate training, but it was not until I was on the tenure track that I started to consider whether that was the appropriate approach to my own research questions. Sometimes it was, while other times it wasn’t, but I was never taught to stop and think about design. Therefore, I made a pedagogical decision to structure my course in such a way where students were constantly thinking about what type of research design would fit best for their project. Also, to make it fun, I allowed students to consider what they would study if they had an unlimited budget, which is every researcher’s dream. Would they travel to Tanzania to interview women on how their involvement in coffee production affects household relationships? Would they explore the polar ice caps to gather evidence to explain how the lack of international cooperation on climate change and governance of public territories leads to far-reaching levels of pollution? Students perform better when they study a topic that they find interesting rather than what is available to them. It also allows for creative thinking and analysis. Also, because the IS major is interdisciplinary, our students are able to see how quantitative analysis may not be the best fit if they are interested in international environmental issues or international development. An example: Last fall, I had a student who was interested in public health and had been involved in a number of medical service trips during her time at WVU. Her initial research question focused on the effectiveness of these trips. It was fun to talk to her as she worked through the many ways she could approach this question: should she concentrate on the amount of services provided by the visiting team? Look at life expectancy? Repeat visitors? Clearly, she had a bias towards a more quantitative approach to answering this question. However, after we discussed qualitative research designs, such as interviews, ethnographies, and narrative designs, she decided that she wanted to take a qualitative approach because who would better know about the effectiveness of these medical service trips than the people who are the target audience. It was fantastic to see her move in that direction as the semester went on and we discussed qualitative and quantitative designs. The culminating experience of the research design course is a public poster session that is on our schedule from day one. I won’t lie: some students are freaked out at the idea of presenting their work publicly, especially when we get to that part in the syllabus on the first day of class. Again, I remind them that learning is a process and that I am committed to getting them to the point where they feel better about presenting their

11

TEACHING RESEARCH DESIGN WITH AUTHENTICITY

147

research design. I stress this every time we have an in-class workshop, where they are sharing a new building block of their project within the group. Again, I invite a research librarian (this time, our digital archive specialist) to be a guest speaker and discuss how to make a good research poster. I share posters I have made in the past as well as posters that other political scientists have graciously shared with me. I also schedule an individual meeting with each student in the days prior to the public poster session so we can review their written research design as well as their draft poster. This allows me to nip possible problems in the bud before the public poster session. I make a very big deal out of the public poster session. I invite all sorts of people from our campus community, including our president, provost, and the dean of my college. Usually, they are too busy to stop by, but I hope to signal to my students that I believe in their abilities to the point where I want the most important people on campus to share in this with them. Sure, many students get nervous, but if I show them my confidence in their work, I like to think it rubs off onto them. I also have asked my program to order refreshments, like cookies and coffee. It provides more of a celebratory air for the session as well. While we have professors and grad students come in to see their work, I am always pleased when I see my students’ friends show up. Not only are they able to share their work with their friends but I am so delighted to see that they took the initiative to invite them. To me, that is the ultimate sign of pride in their work.

Conclusion In writing this reflection, I now realize I designed the class that I would have liked to take as an undergrad. It also illustrates how much I have changed as a teacher since I taught that first methods class as a graduate student. There is no reason to scare students into learning research methods or to carry on that mantle if it has already been established in your department. Years removed, I think about why I chose to teach my first methods class in the way that I did. I think it was because I didn’t know any better. I was 24 or 25 years old with a MS in political science under my belt, and I still didn’t feel prepared. I also felt like I was given a list of things to “do” and “not do” in the classroom: be the expert, but also approachable; act like you care, but be organized; dress formally, but not in a suit; and so on (see Dion 2008 as an example). This type of advice stifled me and increased my imposter syndrome. Obviously, at that

148

C. FATTORE

point in my academic career, I was concerned about student evaluation scores and how that would look in a job packet. But I realize that I spent close to a decade doing what I thought was expected of me, not what I thought should be done. That advice increased my imposter syndrome rather than helped me establish expertise in the classroom. As time went on, I became more comfortable in the classroom as well as in academia for a number of reasons. But one thing that helped me become a better professor was to present my whole, true self in the classroom (see Johnson and LaBelle 2017 for a larger discussion of authenticity as a pedagogical tool). Sure, I have a PhD in political science and a number of publications under my belt as well as a stack of current projects. But, I am also a mom, a wife, and a daughter. As an academic with ADHD, I am scatterbrained and easily distracted. A few months ago, one of my first research methods students (who is also one of my advisees) was telling me about how a certain professor intimidates her. I laughed because that’s what I imagine people say about me. She quickly reminded me that I brought my then-kindergartener to the first day of class with me (his first day was two days after my first day and we lacked childcare) and I had to tell him to hush a few times while I was talking. She said that was what made me real to her. Being authentic allows me the energy to be the best professor I can be rather than trying to uphold these general pieces of advice that have been passed down through “how to succeed in academia as a woman” articles and workshops. My students thrive on my support and encouragement through the research process. I believe this process goes above and beyond my normal job/teaching responsibilities because not only do I want students to engage with the research process, but also feel comfortable and confident in doing so. In my research design course, I am not just an instructor. I am also a cheerleader, a sounding board for ideas, a model of how to proceed in research, and someone who empowered a group of students to believe in their own ability to do research. That is the real me. This is not something forced, but instead, I designed a class that allows me to excel as a professor and also to show students that they do not need to be 4.0 students to “get” political science and international studies research.

References Crenshaw, K. 1989. Demarginalizing the Intersection of Race and Sex: A BlackFeminist Critique of Antidiscrimination Doctrine, Feminist Theory, and

11

TEACHING RESEARCH DESIGN WITH AUTHENTICITY

149

Antiracist Politics. The University of Chicago Legal Forum. Available at: https://chicagounbound.uchicago.edu/uclf/vol1989/iss1/8. Dion, M. 2008. All-Knowing or All-Nurturing? Student Expectations, Gender Roles, and Practical Suggestions for Women in the Classroom. PS: Political Science and Politics 41 (4): 853–856. Johnson, Z.D., and S. LaBelle. 2017. An Examination of Teacher Authenticity in theCollege Classroom. Communication Education 66 (4): 423–439. McCrory Calarco, J. 2020. A Field Guide to Grad School: Uncovering the HiddenCurriculum. Princeton, NJ: Princeton University Press. Smith, B. 2013. Mentoring At-Risk Students Through the Hidden Curriculum of HigherEducation. New York: Lexington Books.

CHAPTER 12

Research Design as Professional Development and Empowerment: Equipping Students to See, Analyze, and Intervene in Political Realities Kelly Bauer

I started my undergraduate education as a math and chemistry major, but like many (Hill et al. 2010), had an unclear image of what being a woman in STEM would or could be. Political science seemed to offer a path forward; as I envisioned, the field did a version of what women in my family did—work in helping fields with an abundance of ambiguity and complexity—by asking big questions about the world, while leveraging research methodology to systematize that nuance and work towards change. Graduate school was a quick primer on how unfounded

K. Bauer (B) Department of Political Science, Nebraska Wesleyan University, Lincoln, NE, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_12

151

152

K. BAUER

many of these assumptions were and are. Gender disparities were particularly personal, while these persisting inequalities are certainly not limited to gender; women make up 31% of APSA members (Teele and Kathleen 2017) and 21% of the methodology section (Roberts 2018). These trends are reflective of how the development of the field privileges particular ways of knowing and being to the exclusion of those with other political realities and knowledge (Ferguson 2016; Fujii 2017; Ravecca 2019). For me, inviting undergraduate research methodology students into these questions is an ongoing personal reflection and intervention into these disparities. Much as I did, most students start a political science major with purpose, ambition, and vision for what the political world could be. My work teaching research methodology courses needs to follow their lead. First, it should equip them with multiple ways of seeing, understanding, analyzing, and intervening in the political world, and second, it should ensure equitable access to and learning in a class that frequently exacerbates inequalities in the field (Verge 2016; Shames and Wise 2017; Barnes 2018). Doing this work requires bringing students into complicated conversations about variations in the purpose and methods of studying politics. What do we know? How do we know that? What do we do with that knowledge? Based on five years of experience teaching a range of undergraduate research methods courses, I have found that organizing these classes around professional development is a useful way to bring students into these questions and equip them to finish the class valuing and applying research design skills. I didn’t expect my career to so strongly pivot around teaching these questions about the politics of knowledge and expertise. But, I was thrust into research methods curriculum on day 1 of my first faculty position (not even day 1 of the semester, but day 1 on campus for a department workshop before new faculty orientation) to redesign major requirements and, particularly, the department’s methods sequence (details of those revisions discussed in Clancy and Bauer 2021). Students’ stress over these courses was palpable, even in those early meetings. During my first semester, one of my earliest memories from late nights in my office was watching students, in tears, shove stacks of SPSS output under the door of my colleague who was teaching the Statistics and Senior Seminar classes. Some would stop by for statistics or emotional support, and we all left frustrated. Students knew those skills transferred into their senior projects, but was that skill acquisition anything more than a hoop to jump

12

RESEARCH DESIGN AS PROFESSIONAL DEVELOPMENT …

153

through to graduate? My frustration increased as I watched students hesitantly present their senior projects (regression analyses) at the end of my first year. Their eyes lit up and they spoke passionately about the significance of the big question that they had spent the semester and often their undergraduate career exploring, but became hesitant and cautious as they discussed how they translated that question into variables in an available data set, or how to extrapolate conclusions from their regression table. That colleague took a sabbatical my second year, and I was the lucky recipient of a two credit Quantitative Research Methods course and a two credit Statistics and SPSS course. I spent the summer reading and reflecting on why these classes and skills mattered, and for what purposes (and, to be honest, learning SPSS). As faculty and as a discipline, we know the intrinsic value of research design skills, but we often struggle to articulate this to students in the classroom. As I developed these and subsequent research methods classes, my teaching experiments have revolved around a key intervention: How can we teach research design in ways that students understand and value how research design and methodology contributes to their own professional trajectories? This work demands that we encourage students to reflect on their initial attraction to the field and future plans, and that we help students situate research methodology skills within those goals. This student-centered approach to teaching methods demands that we recognize that limiting our teaching to specific epistemological or ontological assumptions privilege specific forms of knowing and being (hooks 2000; Bernal 2002; Zuberi and Bonilla-Silva 2008; Smith 2013; Caldera 2017; D’Ignazio and Klein 2020); a more prescriptive approach writes particular students and particular career paths out of the field and renders the research methodology class a hoop to graduation. A moment from a meeting with graduate students after giving a research talk always comes to the forefront of my mind when reflecting on the need to intentionally teach the intersections of methods skills and applications. The department recruited students passionate about making change through local advocacy and equipped them to see and critique structures of power using primarily qualitative methods. Several students attended my research talk and were intrigued about how my research merged quantitative and qualitative methods. They had the instinct that quantitative methods were a powerful tool to critique and name structures of power, but expressed frustration that they did not see how their statistics and methods training integrated with their local activism. Certainly, the interaction between

154

K. BAUER

activism and quantitative methods is contentious (Stauffer and O’Brien 2018), but students should be equipped to intentionally choose their intervention in that tension. Much of my teaching in research methodology courses focuses on helping students think through their use for research skills based on their professional development trajectory. This challenges students to think broadly about their education, projecting themselves five or ten years into the future. Drawing on my work teaching a variety of research methodology courses at Nebraska Wesleyan University (2nd year Introduction to Political Science Analysis, 3rd year Quantitative Research Methods, 3rd year Statistics, and 4th year Senior Seminar) and Occidental College (Research Methods in Politics and Public Policy), I outline a few assignments below that have been particularly useful.

Personal Statements Frequently, my research methodology courses start and end with personal statement assignments. While this is not a genre of writing closely linked with methods skills, the assignment usefully asks students to think about their interest in their chosen major(s) and career field. Most of my classes teach a quantitative, positivist logic of inquiry, but I incorporate interpretivist and critical work to help students more deeply analyze and critique these logics. Personal statements very intimately preview this conversation; in asking students to construct their narrative, they frequently worry that they have yet to decide, or do not know how to make meaning of events in their lives. As Combs and Freedman (2012, 1035, 1034) situate, writing narratives challenges students to “develop and live out preferred stories” and recognize “the power involved in being in the position to decide which stories will be told and retold, and which will not. It shows how the sharing and circulation of different stories contributes to building different communities. It illustrates how stories give meaning to lives and relationships, privileging some people and relationships and making others invisible.” At the end of the semester, students often want update, remake, undo, emphasize, and hide portions of their pasts and futures, and worry these changes are disingenuous. Both conversations hint at future class conversations about what meaning is inferred from different types of data. I print off sample law school, graduate school, Peace Corps, and Fulbright personal statements, and have students workshop them in

12

RESEARCH DESIGN AS PROFESSIONAL DEVELOPMENT …

155

groups to discuss particularly effective or ineffective rhetorical strategies and narratives. Students then brainstorm their own narrative, reflecting on what they care about, stories that define their lives and trajectories, and things that friends and family might say about them. For students earlier in their college career, I recommend that they build off their college application essays. Students closer to graduation usually write for a specific opportunity, and work through translating their experiences for that audience. I encourage all to start with the template of a 3 paragraph, 1 page single spaced statement that defines their early story, shares how they have developed that interest, and discusses future plans. While this assignment does not get students closer to identifying an independent variable or interpreting regression output, students do think about how they relate to and situate themselves in the field, consider data as subject to critique, and have a very transferable piece of writing. Perhaps most importantly, the assignment builds mentoring relationships between faculty and students by facilitating conversations about potential classes, internships, graduate programs, study abroad opportunities, and campus resources. The first time I assigned this activity, a student shared that he spends summers running a fireworks business. My fascination with that summer job gradually evolved into ongoing conversations about his professional interests, which he was previously quick to avoid. That assignment prompted us to make meaning of his excitement about that job, clarifying his interest in political economy and previewing his current work with the Federal Reserve. While many students are unclear on what comes after university, others struggle to articulate how their passion developed. One student knew she would work on education policy and was intentional about directing coursework and internships in that direction. The personal statement assignment challenged her to articulate where that interest came from, and how that early motivation previewed her voice and intervention in the field. A few years later, this reflection on the seeds of this interest got her into Emory Law. While many faculty might see this assignment as tangential to research methods instruction, challenging students to envision their future encourages intentionally planning a path and acquiring skills to succeed in that field.

156

K. BAUER

Survey Research Second, students conduct survey research on a topic related to their professional interests. The assignment requires translating a puzzle to concepts, concepts to variables, and variables to data (Slootmaeckers et al. 2012) and collecting, analyzing, presenting, and critiquing those data. Frequently, students work in thematic groups to workshop individual hypotheses and survey questions, and develop and distribute a group survey. Most often, students distribute an online survey through social media and campus email groups, collecting a sample of primarily college students. During spring 2020, students collected more responses over the phone or from their families. Assessment varies based on what level of research design and methodology skills are being emphasized in the specific course but heavily weight the clarity of the research design and analysis, and their ability to situate and critique the validity of their conclusions. We spend a lot of time talking about sampling in class, but I make clear that they are collecting a convenience sample and are evaluated on their analysis of how biases in their sample might impact their results, not the quality of the sample. This assignment is often challenging for students who see a tension between observation and advocacy, or who are more interested in interpretivist work. Those same students often produce some of the most innovative work. For example, a future minister surveyed local church leaders to explore how the type of church congregation influences how COVID-19 impacts the community’s health, and a future politician explored how the race and ethnicity of survey respondents impact their perceptions of the prevalence of voter discrimination. Simultaneously, students are often more successful when they start the survey project with a working knowledge of the content they choose to research. Because this assignment is situated within a research methodology course, I can’t assume or require that they have a working knowledge of the content, meaning that I frequently question if students want to limit responses on a political partisanship or ideology to Republican or Democrat. Most students leave this assignment with excitement that they are able to translate their curiosities into tangible results.

12

RESEARCH DESIGN AS PROFESSIONAL DEVELOPMENT …

157

Undergraduate Labs Finally, I have turned towards teaching research design and methodology courses outside of the classroom through an undergraduate research lab. Adapting the laboratory model to the social sciences offers students high impact, scaffolded research training, and mentoring learning experiences (Becker 2020; Weinschenk 2020). The lab broadly focuses on immigration politics, and students apply each semester. With the goal of offering students scalable, collaborative exposure to research methods, I try to involve students at various stages of their academic career and different levels of exposure to research methods. This is in hopes of expanding access to and success with research by expanding the timeline over which students engage with research methodology. By creating early opportunities to be involved in stages of research early in their undergraduate education, these students will hopefully be better equipped to scale up their research later in their careers. We work collaboratively through the full research project and students often struggle; unlike in the classroom, there are no right answers or grades, but rather methodological decisions that need to be justified. Particularly in methodology courses, students grow accustomed to thinking about specific and correct answers; once these conversations were translated into a specific project, there were no longer fully correct answers, but rather thoughtful conversations. When teaching methods by doing, students self-reported impressive learning of and reflection about the professional utility of these skills. Not only did they profoundly recognize how they deepened classroom learning through application, but they thoughtfully internalized how research fits into their career. One shared, “I don’t think research in the academic capacity is for me. It is super interesting and I loved learning about the topic but at the end I was more interested in policy solutions.” Another reported “while I enjoy the research, I want to pursue a career that takes [the ‘so what’ questions] a step further and figures out what to do about addressing the implications that research presents.” Beyond the scope of the project and methodology, students also reported that conference presentations, manuscript preparation, and collaborative research were unexpectedly rewarding and transferrable to their career trajectory.

158

K. BAUER

Conclusions As my philosophy towards teaching research methods has evolved, I have increasingly transitioned towards situating research methodology as skills that equip students to see, use, and interrogate knowledge production in their specific career path. The teaching challenge, then, is to embed methodology content into students’ individual explorations of the value and utility of methods knowledge, rather than to deliver specific content about things like case selection, t-tests, or Type 2 errors. For those interested in incorporating this work their methodology courses, there are a few simple places to start, beyond the prior discussion about personal statements, survey research, and undergraduate labs. 1. Add an assignment that asks students to explore what knowledge is, how it is acquired, and how it is distributed in different career paths of interests. Have a class discussion on when, where, and how they are practicing portions of these skills in curriculum, including research methodology. The Association of American Colleges and Universities’ survey of employers is also a useful conversation starter. 2. Add interpretivism and /or critiques of positivism to the syllabus. Students have commented that these readings and conversations brought language to ways they think and learn about the world, and give them insight into which fields work similarly or different. Students who developed their interest in politics during the 2016 US presidential election have been particularly appreciative of the conversation about what data is, where it comes from, and how to think through validity and reliability. And, these conversations are often the liveliest of the semester! 3. Discuss research methodology conversations and controversies happening in higher education news. While there is certainly a need to start with core principles, transitioning into applications and examples deepens student learning. For example, I have assigned articles about Don Green and Michael LaCour’s research on opinions on same-sex marriage to set up conversations on replicability, and a twitter conversation about a randomized controlled trial in Nairobi to discuss research design and ethics debates. Statistics and methodology memes also help.

12

RESEARCH DESIGN AS PROFESSIONAL DEVELOPMENT …

159

As I rework classes into remote formats during the pandemic, intentionally teaching students to discover the intrinsic value of these skills is more paramount than ever. This transition makes teaching methodology courses both more challenging and rewarding. My role is less that of an instructor delivering sequential portions of positivist research methodology and statistics, and more of a mentor responsible for coaching students through different approaches to and applications of knowledge production and dissemination. Their success varies in different portions of the class; some thrive in discussions of interpretivism, some in learning a statistics software, some in the hands-on applications of concepts, and some in articulating their career path. But anecdotally, this variation is a source of validation. Students internalize their own work and intervention in the field, and in doing so, they recognize and empower classmates with different approaches to seeing, analyzing, and intervening in political realities.

References Barnes, T.D. 2018. Strategies for Improving Gender Diversity in the Methods Community: Insights from Political Methodologists and Social Science Research. PS, Political Science & Politics 51 (3): 580–587. Becker, M. 2020. Importing the Laboratory Model to the Social Sciences: Prospects for Improving Mentoring of Undergraduate Researchers. Journal of Political Science Education 16 (2): 212–224. Bernal, D.D. 2002. Critical Race Theory, Latino Critical Theory, and Critical Raced-Gendered Epistemologies: Recognizing Students of Color as Holders and Creators of Knowledge. Qualitative Inquiry 8 (1): 105–126. Caldera, A. 2017. Black Feminist/Womanist Epistemologies, Pedagogies, and Methodologies: A Review of Literature. In College Curriculum at the Crossroads, 35–51. Routledge. Clancy, K., and K. Bauer. 2021. Scaffolding Research Methods Across the Curriculum: An Exploration of Embedded Curricular Design. In Teaching Research Methods in Political Science, 161-176. Edward Elgar Publishers. Combs, G., and J. Freedman. 2012. Narrative, Poststructuralism, and Social Justice: Current Practices in Narrative Therapy. The Counseling Psychologist 40 (7): 1033–1060. D’Ignazio, C., and L.F. Klein. 2020. Data Feminism. MIT Press. Ferguson, K. 2016. Why Does Political Science Hate American Indians? Perspectives on Politics 14 (4): 1029–1038. Fujii, L. A. 2017. The Real Problem with Diversity in Political Science. Duck of Minerva.

160

K. BAUER

Hill, C., C. Corbett, and A. St Rose. 2010. Why so Few? Women in Science, Technology, Engineering, and Mathematics. ERIC. hooks, b. 2000. Feminist theory: From margin to center: Pluto Press. Ravecca, P. 2019. The Politics of Political Science: Re-Writing Latin American Experiences: Routledge. Roberts, M.E. 2018. What is political methodology? PS, Political Science & Politics 51 (3): 597–601. Shames, S. L, and T. Wise. 2017. “Gender, diversity, and methods in political science: A theory of selection and survival biases.” PS: Political Science & Politics 50 (3):811–823. Slootmaeckers, K., J. Adriaensen, and B. Kerremans. 2012. Learning Trajectory of Quantitative Methods–Students’ Experience with Curriculum-based Method for Overcoming Statistics Anxiety. Learning 12: 13. Smith, L. T. 2013. Decolonizing methodologies: Research and indigenous peoples: Zed Books Ltd. Stauffer, K. E., and D. Z. O’Brien. 2018. “Quantitative methods and feminist political science.” In Oxford Research Encyclopedia of Politics. Teele, D.L., and Kathleen T. 2017. “Gender in the journals: Publication patterns in political science.” PS: Political Science & Politics 50 (2):433–447. Verge, T. 2016. “The Virtues of Engendering Quantitative Methods Courses.” PS: Political Science & Politics 49 (3):550–553. Weinschenk, A.C. 2020. “Creating and Implementing an Undergraduate Research Lab in Political Science.” Journal of Political Science Education. Zuberi,T., and E. Bonilla-Silva. 2008. White logic, white methods: Racism and methodology: Rowman & Littlefield Publishers.

CHAPTER 13

Teaching Multidisciplinary Research Methods at a Small Liberal Arts College Jarrod T. Kelly

As a candidate on the academic job market applying for jobs primarily at liberal arts colleges, I anticipated that I would be asked about teaching research methods. While research methodology was not one of my specializations in graduate school, I was prepared to enthusiastically state that I would be willing to teach research methods to undergraduates. While I knew that this would be a difficult course to teach, having done so during one particularly trying summer term of graduate school, I also knew that small colleges and universities are often in need of faculty who are willing to take on this, sometimes, thankless task. After all, having a new hire teach research methods allows more senior faculty to focus on courses perceived as more enjoyable, such as upper-level courses within their subfield. Knowing that teaching this course may be the “price of entry,” so to speak, for many academic jobs, I decided to anticipate this

J. T. Kelly (B) School of Social Sciences and Education, North Carolina Wesleyan College, Rocky Mount, NC, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_13

161

162

J. T. KELLY

question and strategically (or so I thought) mention research methods as one of my teaching interests in every phone interview and campus visit. Within these interactions, many search committees seemed quite interested in confirming that I could (and would), in fact, teach research methods. After being fortunate enough to secure a tenure-track position, I arrived on campus ready to begin my work of teaching research methods to bright and eager political science undergraduates. While I was not assigned to teach research methods during my first semester, through the kindness of a senior colleague, I would begin teaching the course the following spring semester. But, much to my dismay, I was informed that the course is cross listed in political science and sociology. Thus, not only would I be perpetually responsible for ensuring that our political science majors had enough of a background in research methods to successfully complete their upper-level major coursework (to say nothing of preparing them for graduate school and/or professional careers), but sociology majors as well. In fact, every time that I have taught this course, sociology majors have outnumbered political science majors, at times by more than a 2:1 ratio. Teaching research methods to students from multiple majors has required me to critically reexamine almost every aspect of my pedagogy, including textbook selection, course assignments, and even the examples that I provide during class. While this has at times been quite challenging, ultimately I believe that I am a more effective teacher as a result. The approach that I have adopted, after much trial and error, focuses on the process of conducting research, with an emphasis on designing research to address meaningful problems and issues. This process is not unique to any one social science discipline. With a focus on process, I have been better able to tailor the course to meet the needs of students from various academic majors and backgrounds. I will devote the majority of this chapter to recounting my early-career experience teaching a multidisciplinary research methods course, with attention to how my pedagogy has evolved over the course of several semesters. I will share mistakes that I made, some slightly embarrassing, in the hopes that others may learn from them. Throughout, I will discuss several specific pedagogical strategies that I employ to better facilitate a research methods curriculum that is flexible and multidisciplinary. I conclude by reflecting upon what I have learned and how other instructors may best attempt some of these strategies, particularly if considering

13

TEACHING MULTIDISCIPLINARY RESEARCH METHODS …

163

a multidisciplinary, cross-listed course. Overall, while I was initially hesitant to teach a cross-listed research methods course, I have learned to embrace its multidisciplinary nature. It has quickly become a course that I look forward to teaching each year.

A Brief Description of the Course and Its Students Like many institutions, undergraduates in political science and sociology at North Carolina Wesleyan College are required to complete one quantitative research methods course. Political science and sociology are relatively small majors, with approximately fifty students between them. As such, the decision was made at some point to cross list the course. This multidisciplinary course is entitled, “Social Research Methods,” an appropriately ambiguous title for a course that aspires to be equally applicable to both political science and sociology. Enrollment over the past three years, since I began teaching the course, has ranged between six and thirteen students. Sociology majors have consistently outnumbered political science majors, with an average composition of approximately 65% sociology majors and 35% political science majors. Most students choose to complete the course during their junior year, though we encourage students to enroll in the course during their sophomore year so that they may apply what they have learned to upper-level coursework within their major. The course is offered annually each spring semester and, for the past three years, I have been the sole instructor. Prior to enrolling in the course, students must successfully complete an introductory statistics course that is offered within the mathematics department. Students are encouraged to complete this course during the fall semester, immediately prior to enrolling in the research methods course. However, because this statistics course is a requirement for many majors, including those in the social and natural sciences, it often fills up quickly and some students have been allowed to enroll in this course concurrently with the research methods course so that their graduation date is not delayed. Additionally, some students complete the statistics prerequisite several semesters prior to enrolling in the research methods course. Therefore, while students should arrive with some basic background in statistics, their experiences vary quite dramatically and I cannot rely on much, if any, prior statistical knowledge.

164

J. T. KELLY

As the sole quantitative research methods course for both majors, and due to the diversity of students’ backgrounds in statistics, the course must cover quite a bit of content on research design and applied statistical analysis. By the end of the course, I expect students to be able to: (1) develop research questions regarding social phenomena, (2) identify a research strategy (design) to answer specific questions, (3) apply basic statistical techniques to analyze small datasets, and (4) interpret the results, both statistically and substantively, to answer one or more research questions. Overall, I expect students to understand and apply the research process to conduct their own small-scale research project. For a single class, one in which many students arrive with little prior knowledge of the research process, this is a rather tall order. In an ideal world, this would be a two-course sequence, split between statistics and research design, as is common in many social science disciplines. Likewise, I admit that early on in teaching this course I was very much guilty of trying to fit in too much content into one course, to the detriment of student learning. While I will elaborate further on my pedagogical evolution in the following section, the primary goal of the course has always, and will always, remain the same: to develop an understanding of, and an excitement for, the social science research process.

A Pedagogical Journey Year 1: Initial Struggles. My first year teaching multidisciplinary research methods was also my first year as a full-time assistant professor. Unsurprisingly, I was overwhelmed by many aspects of the transition and, in hindsight, I made several mistakes that seem incredibly obvious now. For example, when I thought about how to make the course applicable to political science and sociology majors, I focused on updating the language throughout course materials, with one crucial and notable exception: the textbook. While I worked tirelessly to replace references to “political” science with “social” science, the textbook that I had chosen (used previously when I taught the course in graduate school) contained political science in the title. Realizing this shortly before the semester began, I devoted at least ten minutes of the first-class session to explaining that the textbook was applicable to both majors. This was not a great way to begin trying to win over already-reluctant sociology majors, who did not know me and likely wondered why a political scientist was instructing them on how to conduct research in sociology. Even more unfortunate

13

TEACHING MULTIDISCIPLINARY RESEARCH METHODS …

165

was that fact that, given our status as a textbook rental campus, I was locked into this textbook for the following year. My first revisions to the course focused on rather superficial changes. In retrospect, I should have devoted more time and attention to modifying the course content and assignments to reflect the truly multidisciplinary nature of the course. To illustrate, consider one of the earliest assignments in the course: the development of an empirical research question. This is the first step of a multi-stage research project assignment on any topic of their choice, provided that it is relevant to either political science or sociology. As many of us experienced during our undergraduate careers, it can be difficult to derive an original research question. Students seemed to struggle most with the overwhelming amount of choice that is inherent to the task. While I provided several examples in class and on the assignment description, my best examples were, quite naturally, related to political phenomena. I should have anticipated the result: political science majors submitted higher-quality research questions (though many were rather unoriginal and highly related to the examples provided) than sociology majors. It probably did not help that, when asked by a sociology major for examples of sociological theories, I cited Durkheim’s (2014 [1893]) concept of anomie, which was indicative of my lack of knowledge of contemporary sociology at the time. Unsurprisingly, my first iteration of the course was one of the most disastrous of my teaching career. While political science majors excelled, with an average course grade of approximately 84%, sociology majors lagged far behind, with an average course grade of about 63%, as shown in Fig. 13.1. I shoulder much of the responsibility for this large gap: I simply was not effective at making the course intellectually appealing to sociology majors. Unsurprisingly, this was reflected in my course evaluation regarding whether the course held their interest. While political science majors reported that the course held their interest, with an average rating of three (on a four-point scale), sociology majors largely reported that the course did not hold theirs, with an average rating of two, a full point below political science majors. Year 2: A Renewed Focus. After a rough start to teaching research methods, I began to reflect on how I could reduce the outcomes gap between political science and sociology majors. In doing so, I kept coming back to a basic question, but one of tantamount importance: Why is research methods a nearly universal requirement in social science majors?

166

J. T. KELLY

Fig. 13.1 Mean course grade by major

One or two courses are not going to adequately prepare most undergraduates for research careers, at least not without some additional training. So, what is the point? My conclusion was this: the most important skill that I can teach students is how to identify a problem/question, followed by a process by which they evaluate, and, ultimately, resolve/answer it. A useful byproduct of this approach is that this skill should be equally applicable to all students, regardless of their major. At this point I embarked on an overhaul of the course to provide a more interactive and hands-on approach from start to finish, with a focus on using the social scientific research process to address real-world questions. I did this primarily by refocusing the research project assignment, which students complete in stages over the course of the semester. Specifically, within the first weeks of the course, I required students to identify a problem or issue that affects students on campus. Students could choose any topic they were interested in, but it must be directly relevant to the campus community. Initially, each student individually proposed a topic. I then grouped the topics into themes and placed students into small groups based on shared interests, with each group containing both political science and sociology majors. At that point, they could choose between the proposed topics or come up with something entirely new.

13

TEACHING MULTIDISCIPLINARY RESEARCH METHODS …

167

Once a topic was chosen, students worked together throughout the duration of the course to design and execute a research project to address their topic. At all stages, the focus remained squarely on the research process itself, which helps to avoid traditional disciplinary boundaries. This shift in focus to a more concrete problem of which students had direct knowledge and experience was inspired by the work on experiential, or experience-based, learning (Kolb 1984; Kolb et al. 2000). In this framework, learning is facilitated through the learners’ own experiences, past or present, through which new knowledge can be integrated through a process of critical reflection. As noted by Andresen, Boud, and Cohen (2000), “Much of the impetus for [experience-based learning] has been a reaction against an approach to learning which is overly didactic, teacher controlled and involving a discipline-constrained transmission of knowledge. It supports a more participative, learner-centered approach, which places an emphasis on direct engagement, rich learning events, and the construction of meaning by learners” (225). As such, I sought to better engage students by focusing the project on something in which they share a deep commitment regardless of major: their own campus community. This approach also draws from the Methodology for Interdisciplinary Research (MIR) framework, proposed by Tobi and Kampen (2018). The authors were interested in how to better facilitate interdisciplinary collaboration and found that it was helpful to “put the common goal of the researchers at the center, instead of the diversity of their respective backgrounds” (Tobi and Kampen 2018). Thus, to create an intellectually diverse environment that places students from different backgrounds on equal footing, I designed a project whereby students identified a common interest that is relevant to them all and then placed them into teams of students with different academic backgrounds in order to work together to accomplish a shared goal. Through this new approach to the major assignment in the course, I was able to begin narrowing the outcomes gap between majors. Specifically, the average grade for sociology majors rose a full letter grade, as shown in Fig. 13.1. Furthermore, students developed a much more original and interesting set of projects that sought to address some real issues on campus that personally mattered to them, including school spirit, political engagement, and even the quality of food served in the campus dining halls. While I expressed some initial hesitation regarding this last topic, the students did a commendable job of surveying student opinions and integrating literature linking diet to academic performance.

168

J. T. KELLY

Overall, by relinquishing some control of the substantive content of the projects, while restricting the domain to issues related to their shared environment, the focus became more centered on the research process itself and transcended academic disciplines. As a result, students in both majors were considerably more motivated and enthusiastic about the project, and students from both majors reported higher interest in course evaluations. Even despite a reduced emphasis on substantive political science questions and issues, political science majors nonetheless reported increased interest in the course. More importantly, these changes resulted in substantially improved learning outcomes across the two majors represented within the course. Year 3: Refining the Process (and a Pandemic Curveball). After seeing some success with the process-oriented approach, I sought to address lingering areas of concern within the course in my third year. Namely, students seemed to struggle most in the data analysis stage of the course. By now I was free to choose a more inclusive text, so I devoted more time and attention to applied statistical analysis and selected a textbook with a focus on statistics authored by two sociologists and one political scientist (Whittier et al. 2019). Previously, I aligned course material with each stage of the process, saving data analysis and statistics for the latter half of the course. But now, I sought to integrate this material throughout the course, supplementing the textbook with my own material on the research process and research design. This way, by the time we get to material on inferential statistics, which students need to analyze their project’s data, they would already possess a strong background on data structure, software (i.e., STATA), and descriptive statistics. I continued to deliver substantive content on the research process, designs (observational, experimental, etc.), and concepts (validity, reliability, sampling, etc.), but the course would devote some time each week to building up student’s abilities to analyze quantitative data. All-in-all, the course was moving along smoothly, that is until the COVID-19 pandemic forced the closure of campus and the transition of all courses to an online environment approximately halfway into the semester. Students were just beginning to design their research projects and work within their groups when we learned that we would not be returning to classes following spring break. Therefore, I was forced to make some on-the-fly adjustments to the project, including adjusting it to be individual, rather than team-based. This decision was made based on an informal poll of student preferences. Many students, probably wisely,

13

TEACHING MULTIDISCIPLINARY RESEARCH METHODS …

169

anticipated difficulties with working collaboratively in an environment in which they could not physically meet due to technology limitations, altered schedules, etc. I also revised the assignment to exclude the data collection and analysis stages: students were only required to design a project which they could hypothetically execute, had circumstances been different. I then assigned students to analyze secondary datasets to answer smaller-scale questions of interest, rather than analyzing their own data that they would typically collect themselves, often using surveys of their peers. I also allowed students to revisit their topic and adjust it as they saw fit. Despite all the challenges caused by the pandemic, disparities between student outcomes continued to diminish, as shown in Fig. 13.1. I attribute this to my continued focus on the research process itself, as well as an ongoing concern with making the course inclusive and relevant to students of different academic backgrounds. The gap during this final year of observation narrowed to just seven percentage points, a reduction of about two-thirds from the initial year. In a broader sense, the continual improvement of outcomes among sociology majors is indicative of the flexibility of the process-oriented approach to teaching research methods. While I prefer to integrate a large research project within the course to allow students to connect course material to their own experiences and community, I believe that instructors can make smaller adjustments and continue to reap the benefits offered by this approach.

Concluding Advice: Preparing Your Cross-Listed Course My journey thus far in teaching a cross-listed, multidisciplinary research methods course has been a challenging, yet fruitful, endeavor. While a course of this type is not common, a brief search of online course catalogs shows that it is far from unique. Several other colleges and universities, particularly smaller institutions, do offer research methods courses that are cross listed in more than one social science discipline. At institutions with an undergraduate enrollment of less than 2,000 students, it is common for certain majors to only count a few dozen students (or less) among their ranks. Therefore, with cohorts often consisting of ten or fewer students, enrollment in required courses that are offered annually or semi-annually can be quite low. Departments may feel pressure from administrators to

170

J. T. KELLY

offer courses less often to increase the overall enrollment and better allocate faculty resources. This is likely to be even more so the case following the disruption stemming from the COVID-19 pandemic. One solution is to offer a cross-listed course. If you are considering such a move, or if found in a similar position to myself, consider the following advice based on my experience, including my successes and failures: 1. Superficial changes are not enough. Teaching a course to multiple majors requires more than just “political” for “social.” Rather, more fundamental and comprehensive changes are necessary in order to make the course equally applicable to all students. This includes casting a critical eye over all course content, including assignments, to ensure that students can complete them with little or no knowledge of political science. 2. Familiarize yourself with research traditions in other disciplines, but do not overdo it. Prior to teaching this course for the second time, I browsed through sociology syllabi to become more familiar with their structure, assignments, and key concepts/topics. I also dug up a copy of my introductory sociology textbook and perused a few chapters. While this was helpful in providing additional examples in class that were more relevant to sociology majors, I found that this was not as helpful as I thought it would be. Rather, moving to a process-oriented approach grounded in students’ shared identities as college students proved to be much more effective. Yet, I continue to encounter situations in which a sociology student approaches me with ideas for topics and I feel that I am now better prepared to offer connections to their coursework in sociology. 3. Focus on what students have in common. For me, this is their identity as college students within a shared academic environment. However, there could be other ways to leverage commonalities between students across different major fields of study, such as shared interests or goals. For example, some colleges place issues of social justice at the core of their mission. This shared orientation could be leveraged to bring students together despite different academic goals or pursuits. I choose to identify common interests by allowing students to individually determine a research topic prior to assigning groups, but this could be accomplished through other means such as an early semester survey of research interests. The

13

TEACHING MULTIDISCIPLINARY RESEARCH METHODS …

171

important point is to place the learner at the center and to devise pedagogical strategies to engage learners within a shared context. 4. Think broadly about how the course can be applicable to any student. For me, in reflecting upon my first semester of many failures in teaching research methods, I found it immensely helpful to question what, precisely, I wanted students to take away from the course long after it ended. And, more importantly, to determine what would be applicable to students pursuing a wide variety of career paths (which is always the case, even if teaching students all from the same major). I settled upon the process of scientific inquiry: using scientific methods to answer questions and solve problems. This methodology is applicable to any number of situations in which students need to think critically to arrive at a solution to any given problem. Additionally, by focusing on the process itself, I can make the course much more broadly relevant to students regardless of their background and career ambitions. Overall, I hope that my emphasis on the research process itself, particularly focused on designing a research strategy to answer tangible questions, is a pedagogical approach that can greatly enhance any research methods curriculum regardless of its audience. While those teaching research methods solely to political science undergraduates may prefer to make more connections between methodology and the substantive questions, concepts, and debates within political science, not doing so does not seem to present a barrier to students gaining a broad understanding of the social scientific research process.

References Andresen, L., D.Boud, and R. Cohen. 2000. Experience-Based Learning. In Understanding Adult Education and Training, ed. Griff Foley, 225–239. Sydney: Allen & Unwin. Durkheim, E. 2014 (1893). The Division of Labor in Society. New York: Simon & Schuster. Kolb, D.A. 1984. Experiential Learning: Experience as the Source of Learning and Development. Englewood Cliffs, NJ: Prentice Hall. Kolb, D.A., R.E. Boyatzis, and C. Mainemelis. 2000. Experiential Learning Theory: Previous Research and New Directions. In Perspectives on Thinking,

172

J. T. KELLY

Learning, and Cognitive Styles, ed. J. Robert, 227–247. Sternberg and Li-fang Zhang. New York: Routledge. Tobi, H., and J.K. Kampen. 2018. Research Design: The Methodology for Interdisciplinary Research Framework. Quality and Quantity 52 (3): 1209–1225. Whittier, N., T. Wildhagen, and Howard J. Gold. 2019. Statistics for Social Understanding. Lanham, MD: Rowman & Littlefield.

CHAPTER 14

Less Can Be More: Encouraging Mastery of Research Design in Undergraduate Research Methods Jennie Sweet-Cushman

Arguably, if an undergraduate degree in political science should offer anything it should be an understanding of how objective analysis of elements of the political environment may be conducted. That does not mean all our graduates should go on to conduct political science themselves, but it should mean they are well-equipped to understand the role of social science in a complex society. Anyone who has ever practiced social science surely recognizes that this is a lifelong learning process. There are nearly infinite possibilities in every research study, from options for operationalizing variables to how to recruit subjects to which methodology to use to collect data. Until an undergraduate student encounters a research methods course, these decisions have been largely made out of sight of the students who read, discuss, and learn from the research

J. Sweet-Cushman (B) School of Arts, Science & Business, Chatham University, Pittsburgh, PA, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_14

173

174

J. SWEET-CUSHMAN

generated. It can be exceptionally challenging for an instructor of undergraduate research methods to determine whether the best route is to tackle the breadth of considerations that go into conducting social science inquiry or to reveal the depth in this behind-the-scenes work. In this essay, I will describe the evolution of my undergraduate research methods curriculum from a breadth approach to a depth one. It has been one defined by the struggle to find a broad-enough scope to ensure students appreciate the possibilities, but also sufficient intensity to promote understanding of scientific inquiry. What began as a broad survey of the scientific method has become a more narrowly focused examination of how to effectively deploy just a few methodologies. Recognizing how challenging research methods can be for students, I have finally settled on the pedagogical philosophy that less is more.

Teaching Undergraduate Research Methods I did not start my journey teaching undergraduate research methods with this clear picture of how to approach the course. In fact, it started with a dream academic job interview where the hiring committee asked unexpectedly if I could teach research methods. At this point, even the greenest of graduate students know to just say “yes” and proceed to wax poetically about one’s commitment to training the next generation of social scientists. Only after you get the job offer do you panic about your ability to actually teach research methods. Really, how bad can it possibly be? I had a research methods course in graduate school. I wrote a dissertation that used principles of methodological training (sort of). Once that dust settled, I soon realized the real challenge in teaching undergraduate methods isn’t being capable of teaching it, but rather what to teach of it. There are so many foundational concepts, like ethics, theorizing, writing literature reviews, and measurement, you could fill an entire course with just those introductory elements. However, doing so would leave students unprepared to actually develop sound ways of testing hypotheses. Leave out the foundations and their methodological approaches are less likely to be well-informed. Focus too much on foundations and methods and how will students ever understand how to analyze the data they generate? I am a decade into a career as a political scientist and I am still learning. It is so much to translate for students and the pieces are so interdependent.

14

LESS CAN BE MORE: ENCOURAGING MASTERY …

175

Teaching Breadth Over Depth For that first course, I decided to teach all of it. I designed the course as a broad review of the entire topic of social science inquiry since that seemed to be how textbooks were oriented and I added in relevant academic readings that employed those methodologies. We would learn about a thing, witness the thing, do the thing, and take a test on the thing. The logic seemed sound and not dissimilar from how I presumed other science courses might proceed. My syllabus was a web of small assignments with strict and rapid-fire deadlines designed to walk students through the entire research process: pick a topic, write a research question, develop a hypothesis, propose a methodological approach, etc. The problem was the material was so abstract—none of the students had experience doing research—that they didn’t fully grasp anything. Worse yet, the concepts were so interconnected that once they were lost there was no digging out. I was constantly having to steer students toward approaches that were in the realm of possibility because, as novices, they had no idea what was feasible or even practical to attempt in a semester’s time. Students hated it. I hated it. My colleague teaching the senior research capstone course hated it because the students weren’t prepared to conduct their own research. Subsequent years employed much trial and error. We spent a week or more doing ethics training so they could be certified by our Institutional Review Board because ethics are important, right? We read multiple examples of others’ field research, experiments, surveys, observational studies, etc., so students would recognize an effective use of a method and see how authors analyzed the data. Students struggled to understand the examples and frequently didn’t read them. I walked them through my own work and decision-making processes. I had distinguished guests present to the students about how they conducted their work. Students were interested, but it didn’t seem to connect to anything they might do in their own roles as social scientists. The students did group research projects. That went badly. They did individual research projects. Those went worse. They did a combination of group and individual research projects. Horrible. They only proposed research projects but didn’t execute them. There was no substance to the methods they proposed for the hypothetical projects. They could, at best, write two or three sentences about their plans despite having learned

176

J. SWEET-CUSHMAN

about the multitude of decisions that go into numerous methodological approaches. They presented their findings publicly on posters. They made slides of their findings and just submitted those. They answered questions about their proposed research plans with a group of faculty and seemed genuinely confused by the question “what is your methodology?” Students were exhausted by the amount of work they had to do to step through all of both conceptual and technical components of actually conducting research and so was I. They were maybe able to answer a multiple-choice exam that asked questions about what a representative sample was, but they didn’t seem capable of determining how they might go about either collecting or finding data that employed one. Most of all, how to handle statistical training was a constant struggle. While most students fulfill a quantitative general education requirement with a statistics course, it was a math course and virtually none of them were capable of doing applied statistics when they arrived in my methods course. If they were going to be able to analyze data they generated in their own research, they would need to know which statistical tools to use. I could usually keep them with me only long enough to run descriptives, but as the statistical analyses became more abstract, fewer and fewer students were capable of processing when to use them. Virtually all of the students that chose a quantitative senior research project wound up in my office six months later for help analyzing their data—a process for which they had never budgeted sufficient time because they never gained a sense of the reality of doing data analysis. Some of the class projects generated during these years were not bad and many students wrote well-composed literature reviews and did well on exams. Inevitably I would optimistically assume their senior capstone projects, the following fall would be adequate, but every year the skills they seemed to grasp in my class never transferred to their own projects. I would sit in stunned silence when students would present their proposals with methodologies that were little more than plans to do a literature review despite having been exposed to and learning about a multitude of approaches that were suitable for social science. One year while seniors were presenting these initial proposals for their senior research projects, it occurred to me that these students really were bright and curious with fantastic ideas for research projects but, even after an entire course specifically designed to teach them how to effectively test those ideas, they were still scrambling for methodological approaches they understood well enough to apply. They knew vaguely of too many things to know deeply

14

LESS CAN BE MORE: ENCOURAGING MASTERY …

177

about anything well enough to integrate it confidently into their own research. More was just less and they were falling back on what they did know—how to write a traditional “research” paper—from years of executing what my colleagues and I had been calling research papers. (I don’t call writing assignments this anymore for precisely this reason).

Teaching Depth Over Breadth I ultimately recognized that I needed to spend more time acclimating them to how and when to use a particular methodology in hopes of building a foundation for their own work as nascent researchers. However, a single semester offers so little time and I knew I had to pare down significantly the material I was teaching in the class. I got buy-in from my colleague teaching the capstone course to limit students’ options for conducting their senior research to only a finite set of methodological approaches. I would cover these research methods in their junior year rather than allowing them free reign to conceive of and plan their projects. Given the relatively little success students had with the more open-ended model, it was not a hard sell to make. Each semester now focuses only on a few components of research design. First, I focus on the basics by emphasizing setting up good inquiry decision-making regardless of methodology (operationalizing, hypothesizing, theorizing, etc.), for which there is significant repetition throughout the semester. Second, I teach a limited set of three methods, which we practice through small exercises that take us through mini research projects. Finally, our department coordinates so that these three methodologies become the foundation for their senior capstone projects the following semester. These limited options have meant that they can comprehend and execute an entire concrete research project, rather than have a vague concept of how one might conduct research in the abstract. For our majors, and their patterns of career placement and graduate education, three approaches made the most sense: content analysis, surveys, and policy analysis. Though it darn near broke my political psychologist heart to discontinue teaching experiments given that we always had the most fun learning experiments and it is my preferred methodology. There is nothing magical per se about these three, they just fit our students’ needs best and offer both quantitative and qualitative components. Including methodologies that could be successfully used by non-quantitative-minded students were purposeful, since I recognized

178

J. SWEET-CUSHMAN

that I could use this reorganization of the course to improve students’ methodological skills or their statistical ones, but likely not both. I set out to trim my syllabus down to only those three methods and to build the crucial supporting concepts around them. This resulted in a three-foci approach to the “less is more” strategy that I now employ. First, I didn’t compromise any of the foundational concepts except ethics, which I regret, but as undergraduates their research will always be supervized. If students will be conducting human subjects-based research that requires review they are directed to complete the appropriate training independently. I still teach these foundational concepts linearly, starting with theory building and reasoning and conceptualization and moving on to measurement and sampling. With fewer methodologies on the syllabus, though, we have time to practice and apply, not just memorize terminology for a midterm exam. Once this groundwork is laid, we move on to learning the three methodologies. In doing so, I move the foundational concepts from abstract to more concrete, tying them specifically to application in the methodologies. At this point, I allocate a couple of weeks to each of the three methodologies where we review the basic principles of the method, when it is appropriate for use, and general steps in its execution. We then read an example of social science that employs the methodology— discussing the foundational concepts and design of the research. What was the author’s conceptualization of this entity they sought to understand? Did they use inductive or deductive reasoning? Can you identify the sampling frame? What statistical analysis tools did the author use to test their hypotheses? The same questions all instructors of undergraduate research methods are likely provoking their students to seek answers to when given examples of peer-reviewed research. At this point, these questions are still difficult for them to answer so there is still an opportunity for clarification of basic principles. Then, I move to deepen the understanding of the methodology experientially. We try out the methodology as a group—conducting our own mini research projects from beginning to end in as complete a way as is possible. For example, for survey methods, we will typically consider research questions that could be well-addressed by surveying the student body. This past spring’s class was curious about whether taking a class in American government affected student political participation. This question led us to talk about foundational concepts around theorizing

14

LESS CAN BE MORE: ENCOURAGING MASTERY …

179

youth civic engagement, discussing validated measures from the American National Election Studies (ANES), and looking at previous work that had tested relevant hypotheses around political participation and education. We then wrote survey questions, revised them, pre-tested them, and ultimately collected a convenience sample (sampling, a foundational concept) of student responses. We did similar modules for the other two methodologies. For several years in a row, the content analysis exercise employed presidential campaign ads using the ad repository available at www.livingroo mcandidate.org, which offers a sample of ads that students can draw their content-based data from. We learn about coding, then code the ads to address our hypotheses, building a codebook as we go along. We almost never find support for the hypotheses students generate, which offers a great opportunity to discuss important and common features of conducting research like null results, bad theory, small n issues, Type I and Type II errors, and more, in a way that is not just a paragraph in a textbook. The third methodology, policy analysis, is executed differently. Because policy analysis is not necessarily hypothesis driven, it can be very difficult to find a starting point and for students new to the process to establish a logical flow for their analysis. I use Eugene Bardach’s “Eightfold Path to More Effective Problem Solving” (2012) because it gives students a defined process to guide them. As a class, we agree on a contemporary public policy issue, I provide a background reading to get students started thinking about the policy environment in that area, and then we start working through the steps in the process together. Depending on the specific policy we examine, certain steps might not fit precisely (or at all), so working through it together allows an opportunity to encourage students to use the steps as a guide, not a rigid structure. The issue of urban homelessness has worked well for this exercise, though I like student interests to dictate each semester’s project so they feel invested in these small projects. One advantage of these small research projects has been that they each generate their own datasets. I could build the process of considering statistical analysis into the process of designing the measures that would be used—a feature of teaching data analysis that is lost if you use existing datasets. Students then learn how to input and clean data organically at the end of each methodology module. By the time we consider how to test our hypotheses, students are more familiar with their dataset and have

180

J. SWEET-CUSHMAN

a better appreciation for what it is and is not capable of doing. They, for example, decided to collect respondent education on their survey as a categorical variable, knowing it would need to be coded as a numerical one in order to do certain analyses. Doing data analysis this way feels much more natural. Rather than teaching a set of tools that they may, or—more likely—may not, discern when to draw upon, students problem solve to consider what they need for their data and for their hypotheses. Just like, dare I say, a social scientist might do in analyzing their own data. This approach also allows for some repetition between the three methodologies that helps to reinforce concepts and processes. We always start with descriptives and frequencies, assessing normality, etc., which is just good dataset hygiene regardless of your methodology. We cover policy analysis last in this sequence of methodologies because there may or may not be statistical analysis involved depending on the evidence the students collect. Instead, we will construct a Goeller scorecard, a useful tool for any social science major to take into their post-college career. If there is statistical analysis required, it may inform the scorecard. The final result of this “less is more” approach to teaching research methodologies is that our juniors have learned the foundational concepts in the abstract, then applied them thoroughly three times in accessible methodologies with guidance and correction from me. As seniors, we move them to do this more independently, but while still incorporating the foundational concepts and using one of the three methodologies they learned as juniors. We pre-determine the topic so we can still offer guidance with theory, background, validated measures, etc., but only as loosely offered research questions that are well-served by these methodologies. Students should then feel confident in the next steps. My syllabus has significantly fewer topics on it in this new version of the course; about one-third fewer. Nonetheless, students seem to learn more and at a different, deeper level of learning—something approaching mastery rather than mere recognition or even understanding. For the undergraduate research method student, especially one preparing for their senior thesis project or research capstone experience, it is all upside: less to learn with more opportunity for mastery.

14

LESS CAN BE MORE: ENCOURAGING MASTERY …

181

Remaining Challenges For a faculty member taking on this approach, there is a singular drawback. Because students do try out these methodologies organically and in real time, it is difficult to prepare for the collective decision making that students will do—especially if the class size requires multiple groups or you’ve decided to incorporate an individual component (my students work independently, but decisions are made as a class). As the instructor, you have to be able to teach to whatever direction they may want to take the project in and put the brakes on a direction that might result in less effective pedagogy or an approach that could be too complex for them to manage with undergraduate skills. You have to resolve to be comfortable with that uncertainty…kind of like you maybe were that first time you wound up teaching the class because you confidently asserted that you could.

CHAPTER 15

“Research Methods: Who Am I and Why Am I Here?” Robert Postic

Welcome to Teaching Research Methods I’ve been teaching at the university level for almost 20 years and was recently promoted to the full rank of Professor. Even though I’ve been fortunate to consistently receive very good to excellent student evaluations, I get anxious when I see the most recent results arrive in my email inbox. That’s particularly so with research methods. I’m not sure why I worry about them, but I do. In the most recent evaluation for my methods course, every student gave me the highest rating in all categories. I felt affirmed, ignoring the fact that only 3 students had submitted a completed evaluation. Here is my journey in teaching research methods. In the summer of 2006, as an adjunct and ABD (all but dissertation), I was offered the opportunity to teach research methods for the fall semester. Even though I had already taught a number of substantive

R. Postic (B) Department of Behavioral Sciences, The University of Findlay, Findlay, OH, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_15

183

184

R. POSTIC

political science courses, this would be my first time teaching methods; my level of angst was high. Nevertheless, as an adjunct, I never turned down classes, regardless of my feelings of competence in the content area. I simply figured I had to stay one step ahead of the students, which I typically could manage without difficulty. The faculty member in charge of the course schedule set up a time to review the course with me and go over how the course fits within the department’s program. Because I expected there to be a lot of information to discuss, I set aside a fair amount of time to meet with him. I quickly discovered the reason I was offered the course is that none of the regular faculty wanted to teach it. The meeting ended up being a short one. While I thought we would be reviewing department learning outcomes and course objectives as well as the department philosophy regarding methods (for example, quantitative vs. qualitative approaches), I was simply handed three textbooks, one syllabus, and was told, “Students don’t like research methods. But they have to take it; it’s a program requirement. Here’s the syllabus and the required textbooks. They provide an overview of what we want accomplished. Good luck!” I realized I had just experienced the quickest professional development seminar ever. The angst I already had been experiencing was now amplified. Even though I was intimidated by the topic, I remained excited about the prospect of teaching methods. After all, I thought, having taught methods would serve me well on the tenure-track job market, which ended up being true, as well as providing me with greater confidence in conducting my own research and data analysis. Increasing my knowledge was important because I had always felt my graduate school methods courses could have been more robust. In my doctoral program, I had taken two required statistics courses and one methods course. That was it. I performed well in the classes, but had a lingering feeling that my knowledge was subpar. To be sure, I was more aware of what I didn’t know than what I did know. Reading articles in the latest journals (such as the American Political Science Review) only reinforced those feelings of inadequacy. Although I didn’t always embrace it, as I was preparing to teach methods for the first time, I would remind myself: my graduate courses provided me with a level of knowledge that was more than adequate to teach undergraduate research methods. When I met the class for the first time, the classroom was actually overfilled because there were students who had wait-listed the course. I knew the overfilled classroom had nothing to do with me. The course was

15

“RESEARCH METHODS: WHO AM I …

185

cross-listed across two disciplines (criminal justice and political science) and was a required course for each discipline. Remember, “they have to take it.” A few years later at my current university, a student who was failing my methods course pleaded with me: “But I have to pass this course. I need it to graduate.” I looked at her and said, “I know you need it to graduate. No one takes methods because they want to. In that class, everyone has to take it. You think the rest of the students are there by choice?” This seems to be the nature of methods. It’s a course students feel forced to take. In any case, back to my first-day teaching methods. I started out with my usual ABD introductory speech explaining that I did not have my Ph.D., which meant they should not refer to me as “Doctor.” Indeed, I said “Mr. Postic” was good enough. But I thought it important to attempt to explain what ABD meant. In the midst of all of this, from the back, one student finally could take no more and blurted out: “Do you even have a degree?!?!” Having a bit of snarky streak to myself, I looked at the student and deadpanned, “No, but I did stay in a Holiday Inn Express last night.” Most of the class found the exchange funny and laughed. The next day, the student dropped the course. That’s how I began teaching research methods. There’s no doubt that my response to the student’s question was a defensive one. I had my own doubts about my abilities to teach the course. But I wasn’t going to allow that to show to students. In academia, it’s common for faculty (part-time or full-time) to experience imposter syndrome. That’s what I was experiencing. Later in the same semester, I finally felt as if I were getting into a rhythm with the course and was doing quite well teaching the concepts and making connections with students. As I was helping students through a statistical analysis, I heard one student say (rather loudly) to his friend: “I don’t even understand the point of this class.” So, I thought, this is research methods. They won’t get it unless it’s spelled out for them. The course really isn’t like teaching Congress or the American presidency. I need to find a way to lay it out in detail.

Teaching Philosophy, Research Methods, and Learning Outcomes I have always embraced a rather practical teaching philosophy that revolves around five “BEs”: (1) be engaging, (2) be knowledgeable,

186

R. POSTIC

(3) be concerned, (4) be respectful, and (5) be adaptable. In teaching research methods, all of my BEs have been challenged, with the first two presenting the most challenges. It’s tough to be engaging in a situation in which students are (a) forced to be there (rather than choosing to be), and (b) don’t have a sense of why the course is important or relevant. Additionally, as I’ve noted, I had serious questions about my own knowledge and capabilities. In the fall of 2006, the only thing I really felt was that, if I were to continue to teach methods, I needed to find a way to adapt and communicate the importance and relevance of methods as well as increase my own knowledge. If I didn’t, the semesters would be long for both me and the students. Approximately 14 years later a lot has changed, but much has remained the same. I have found ways to be engaging and my knowledge has increased, but students’ questioning the point of the course remains. Every once in a while (but much less frequently than previously and certainly not in the middle of a lecture), I receive a comment on a course evaluation of “I never understood the point of the class.” Within those 14 years, a whole lot of learning took place on my part regarding “What is the point of a methods course?” and “How do I make the course relevant?” and “Why am I teaching it?” A related question quickly became, “How can I improve my own knowledge?” We are often given the advice that it is OK to admit we do not know the answers to student questions, and it is appropriate to respond with, “I don’t know the answer to that question, but by the next class, I will find out for you.” And I suppose that advice is fine. At some point, however, we should have the answers to questions students ask. And as the chair of a department, I now find that response more troubling and questionable than I ever did as an instructor. If we do not have the answers, then why are we standing at the front of the classroom and not sitting in one of the seats listening to someone who does have the answers? That was the beginning of my path to effectively teaching undergraduate research methods rather than just dispensing information. It focused on increasing my knowledge, delivering content in a manner that makes students actually enjoy the course, and making the course relevant to students so they see the point of spending 3 hours a week in class with me. As I left the adjuncting treadmill behind and was fortunate to move on to a tenure-track position, my approach to teaching methods was shaped in the interview process for that position. With joining a new institution, a necessary reevaluation of what a methods course should look like took

15

“RESEARCH METHODS: WHO AM I …

187

place. Or maybe better put, for me, what took place was an evaluation of how such a course should be structured, what the learning outcomes should be, and how it should be taught. While these are things I had thought about previously, I didn’t think about it before I started teaching the course. To some extent, that goes with the territory in the world of adjuncting: one takes the leftovers and does the best one can with them, having few choices and no time for reflection. Still, as a regular faculty member, one expects a little more latitude or academic freedom. Even given that expectation, from the initial interview for my new position, it was suggested to me how the methods course should be structured. The vision the department chair had was for the methods course to be connected to the capstone course. Yet, within that context, I had the freedom to consider what my learning objectives would be. One of the learning outcomes I have come to emphasize is helping students understand that they will always be surrounded by research questions. This learning outcome goes directly to the point of the relevance of the course and why they are forced to take it. Many times, we state that we want our students to be “lifelong learners,” but we do not really specify what that means. Additionally, within the context of our methods course, students are challenged to define what a research question is. This is important for my methods course because we tie it to our capstone course. In methods, students are asked to identify a research question, construct a hypothesis, choose a research methodology, and complete a literature review for the research question. In the capstone course, which (ideally) takes place the next semester, students carry out their research projects. Most students collect original data by administering a survey they created. Within this context, I have two main goals for my methods’ students. First, I want them to understand what a research question is and how to begin to address it. Research methods provides students with those tools as it teaches them the research process. And second, I want them to realize they will be confronted with research questions for the rest of their lives, and they need to understand how to address them. That goal teaches them the course’s relevance. To achieve that goal, providing students with real-life examples or illustrations helps students to understand the course is important and relevant. These twin goals of understanding both the process and importance/relevance of research methods are interwoven in both my teaching and research.

188

R. POSTIC

In order to accomplish the first goal of understanding a research question and how to answer it, much of each semester is focused on helping students understand how data are collected, analyzed, and interpreted. In providing students with real-life examples, we are fortunate that we have no shortage of them. We only need to turn to Facebook for the most recent unsubstantiated meme by that Facebook friend we all seem to have or read the most recent research in a peer-reviewed journal. And to be clear, after graduation, most of our students are much more likely to encounter the former rather than the latter. Therefore, I am as likely to use the Facebook example as I am to use a recent peer-reviewed article. That is to say, in my teaching, I use both examples. I believe this keeps the course fresh and interesting. And my point is clear: research methods is one of the most relevant courses to so-called real life, even though students may not understand that when they first walk into our classrooms. I also illustrate the point by sharing my own research, illustrating how we can take what is around us and see the inherent research questions that are presented to us.

Moving Methods to Online Modality Through the first few years of teaching methods, I taught in a face-toface environment. Even so, I had taught a number of other political science courses in an online modality. So, when I thought about offering methods as an online course, I already had considerable experience teaching online. In the spring 2018 semester, I decided to offer methods as an asynchronous, online course. There were a number of reasons for this decision, not the least of which was concern over future enrollment numbers. With more students turning to online courses, I felt this was an area that the program needed to explore. That’s especially so since there seems to be an existing narrative that it is not possible to teach certain courses online. Research methods is typically viewed as one of those unteachable courses. So, besides concerns for enrollment, another reason why I chose to offer the course online is I wanted to demonstrate that a high-quality methods course could be delivered online. In spring 2020, the impact of COVID-19 on course delivery for all educational institutions was substantial, with many universities moving to an online modality for all courses. The decision I made in 2018 seemed fortuitous. My spring 2020 methods course was already online with students understanding what was required of them. That is not to say

15

“RESEARCH METHODS: WHO AM I …

189

adjustments did not need to be made. Given the academic, emotional, and physical impact that COVID-19 had on students, I had to make adjustments to the course. Still, the adjustments were more of tweaks to the course rather than a complete re-conceptualization of the structure and delivery. In a very positive way, the decision to offer methods online also affected my research agenda. In the spring of 2019, I responded to a call for chapters for a book on teaching research methods. My chapter revolves around teaching methods online and whether an online methods course can have similar outcomes in student performance and student satisfaction as face-to-face offerings. Using student records from all the years I taught methods at my current university (2008 through 2020; N = 236), I found no statistical differences in either student performance or student satisfaction between modalities. In both, students are able to succeed and are pleased with the course (Postic, 2021). Based on teaching other courses online, I intuitively believed there would be no differences between the two modalities. To students, I want to communicate that this is the point of the research process and research methods: Do we really know what we think we know? I share this story because I believe it fits well with the narrative I tell my students about research questions being all around us. I now share the above research with my students as a way of illustrating the practical importance and relevance of research methods. In the past, my main example was an original survey I (and my co-author) created and administered to college students revolving around the issue of gay marriage and the use of the word “gay” as a slur (Postic and Prough 2014). Making methods relevant and helping students understand the importance of the course have always been challenges. This is something that we continually need to overcome. I believe that by connecting real-life situations and my own research to the course, students begin to understand its purpose, its importance, and its relevance.

Professional Development Addressing Continuing Angst As I have mentioned or alluded to more than once, I began teaching methods with a sense of angst or inadequacy. For the first couple of years of teaching the course, those feelings never went completely away. After I had established myself at my new university, I was approached by our

190

R. POSTIC

Center of Teaching Excellence to construct a semester-long workshop on SPSS for faculty. While I had come to terms with teaching methods and statistics to undergraduate students, teaching SPSS to my peers was daunting. Before I would agree to do that, I realized I needed more methods training. Thankfully, I lived not too far from the University of Michigan, which houses the Inter-university Consortium for Political and Social Research (ICPSR) summer program in quantitative methods. So, in 2013, I applied for and was awarded a scholarship to attend ICPSR. As I noted in my letter of application for the scholarship: “Even though my graduate education was an extremely positive experience, I have to admit that my methods training was not very demanding. The lack of high-quality statistical training remains a hole in my portfolio that I need to fill.” The summer-long workshop filled that hole. Not only did it provide me with the tools and confidence to better teach my methods course, I felt confident enough to offer a 12-week SPSS workshop to our faculty. It has been quite some time since I had to respond to one of my methods’ students with, “I don’t know the answer to that question, but by the time of the next class, I will have it for you.” Now, my answers are typically along the lines of “That’s a great question. Here’s the answer to your question, why it’s important, and how it relates to the real world.”

Reflecting on the Journey In some ways, when I think about my journey as a research methods instructor, I can’t help but think of the Grateful Dead song “Truckin’” and conclude “what a long, strange trip it’s been.” From the beginning as an adjunct and even into a tenure-track position, what I teach has been defined for me. As an adjunct, one feels compelled to simply do what one is told by those doing the scheduling (whether that is the chair of the department or one of the regular faculty, as was the case in my situation). After all, these are the people who are determining if you’ll be teaching the next semester or not. But even when one is in a tenure-track position, there’s a tendency to do what others have done before. While they are not deciding if you’ll be teaching the next semester, they may have a say in whether or not one receives tenure. So there is an implicit pressure to conform. For our courses, we spend a lot of time thinking about the learning outcomes and course objectives. We define measurable outcomes and

15

“RESEARCH METHODS: WHO AM I …

191

construct authentic assignments to assess those outcomes. And periodically, we reflect on whether those outcomes need to be revised. To some extent, we need to do that with our own teaching. Over the years, I’ve learned a lot of things about myself, about my students, and about teaching research methods. I’ve learned that students walk into the classroom believing they dislike a course they are required to take and will question its relevance. Regarding research methods, this seems to be a universal attitude of students. What I’ve discovered about myself and teaching methods is that the course can truly be one of the most interesting and relevant courses a student can take. Before students believe that it’s interesting and relevant, I needed to believe it. I have also learned that I actually do know a lot about methods. I don’t know as much as some, but I know more than enough to teach undergraduate research methods (and the occasional faculty workshop). I recently completed 13 years at my current university. In the spring of 2020, I received a promotion to the full rank of Professor and am also chairperson of the department. In designing and teaching my courses, no longer do I worry about the impact that others in the department, college, or university may have on my career. I’m well established. While I may worry about other aspects of my job, I don’t worry about that aspect. In those 13 years, I have also developed the skills to be a highly effective methods instructor. But unfortunately, some type or level of angst never goes away. Each semester, I consider questions such as: ● How can I overcome students’ resistance to a course they feel they have to take and are not there by choice? ● How can I get students to understand the relevance of methods and what new examples can I provide students so they understand what a research question is? ● How can I deal with my own angst about teaching methods? I suppose it’s good that I have some remaining angst. If nothing else, it motivates me to approach each semester with a sense of newness. This is not a trivial thing. Faculty tend to teach the same courses each year. It most likely even extends down to the semester. For example, if it’s the Fall semester, I must be teaching Introduction to American Politics. That’s simply the rhythm of academia. While that rhythm provides a sense

192

R. POSTIC

of consistency and familiarity, it can also lead to predictability and staleness. Within that framework, it would be easy for our teaching to become flat and routine. Our students deserve better than that. As long as anxiety over our teaching and abilities does not become something that debilitates, anxiety can be a positive and not a negative. It all depends on our response. Like our students, it is important that we also be lifelong learners of both our content and of ourselves.

Conclusion Now that I am 15 years removed from teaching my first methods course and have accomplished a lot professionally, I have had the opportunity to reflect. But just as importantly, I also have the freedom to make choices about what I teach. I started teaching methods because I had to do so. In one sense, I had no choice. Adjuncts rarely do. Or more to the point, their choices are (a) teach the course offered or (b) don’t earn a salary. While it’s a choice, it’s not much of one. That’s not what my choices look like anymore. I teach research methods because I want to teach the course and not because I have to teach it. Due to teaching the course for so many years and my own professional development, I know much more about methods than what I used to know. While students may have negative perceptions as they enter the course, they do not need to have those perceptions when they leave my course. I can change students’ perceptions by constructing engaging lectures and authentic assignments as well as providing relevant and timely illustrations. The overwhelming majority of my students will not go on to graduate school and become researchers, but they will walk out of my class understanding why they had to take the class and how it enriches their lives. Research methods is unlikely to ever become their favorite class, but it will serve them well the next time they see that meme on Facebook from the friend we all seem to have.

References Postic, R. 2021. Effectively Teaching Research Methods in an Online Course. In Teaching Research Methods in Political Science, ed. Jeffrey L. Bernstein‚ pp. 131–144. Cheltenham, UK: Edward Elgar. Postic, R., & E. Prough. 2014. That’s Gay! Gay as a Slur Among College Students. Sage Open. https://doi.org/10.1177/2158244014556996.

CHAPTER 16

The Inquiry’s the Thing: Teaching Quantitative Research Without Teaching Statistical Software Debra Leiter

Research is an exciting and engaging process from beginning to end. The combination of creativity and rigor that scientific inquiry teaches us extends beyond the academic realm into our social and civic lives. We all do research, all the time, even if we do not think of it as such, from using online review aggregators to friends’ recommendations for products. At a societal level, we have seen the rise of data-driven policymaking, data journalism, and the general rise of big data. We live in a data-rich, research-intensive environment, and there has never been a more important time to understand and use quantitative research. Of course, beyond its immense pedagogical value, research is fun.

D. Leiter (B) Department of Political Science, University of Missouri-Kansas City, Kansas City, MO, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_16

193

194

D. LEITER

But many students shy away from quantitative research, thinking it is beyond them. I maintain that the research process is learnable by all. The key to teaching research design and methods is to give students the opportunity to complete research themselves. By diving into research, students gain hands-on opportunities to immerse themselves in the process, and identify the strengths and weaknesses of the approach. Most approaches to teaching quantitative research rely on using analytical software. Yet this type of programming can make research seem more distant and unapproachable. While we can, of course, make research as complicated as possible (and indeed, many of our reviewers demand it), at its essence, research is a straightforward, systematic process of inquiry that all students can engage in. I have made undergraduate research and research design the cornerstone of my teaching by having students build full research projects relying on online analysis tools to great success.

My Pedagogical Evolution Is there anyone more naively ambitious than a first-year assistant professor? I sprang out of graduate school and into the classroom, bound and determined to infuse empirical political science research into all of my classes. I was certain that my students were going to love research as much as I did, and that they were going to do research “the right way.” However, at the beginning I found that students were neither performing to my expectations, nor responding to the material. Instead of developing enthusiasm for research, I was seeing frustration and, perhaps worse, disinterest. Completion rates for research-based assignments were lower, and anxiety was much higher. Students would come to my office hours perplexed or concerned. I felt that I was failing at a fundamental level. I loved research but my students did not. For two years, I worked to identify what was going wrong, and students proved to be the key to solving the problem. Students would come to my office hours and, once we worked through the assignments together, attitudes changed completely. Until working with me one-on-one, they typically thought of it as something well beyond their capabilities or interests. I realized my initial strategy failed to account for a number of challenges that my students would face in understanding, crafting, and implementing research. It was only through recognizing these problems and demystifying research that students could successfully become researchers themselves.

16

THE INQUIRY’S THE THING: TEACHING QUANTITATIVE …

195

I identified two major barriers for student researchers. First, research language is not intuitive. When we leave graduate school, we leave a supersaturated research environment. We have so internalized this world that we sometimes forget it is a specialized, learned language. This challenge is not just complex research terminology; even fresh off of my degree, I understood that “endogeneity” and “multicollinearity” are not commonplace.1 Even commonly used terms like “variable” or “theory” can throw students off, since we are using them in a very specific and often quite distinct way. One illustrative mistake I made was having students read empirical research without explaining how to read research papers. Without a guide on what to focus on, the table in a research paper is a list of numbers and a graph is simply line-based artwork. Once I taught students the key to reading quantitative research, and started choosing research articles not just for their content but also their clarity, things began to improve. Another issue that I failed to account for in the early days of teaching was a common anxiety toward mathematics. While math anxiety has been well documented (e.g., Ashcraft 2002; Lyons and Beilock 2012), I did not account for it when I began teaching empirical political science. Students frequently confessed to having little mathematics acumen. If they saw numbers and statistics, they shut down. I admit to finding this oversight particularly galling, as I had my own history of math anxiety. When I was younger, I never believed I was particularly skilled in mathematics. I now recognize that this was not based on any particular poor performance, but stemmed from a belief that math was hard. In undergrad, I avoided mathematics classes where possible. When I performed well in these classes, I tended to discount that performance or think that they were not “real” math classes. As research has consistently demonstrated (Ashcraft 2002), math anxiety affects our willingness to try things to a greater degree than it affects our performance, and I had certainly experienced that. By not accounting for students’ worries about their mathematical skills and forgetting my own experience, I was creating an unseen and unnecessary barrier to their success. In order to combat these problems, I had to think about my own experiences as a student researcher. While I use quantitative methods frequently in my own research, I had forgotten how challenging it had been to not just learn these skills, but also be comfortable identifying myself as a researcher. Whether it stems from the well-documented gendered doubt about mathematical skills to imposter syndrome (Clance

196

D. LEITER

and Imes 1978), I had a tendency to discount successes. In revisiting my own journey to becoming a confident researcher, I recalled that I had reoriented my own thinking about what the goals of quantitative research are. While the mathematical underpinnings of statistics are important, in the end, quantitative social science is a way of describing relationships. This self-analysis led me to two conclusions. The first was that I had forgotten, after so many years in academia, that my early worries were likely shared by many of my students. The second was a reminder that while data analytics can be as complex as we need it to be, it is built on a foundation of description and relationships, and that in reminding myself of what was important in quantitative analysis, I had a key way of solving math anxiety by focusing on story. Presented with these problems and concerned both about the quality of student learning and issues of student frustration, I realized that I had not prepared my students to learn about research and research design effectively. I was losing student interest in research because they were getting lost in the weeds of the methods, rather than understanding how the scientific method worked or why they might enjoy it. In order to ensure optimal student outcomes, I had to develop a new approach to teaching empirical research and research design that would require me to think about research in a fundamentally different way.

Teaching Research Using Online Analysis Tools In order to shift my teaching priorities, I had to identify the key goal of teaching empirical social science research: understanding relationships. Science can be viewed in many different ways, but in the end, it is fundamentally a process of inquiry, a way of asking and answering questions that allow us to evaluate the quality of answers. As noted by the online comic strip XKCD, a source frequently used by teachers of quantitative research, “Ideas are tested by experiment. That is the core of science. Everything else is just bookkeeping” (Munroe 2008). In my fervor to teach quantitative social science, I had forgotten to build a solid foundation of social science first. Through conversations with students and colleagues, evaluations of pedagogical research, and not a little trial and error, I developed a new emphasis on applied quantitative science that focused on process over outcomes. By shifting my focus toward process, I was able to free my teaching from certain types of constraints, including the need to teach

16

THE INQUIRY’S THE THING: TEACHING QUANTITATIVE …

197

a statistical programming language, and move toward my primary goal, which was giving students hands-on experience in the process of science. One key puzzle was how to give students the opportunity to perform simple statistical analysis with incredibly high variance in experience with computers and mathematics. Most courses I was familiar with used common statistical programming software to have students analyze data. However, this approach made it challenging to balance all three key skills– substance, research design, and programming–in one class. As I was not teaching a dedicated research methods course, I had to find a way of integrating research into a substantive course, such that the goals of the course were complementary. The answer to this puzzle was to use existing online tools. The explosion of quantitative data, and especially big data, has been met by increased interest in and demand for data access (Grimmer 2015). Empirical data is everywhere, in businesses, media, education, and even entertainment, with websites like Nate Silver’s 538.com gaining attention outside the academic world. This trend, combined with efforts to increase access to publicly-funded data collection efforts and share data resources, has led to a spike in the availability of online analysis tools. These preexisting tools, along with the growth of programs to allow you to create your own, have proved to be a boon for teaching. In Table 16.1, I have listed a few examples of Online Analysis tools and the types of analysis you can perform. Online analysis tools allow users to perform simple quantitative assessments of large data sets. While there is some variation in how they work, these tools allow users to search codebooks and identify variables of interest. Once identified, users can engage in basic analyses, including descriptive and bivariate statistics. Some sites even offer more complex analyses, including time series, multivariate statistics, and heat maps. Importantly, as a best practice, most of these online analysis tools will allow users to produce data visualizations, which means that even simple statistical analyses can be presented using eye-catching graphs and charts. By design, online analysis tools are open to anyone who is interested in examining data. As such, they tend to be user-friendly and self-contained. Using online analysis tools reduces student costs in two ways. First, students do not have to buy software in order to analyze data, meaning that this is an open education resource for students.2 Second, students are not learning a new programming language or software, and can instead focus on the data analysis and outcomes themselves, reducing the start-up

198

D. LEITER

Table 16.1 Sample online analysis tools Online analysis tool Data type

Analyses available

Graphic capabilities

World Values Survey (WVS)

Descriptive Statistics, Cross Tabs (2 and 3 level), Maps, Time Series

Simple Bar Charts and Maps

SDA American National Election Study (ANES)

General Social Survey (GSS) Data Explorera

British Election Study Data Playground (BES) V-DEM

Afrobarometer

Shiny R Studio

International Social and Political Survey Data from 80 countries from 1981–2020 American Presidential Election Survey Data from 1948–2016

Descriptive Statistics, Cross Tabs (2 and 3 level), Means, Correlations, Regressions, Logit, Time Series American Social and Descriptive Statistics, Political Data from Cross Tabs (up to ten 1972–2018 levels), Correlations, Regression, Time Series British Election Panel Descriptive Statistics, Survey Data from Cross Tabs, Time 2014–2015 Series Cross National Time Series, Cross Expert Data on National or Single Country and Regime Country Descriptive Level Political, Statistics, Single Economic, and Social Country Variable Performance Comparison Cross National Social Time Series, Cross and Political Survey national or Single of 37 African Country Cross Tabs, Countries from Maps 1999–2018 Platform to Create Open Interactive Data Analysis Programs from R

Simple Bar Charts

N/A

High quality Pie Charts, Bar Charts, Line Graphs Wide variety of graphical presentation styles

Simple Bar Charts, Detailed Maps, Line Graphs

High quality graphical presentation

a The GSS Data is also available vis the Berkeley SDA archives

costs to quantitative analysis. Faculty do not need to make the tradeoff between either teaching in an unfamiliar language for which the university has a site-license, or requiring students to pay for a semester-long license and increasing the costs of the class. By embracing these tools, I have been able to add data analysis to any class without always having to shift the focus of the course.

16

THE INQUIRY’S THE THING: TEACHING QUANTITATIVE …

199

Implementing the Approach in My Classes Two years after I had begun teaching, I proposed a new class, Introduction to Comparative Politics and Research, for my university’s undergraduate research designation. Many universities had begun to emphasize undergraduate research as a cornerstone of their curriculum, and I wanted to use this new program as an opportunity to test my theory that first and second year undergrads could complete research projects in a substantive course. This course had two goals: teach first- and second-year students the fundamentals of comparative politics and give them the opportunity to produce their own research projects. I designed the class so that two days a week would focus on core comparative politics concepts, and the third lecture period was a designated research workshop. During the substantive days, I made sure to discuss the material emphasizing key research design elements, to reinforce what students had learned in the workshops, and ask students to discuss how their projects related to the substantive material. During the research workshops, we would discuss one step of research design, and then students would have an opportunity to work on their own projects. Students were required to choose a dependent variable from the World Values Survey Online Analysis tool (Inglehart et al. 2014), which allowed students to complete cross-national and over-time analyses of more than 80 countries over a 40-year period. Each week, students would turn in a worksheet that built on their previous one, thus constantly revising and fine-tuning as the project developed. The final project was a poster, presented in person at a poster session held on the last day of class. The first semester teaching this course was equally busy and rewarding. Teaching it the first time, I still fell into some of the same mistakes I had made in earlier semesters. I made adjustments throughout the semester, and found that being an active mentor was critical. While the combination of mentorship and grading responsibilities were certainly higher than a non-research-intensive course, the last day of class was a revelation. Students were so proud of what they had accomplished, and excited to share it with their colleagues. Many students who had told me that they could not do research were bringing their colleagues over to their posters to discuss what they had found. While not every student had the same level of enthusiasm for their project, there was a clear change in how students approached research. Students learned that they could do research, because they had done it themselves.

200

D. LEITER

Since developing this course, I have expanded social science data projects to nearly all my other courses. In British Politics, using the British Election Study Data Playground (Fieldhouse et al. 2019), students write their own public data-based blog on British Politics. Not only does this give students the opportunity to analyze data, but it also gives them a public platform to share their work.3 In my writing-intensive course, I have students use Berkeley’s Survey Documentation and Analysis (SDA) platform for the American National Election Study to identify causes of Congressional discontent, and encourage them to include it in their final paper. In European Politics, I have them write opinion pieces supported by data maps from V-Dem (Coppedge et al. 2020). Whether the cornerstone of the class or simply a one-off assignment, the end result is that students get a taste of what they can do with quantitative analysis, and more than one student has gone on to do independent undergraduate research. By focusing on process and using straightforward analyses complemented by online analysis tools, students realize a world of possibility that heretofore they had not seen.

Strategies for Success As I approach a decade of teaching students research, I have learned that teaching research is a constantly evolving process, and each semester I try something new, which sometimes works and other times does not. Here are a few things I have found to be consistent winners. Enthusiasm: Encourage student enthusiasm. The students that performed best were those who were most excited about the subject they were studying. Faculty play a key role in getting students excited. Encourage them to choose something about which they are curious. Emphasize the creativity of research and the fact that they are contributing something new to the world. Provide opportunities for them to share their results as it relates to the substantive material you are teaching. Show them how much fun research can be. Also, make sure you are as enthusiastic about your own research as you want them to be about theirs. I think we are too cautious in discussing our own research with students, as it is the best way to present an insider look at the process. Not only do you know and love your projects (no matter what Reviewer 2 says), but you are putting a face to it, reminding them that it is not an abstract ‘scientist’ but an actual person behind the

16

THE INQUIRY’S THE THING: TEACHING QUANTITATIVE …

201

work. A side benefit to this is that I have improved more than one of my projects through thinking through how to present it to students. Constraint: Balance constraints and flexibility. When designing the research project, I developed a relatively constrained set of expectations for the project. Students all use the same dataset, develop certain types of theories and empirically testable hypotheses, etc. These constraints, which may frustrate some, set clearer expectations and help you to scale the course for larger classrooms. Once you have set up the methodological constraints, however, let them be as creative as they want. Try to choose datasets and projects that allow for a wide range of questions. This will help them to take ownership of the project, and certainly will make for more interesting projects. Shared Class Project: Have a shared class project that acts as an example for each step in the research design. I choose a project that can be approached from multiple perspectives, which is a great way to get students to brainstorming. By having a concrete example, and working together to develop a plan for the project, students seem to have a much better grasp on expectations. I now have an example project for all empirical assignments in my classes. Paper Alternatives: Consider alternatives to a research paper. In academia, we frequently have to use alternatives to traditional articles: presentations, posters, blog posts, media interviews, etc. By focusing on formats that emphasize the content of the research project, especially posters, students see the building blocks come together. Using formats like blog posts ensure not only that students can engage in public scholarship, but also that they do have a fundamental understanding of the results, since they cannot hide behind technical writing (Fig. 16.1). Process: Focus on process, not outcomes. Students stress about null findings. It is critical to reinforce that their grades and the quality of their projects are not based on what they find, but on how they put the project together. To reinforce this, I often choose a class project where I know my proposed theory will not be supported, to reassure students that null findings are perfectly fine. This is not only an important lesson for students, but a key reminder to us researchers as well.

Conclusion Teaching research design and the research process is important, and yet is a singular challenge. In my early days as an instructor, I forgot my

Fig. 16.1 Poster template

202 D. LEITER

16

THE INQUIRY’S THE THING: TEACHING QUANTITATIVE …

203

own experience in learning research, and in doing so, did not effectively account for student concerns in learning something perceived as distant, complicated, and perfect. In order to effectively teach quantitative social science, I had to go back to the building blocks of research. In doing so, I have developed new strategies that show students not just how to do research, but the value and satisfaction that come from completing your own project. Moving away from using dedicated statistical analysis software to online analysis platforms has been critical in increasing student success. When students complete their own research project, something happens. They learn that they can be the experts, and that they can create, design, and answer questions in an effective and convincing way. The pride and palpable excitement when they present their research to their colleagues is infectious, and it is impossible as an instructor not to feel the same. Teaching research design is not just important for students. Every time, it is a reminder to us of why we love to research and how, in the end, beyond all the complexities and frustrations, the inquiry is the thing that we return to again and again.

Notes 1. Endogeneity refers to the case where the independent variable is correlated with the error term, which occurs when your independent variable is not randomly assigned. Multicollinearity refers to two independent variables that are highly related to each other, such that we cannot tell the independent effects of these variables. As a sign of their obscurity, neither is found in Microsoft Word’s spellcheck dictionary. 2. There are examples of free data analysis software, including the most commonly used in political science, R (R Core Team 2017). 3. For examples of student work, please visit https://ukkcblog.wordpress. com/.

References Ashcraft, M.H. 2002. Math Anxiety: Personal, Educational, and Cognitive Consequences. Current Directions in Psychological Science 11 (5): 181–185. https://doi.org/10.1111/1467-8721.00196. Clance, P. R., and S. Ament Imes. 1978. The Imposter Phenomenon in High Achieving Women: Dynamics and Therapeutic Intervention. Psychotherapy:

204

D. LEITER

Theory, Research & Practice 15 (3): 241–247. https://doi.org/10.1037/h00 86006. Coppedge, M., J.Gerring, C. H. Knutsen, S. I. Lindberg, J. Teorell, D. Altman, M. Bernhard, A. Glynn, and S. Fish. 2020. V-Dem Dataset 2020. Varieties of Democracy (V-Dem) Project. https://doi.org/10.23696/VDEMDS20. Fieldhouse, E., J. Green, G. Evans, H. Schmitt, C. Van Der Eijk, J. Mellon, and C. Prosser. 2019. BES British Election Studies, 1969-British Election Study, 2017: Face-to-Face Post-Election Survey. UK Data Service. https://doi.org/ 10.5255/UKDA-SN-8418-1. Grimmer, J. 2015. We Are All Social Scientists Now: How Big Data, Machine Learning, and Causal Inference Work Together. PS: Political Science & Politics 48 (1): 80–83. https://doi.org/10.1017/S1049096514001784. Inglehart, R., C. Haerpfer, C. Welzel, K. Kizilova, J. Diez-Medrano, M. Lagos, P. Norris, E. Ponarin, and B. Puranen. 2014. World values survey: All rounds— country-pooled datafile version. https://www.worldvaluessurvey.org/WVSDoc umentationWVL.jsp. Lyons, I.M., and S.L. Beilock. 2012. When Math Hurts: Math Anxiety Predicts Pain Network Activation in Anticipation of Doing Math. PLoS ONE 7 (10): e48076. https://doi.org/10.1371/journal.pone.0048076. Munroe, R. 2008. Unscientific. XKCD. March 17, 2008. https://xkcd.com/ 397/. R Core Team. 2017. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing. https://www.R-pro ject.org.

CHAPTER 17

Teaching Research Design: The Gender and Politics Lab and Reflections on the Lab Model for the Social Sciences Amanda Bittner

Introduction In 2017, I founded the Gender and Politics Lab at Memorial University, nine years after I graduated with a PhD and began teaching on the tenure track. Over the last decade or so I have supervised nearly 30 undergraduate and graduate student theses, and I have taught thousands of students at all levels, in classes ranging from introduction to politics to more advanced courses related to gender and politics, public opinion, and voting, as well as graduate-level research design and empirical research methods courses. My approach to teaching and learning has changed substantially over time, but my underlying teaching philosophy has not shifted. The Gender and Politics Lab has allowed my approach to crystallize because the lab provides the framework for a group of undergraduate

A. Bittner (B) Department of Political Science, Memorial University, St. John’s, NL, Canada e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_17

205

206

A. BITTNER

and graduate students to come together to work on their own research, to work together in a collaborative fashion either as research assistants for me or on research projects initiated by them, as well as to work on organizing workshops, talks, and events for the wider community. Students are involved at all stages of the research process, and they work together to design and execute projects. I am there to offer encouragement and guidance, but I seek to be as non-directive as possible, letting them run with their imaginations and pursue projects that excite them and motivate them. In weekly lab meetings, we check-in related to our various projects and initiatives and we troubleshoot issues: students are encouraged to provide critique, feedback, and solutions to one another. Students run workshops to teach each other how to do specific things (e.g., how to scrape Twitter, how to code data, how to use project management technology), they work together to write grant applications to support their research, and they present their own research as a trial run to get feedback before attending conferences. This essay will describe my teaching philosophy, how my teaching practice has evolved over time, how I moved to a lab model of student supervision, and the activities of the Gender and Politics Lab in relation to the teaching and training of research design in political science. Readers will leave the essay with an understanding of the benefits of collaboration, peer mentorship, and co-learning: these experiences boost confidence, produce a sense of belonging, and strengthen the knowledge that students have, as they learn by teaching each other.

Teaching Philosophy In writing this essay I went back to read my teaching philosophy statement from my first annual review, which took place in 2009. My teaching philosophy statement (then) read as follows: I believe that one of the most important services that a university can offer students is not simply to provide them with an opportunity to get a degree and (hopefully) a job, but that it provides students with the opportunity to learn to learn. To think independently, critically, to question what they are told, and to communicate their thoughts effectively – both written and orally – these are the tools that as university instructors we can and should provide to our students. In order to facilitate the acquisition of these tools, I think it is fundamental that my students sense my own enthusiasm (as the

17

TEACHING RESEARCH DESIGN: THE GENDER …

207

instructor) for the course material and for teaching in general, and that I ensure that students feel that they are able to approach and talk to me – it is very important that I make myself available and accessible to students. Connecting with students, in my mind, is only really possible if we enjoy what we do, we think it’s important to convey that enthusiasm, and we allow students a “window” into that enthusiasm by ensuring that they feel comfortable talking to us.

I pursued a graduate degree to become a professor because I love what I do–I love what I research, and I love teaching others all about politics and the institutions that frame how we operate on a daily basis as political beings in Canada. As an undergraduate, I experienced first-hand what it is like to take courses from enthusiastic instructors as well as instructors who were less interested in the course materials, and there is no question as to which teaching style I benefited more from as a student. The first Canadian Politics course I took as an undergraduate was taught by a young professor who was extremely enthusiastic about what he was teaching us, and he made me believe that politics was fun, that federalism was fascinating, and that it would be really cool to sign up for a weekly email alert about current cases being heard by the Supreme Court of Canada. I was receptive to all of these ideas because of this professor, and it is because of his enthusiasm and his desire to ensure that we “got” what he was teaching us that I learned so much in that class. Much of my own personal teaching style emulates what I saw as a 19-year-old undergraduate, and I try very hard to ensure that I communicate my own enthusiasm for both Political Science and my own specific research area when I teach a group of students. It is this excitement for research, combined with the instructor’s willingness to be approachable and accessible that motivates students to learn and to get more out of the university experience. It has been my goal to emulate this model in my own teaching career. There is very little about this teaching philosophy that is not still true. I think conveying my enthusiasm and love for the material is one of the most important influences I can have on my students, and letting them know that I am there, and available, to discuss classroom materials, research questions, research design, their theses, and so on, is something I remain committed to. Recently I spoke with a graduate student who I am supervising: she told me that she was excited to talk to me that day because nobody else in her life wanted to discuss her experimental design the way I do and that she was feeling lonely with her work. Of

208

A. BITTNER

course, I understand this, my family and friends also don’t like to discuss survey design or the nuances of Likert Scales versus other types of question formats, but I do so love the opportunity to discuss these things with my colleagues and students, and I do my best to give them the tools and opportunities to discuss these things with each other when I am not around.

How My Teaching Practice Has Evolved Over Time My teaching practice, when I first began teaching, was centerd around making myself available and doing my best to be responsive to student queries and engage in discussion with them whenever they wanted to– lots of office hours, chats in hallways, lingering after class to talk about the material covered that day, and so on. I still do these things, but the main source of change over time in my teaching practice has evolved to include two key things: encouraging creativity and emphasizing collaboration over competition. 1. I encourage students to be creative in their course assignments now, rather than sticking to traditional essays or tests. Many students flourish when they are given the opportunity to make different choices, and I have encouraged students to consider submitting videos, podcasts, plays, short stories, and other creative endeavours, still based on research. Not all students take up the opportunity to do this, and many still submit traditional essays for their final projects, but I have received some really first-class work from students who relish the opportunity to showcase their knowledge and research through their creativity rather than feeling boxed in by a traditional essay. 2. I encourage students to work together and discuss their common problems as much as possible. In my graduate empirical methods course, for example, I encourage (usually terrified) students to work together through their labs and help each other with coding and trouble-shooting their assignments. I encourage my senior graduate students to talk to and mentor my junior graduate students, encouraging them also to complain about me to each other. One of the most challenging things about graduate school is feeling alone and confused, and imposter syndrome is a constant issue for all students. I remind my students regularly that I still feel this way, and that

17

TEACHING RESEARCH DESIGN: THE GENDER …

209

they probably all feel this way, and that they don’t have to feel this way alone, but that talking to each other about their problems, whether linked to courses, their research projects, their theses, or something else, is worthwhile and the best way to get through the most challenging of days. I do my best to be open and honest about my own challenges, perceived inadequacies, and self-doubt. The fact of the matter is, most of us feel like this most of the time. And most of us face similar challenges in our research process (e.g. narrowing down a research question, determining what data we will use or collect, sorting out how we will actually analyze those data, feedback from others with requests for revisions, and so on), and encouraging students to talk about these issues with one another honestly and from a point that centres the research rather than the researcher, means that students are able to recognize their commonalities and see each other as collaborators and colleagues rather than as competitors. The more opportunities students have to work together, rather than working independently in isolation, the better. Thinking about research and research questions collaboratively allows for a greater level of creativity and allows students to generate new and interesting ideas, through discussion and opportunities to engage with each other. The lab model works wonders and allows me to bring students together to work with one another in an official capacity. This allows me to take on a cheerleader role and enthusiastically support their collaborative creative process.

How I Moved to a Lab Model of Student Supervision I was motivated by a couple of colleagues from across disciplines at my university who had begun their own labs and were pushing the boundaries of how we work together, teach, learn, and mentor in the social sciences. I have had a large teaching and supervisory load since the beginning of my career–for many years I was the only woman in my (small) department of approximately 12 faculty members, and I did the bulk of teaching and training related to gender and politics, despite that this was not my area of research specialization before I joined my institution. As a newly emerging

210

A. BITTNER

scholar of public opinion, elections, and voting, I quickly pivoted toward gender and politics in my first year on the tenure track, as I was suddenly highly aware of my gender as the youngest, most junior, and only woman in my department. In 2017 I had a conversation with a junior colleague in Political Science, I been lamenting that I was giving my students the same advice over and over, year after year, and expressed my desire to get them to work together and mentor each other a bit and work collaboratively on projects. She suggested that I should start a lab, like the one that my colleague Dr. Max Liboiron had begun a couple of years prior (she is a member of the Geography Department at Memorial and runs the Civic Laboratory for Environmental Action Research (CLEAR), which is a feminist, anti-colonial lab specializing in monitoring plastic pollution). I looked at my colleague like she had grown a second head, “Can I really do that? What the heck would a lab in political science look like?” and she said of course I could do it, and that I should talk to Liboiron about her own lab and how it worked and ask her for advice. So, I did. Liboiron generously met with me, gave me a tour of her lab, invited me to attend a lab meeting, and shared her lab’s foundational documents and lab manuals with me. A short while later I met a new colleague in the Human Kinetics Department, Daniel Fuller, who holds a Canada Research Chair in Population Physical Activity at Memorial— we met because our kids were going to the same daycare. He runs a lab called the Built Environment and Active Populations (BEAP) lab, and it is filled with students and post-doctoral fellows from across disciplines: social sciences, computer science, statistics, and human kinetics. He met with me, gave me a tour of his lab, shared his onboarding documents with me, and discussed his lab’s workflow and collaborative processes. I still didn’t know how to run a lab or what my lab would “do”, but thanks to these colleagues I had a better sense of how other social scientists were running their labs, and how students were working collaboratively on projects related to the lab’s mission, and the work of the lab’s principal investigators. While I didn’t quite feel like I knew what I was doing, I felt like I knew that I wanted to start a lab, and I decided to do it, and founded the Gender and Politics Lab that fall. I invited all the students I was currently supervising to join the lab (they were not required to join the lab) and then sent an email to a few other very bright students who I thought might be interested in working collaboratively as a group. Nobody said no to joining, but nobody felt like they

17

TEACHING RESEARCH DESIGN: THE GENDER …

211

knew what their role would be or what a lab was. That first academic year, 2017–2018, was entirely experimental. We defined our activities as we went, carving out priorities that interested us, applying for grant funding to do research in our focus areas, and organizing university-wide and community events on topics that appealed to us. Our lab meetings involved troubleshooting issues that came up related to these activities, as well as a great deal of time spent thinking about what it means to work in a lab as a social scientist. Four years later, I’m still not 100% sure I know the answer to this question. Unlike science labs, ours is small, without steady funding, and with limited physical space. My students are paid out of a series of tiny grants that I cobble together from year to year: 2019–2020 was the first year in which every lab member was on my payroll, all for about 5–7 hours a week of work throughout the year. We have a small office on the far end of campus, it fits four desks. I have repurposed two of my old computers so that students have access to technology in this space. The office does not fit all lab members, and we cannot meet there, so once a week I book a board room on campus so that we can get together in person for lab meetings, and we sit around a big conference table. We begin with a check-in, where we go around in a circle to discuss how we are doing (these updates can be both personal and professional, depending on what a student feels they need to talk about), including an update on all the projects we are leading and working on, issues we have encountered on those projects since our previous meeting, and any other thing that we want to discuss. I go last. Along the way, students chime in and provide each other with support and advice and with tips and tricks if they have them. These check-ins provide students with an opportunity to be honest and vulnerable with each other, and on more than one occasion someone has cried (including me) about some issue that has popped up, and others commiserate and try to help. Sometimes it is not possible to help. Research ideas have come out of these check-ins—for example, we began a research-based twitter campaign with a hashtag #sorrynotsorry because of frustration one student expressed over the way that women seem to need to apologize for their behavior and successes. By going around the circle in this way and giving each person space to check-in about their projects, discuss issues arising in their lives, and vent about things they see, students are able to help each other and generate new and interesting ideas. This approach facilitates community, collaboration, and creativity.

212

A. BITTNER

Activities of the Lab in Relation to the Teaching and Training of Research Design in Political Science At the graduate level, I teach empirical research methods in political science, and I spend a lot of time in that course getting students to think carefully about research questions, operationalizing their concepts as variables, and designing feasible research projects of all types. In the Gender and Politics Lab, we spend a lot of time on these same activities, only the focus is on getting students to identify issues they care about, questions they want answers to, and how we might actually execute these projects, whether on an individual level or as a group. In addition, students in the lab work as research assistants for my ongoing research projects, and are able to develop a number of skills related to research, including grant proposal writing, literature reviews, data collection and entry, data analysis, and writing up and presenting results. I try as best as I can to get my students to be involved in my own research activities, and to show them the complicated, dirty, and imperfect research process, to familiarize them with the massive extent of failures and rejections I experience, and to show them the process of submission and revision of my work. Over the last few years, my research assistants have moved from simply helping in the background to being full-blown collaborators with coauthorship roles, including designing their own grant proposals to fund research that interests them, ethics proposals, gathering data, analyzing those data, and presenting their work at conferences. By providing them with an insight into this process, they have all made major headway with their own projects as well, having a greater understanding of how we formulate research questions, how we ensure that our writing is clear and concise, and how to navigate the peer review process. This type of faculty-student collaboration is increasingly the norm in the social sciences, and in our lab we have begun to have conversations about the ethics of co-authorship, author order, and I wonder if the science model of authorship is something we should consider more seriously in the social sciences. On student-generated projects, students are clearly co-authors with their names listed at the top of the list in the author order, but I think we need to start asking ourselves whether research assistants should also be listed as authors on papers, even if they take a very junior/supporting role. I do not have an answer to this question, but increasingly I have been thinking that if students have been paid

17

TEACHING RESEARCH DESIGN: THE GENDER …

213

to work on a research project, perhaps they should be included as authors on that project. The lab is currently working collaboratively on three main student-led projects linked to gender and politics, all of which are in different stages of data gathering and analysis, and two of which they actually applied for and received grant funding to gather data—one project on the role of women in local politics in the province of Newfoundland and Labrador, and one on childcare policy in the province. I am incredibly proud of these brilliant students, and few things make me as happy as watching them work collaboratively and constructively on joint projects. Probably the only thing that I enjoy more when I watch them do their work is how supportive they are of each other and how much they try to help each other in other aspects of their lives as well.

Conclusions Many colleagues have asked me about the lab, whether I like doing it, and whether it cuts down on my supervisory load. My answer to this question is that I love the lab model, I am confident that it provides students with a greater learning experience than they could get either in the classroom or by being supervised by me in a traditional supervisory model. It is more work, however, not less, to run a research lab, but the work is more rewarding. I get to watch my students do extraordinary things, with each other, things that they would probably not do if they were simply in a regular bilateral supervisory relationship with me. Their orientation toward research shifts because they work together in the lab: they learn more, they do more, and they inspire each other to do things that they otherwise would not do. Of course, the lab model is not without its challenges. Yes, it is certainly labor-intensive. It takes a lot of work to coordinate a group of students this large, and there have been situations that, frankly, I have not really enjoyed having to deal with. There have been times where not all lab members have pulled their weight, and I have had to help students navigate project management where there is a non-committed team member who consistently does not deliver on promises, thus delaying the project. There have been episodes of interpersonal conflict between lab members, and we have had to have conversations about how to work as a team, how to collaborate, how to work (and disagree) respectfully, and so on. Like any team, collaboration is rewarding but there can be hiccups along the

214

A. BITTNER

way. Leadership is challenging, team management is challenging, but the students are learning (by doing and by seeing) how to work with others and how to manage stress, conflict, and how to move a project forward from start to finish. I do not think that I would revert to my old way of supervising students on a one-on-one basis. I am still learning how to run a lab successfully, and I have made many mistakes. Running a social science lab is incredibly rewarding, and what has made it even better in recent months is hiring a highly capable student as a lab manager to help manage projects, keep track of student hours and availability, and to keep me on task. As a student recently said to me about our new lab manager, “OMG this is working so well, she is so good at managing us and our projects and because of her I feel calm and cool about all the things we are doing and my role in all of it. I love working with her,” and I can say without a single doubt in my mind that this is 100% accurate. Having an incredibly capable lab manager has meant that I am no longer the coordinator of this group, and I can step back from this role to allow students to lead, to work together more cohesively, and to flourish. I am reminded on a daily basis of how incredibly privileged I am to get to work with students and to be able to walk alongside them as they collaboratively learn and develop their leadership, management, research, and writing skills.

CHAPTER 18

Researching & Teaching Political Science Through Arts-Based Inquiry Methods Michaelene Cox

What role does art play in political inquiry? I wonder if a sense of self-doubt accompanies most instructors who set about to transform customary research practices in their classrooms. In my case, a feeling of adventure and even liberation characterized the early planning stages of a new upper-level political science seminar, but nagging concerns quickly took root as the semester approached. Yes, I believed there was value for introducing some degree of art education and visual arts-based research methods in most any social science course. But perhaps from naiveté and inexperience, was it just a case of hoping so? Would my students meet learning outcomes connected to a seminar topic about human security through a novel alternative to so-called standard research activities in our discipline?

M. Cox (B) Department of Politics and Government, Illinois State University, Normal, IL, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_18

215

216

M. COX

It is fortunate that nowadays a greater number of students are exposed to research design in their political science courses and becoming, we hope, more skilled at negotiating questions through some of the quantitative and qualitative approaches they have learned. At the same time, however, visual literacy is not widely cultivated by most of those devices adopted, and yet arguably remains a badly needed skill for advancing career and lifelong learning development. This presents an opportunity for us to build visual competences and cultivate fresh perspectives of a subject matter by engaging students in learning activities that, among other things, entail systematic observation and interpretive analysis of imagery. Thus, this chapter serves as an introduction to arts-based research design (ABR) to augment qualitative choices that students of political science typically face when assigned a research project. Gaining an appreciation for ABR can concurrently lead faculty to adopt such practices for their own research projects, including Scholarship of Teaching & Learning (SoTL). Arts-based research can help bridge the gap between science and art with attention to creativity, critical analysis, and aesthetic expression, and offers a transdisciplinary and holistic way to examine complex problems. Applying creative arts processes to political science research projects is instructive and exciting, and as this autobiographical reflection essay aims to convey, can begin with even the smallest and most tentative of steps.

Responding to Convention After nearly two decades of teaching undergraduate and graduate students, and experimenting with various approaches along the way, I continue to think about what works and what doesn’t work in leading students to greater understanding about a subject matter and an appreciation for the transdisciplinary nature of political studies. At the same time, I also think about ways to help them develop analytic skills and the confidence and creativity to approach problems in and out of the classroom with competence. It’s fair to say that teaching continues to be a work in progress and perhaps for some of us, kickstarted by heredity. My grandmother was a country school teacher back in the days when an eighth-grade education provided her with sufficient credentials to teach in rural Kentucky. She was a perceptive and outspoken woman who believed that I had the makings of a good teacher, too. But you can’t teach without learning, she emphatically stated, so she handed over a pamphlet that

18

RESEARCHING & TEACHING POLITICAL …

217

argued on behalf of Alaska’s statehood and a large chalkboard. My career in education subsequently began as a young girl scrawling the alphabet and drawing maps of our newly forty-ninth state for the edification of a younger sibling and neighborhood children. Learning to teach others to learn, needless to say, is a dynamic process and often fueled by our own sense of wonder and discovery. My professional teaching experience emerges from quite dissimilar environments, from a small private liberal arts school with just over 600 students to now a large public institution with more than 20,000 students. Altogether I have taught more than 20 different courses in the fields of political science, history, peace studies, and geography to diverse student populations: majors, minors and those outside the discipline; freshmen to graduate students; veterans and other non-traditional students; students from different cultural backgrounds and from rural and urban environments; first-generation college students; and those ranging from the least-academically prepared to the most accomplished honors students. Readiness and adaptability to different student populations, institutional cultures, and subject matters is made easier because my identity as a teacher is one and the same as that of a lifelong learner. The roles are both informed by unbounded curiosity and a belief that teaching and learning is a problem-solving enterprise. This likely accounts for a willingness to experiment with new ways of knowing. But it wasn’t always so. Years of statistical training in graduate school and a quantitative dissertation put me on track to teach and research from that same methodological orientation. It was, after all, comfortable. Plus, for quite some time, the need to juggle a wildly diverse course load was also needed. As colleagues were gradually hired to specifically teach methods and enable me to focus on topical courses, I began to feel a disconnect. Motivation dwindled to continue scaffolding research projects in my courses simply to prepare students later for their methods courses. The latter primarily focus on quantitative approaches, with most undergraduate and graduate students appearing to tolerate stats just to meet our program requirements. It was a re-awaking of sorts to recognize that a variety of non-quantitative methods often gets short shrift in some political science curriculum. I realized that case study and ethnographic methods, for example, hold much appeal for students, and that they are indeed suitable tools to pursue many of the questions raised in my courses. To gain greater competency and experience, I enrolled in some graduate

218

M. COX

research classes from various other disciplines that emphasized qualitative approaches. Soon, the nature of my student research assignments shifted, as well as the methodological directions of my own scholarship. Here now is fair warning to readers who wish to follow that model of professional development—those sporadic efforts in continuing education can be addictive and quickly morph into a colossal transformation of an academic life! A brief account of this, and how arts-based research made its way into my classroom, comes from pondering the atypical nature of my research agenda. After all, there are too many discoveries ahead about the politics of life to take just one route. A foray into the world of art came about while pursuing a graduate degree in visual studies after earning tenure. The potential of visual arts, with its ability to generate unexpected ways to conceptualize and resolve political dilemmas is intriguing, isn’t it? Vision and artistic expression are political. How might politics be visual? Exercising the freedom to work both within and outside traditional boundaries in such ways should demonstrate a commitment for learning to think and work in innovative ways, and for communicating the significance of this in the classroom. When introducing visual discourse to my students in international relations, international law and human security, they immediately intuit its value for interrogating problems of social justice, locations of power, armed conflict, national identity, and other political matters. When conducting faculty SoTL workshops, I find that instructors from diverse disciplines readily appreciate an opportunity to similarly engage in cross-border adventures. These efforts plainly illustrate the symbiotic relationship between research and teaching practices for the scholar–teacher.

Adopting ABR for the Classroom As the opening paragraph of this chapter indicates, I wonder at the implications of introducing projects in the political science classroom that borrow methods from outside the discipline, or at least contravene from established program norms. For instance, if my students pursue research questions by drawing upon images that they have created, collected from archival records or from study participants, what criteria might be used to evaluate learning? A review of the literature suggests that there is considerable ambiguity in assessing merits of ABR in general, and further, indicates that there is much room in the scholarship and practice of art

18

RESEARCHING & TEACHING POLITICAL …

219

education to design appropriate evaluation instruments for measuring student achievement. Yes, I mention art education here! A search for greater teaching and research competency in political science these days takes me beyond visual culture domains and into the field of art education as well. Visual literacy requires shoring up on theory and practice from an assembly of communities, despite some fundamental tensions between them. Defining ABR. A pioneer of arts-based research in education, Elliot W. Eisner noted that differences in scientific and artistic methods of inquiry fall on a continuum of qualitative research such as the basis of knowledge, ultimate aims, and criteria for appraisal (Eisner 1981). Although spirited debate about qualitative research and the nature of scientific practice had been swirling about for at least a decade or so, a turning point in the intersection of art and research was reached by the 1990s when scholars widely acknowledged a place for arts-based practices within an assemblage of previously well-established qualitative paradigms. Those new practices were heralded by some as an innovative methodological genre (Sinner et al. 2006). Nevertheless, recognition of arts-based research has still not produced a fully decisive description of the practice. For example, differences between research that uses art and research that is art remain open to discussion today (Smithbell 2010). It’s common to simply regard ABR functions as a set of methodological tools that employ artistic processes at any stage of inquiry, and although typically utilized in the representation phase of social research, can also be used to collect data and conduct analysis and interpretation across a variety of disciplines (Leavy 2015). Those creative processes can draw from the visual, literary, performing, and sound genres. Some examples of visual ABR tools especially useful in addressing political science questions in my classroom include photo essays, photo-voice, and photo-elicitation. Integrating the creative arts in a pluralistic methodological fashion in such ways can offer a holistic, evocative, and empathetic understanding of the world. While there are still unexplored arenas and even tensions about its forms, rigor and applications, the adoption of visual research methods continues to rapidly expand and inform art teaching and learning practices. Visual arts-based research has found a relatively comfortable place within the field of art education to flourish. Likewise, ABR is moving from the margins to becoming more widely used in a number of social science disciplines such as anthropology, geography, history, and sociology. Several of my colleagues in other departments regularly instruct

220

M. COX

students in visual ethnography, for example. It helps that such fields are already populated by a good share of interpretivist scholars. On the other hand, visual inquiry has not been embraced yet by many in the field of political science; the discipline is dominated by those who value conventional positivist methods. One of the major barriers to widespread acceptance of ABR in some disciplines such as political science appears to be epistemological disagreement about what constitutes valid interpretive research. I don’t aim to review nuanced methodological-philosophical grounding of truth claims or to otherwise legitimize interpretive methods when stacked up against quantitative and other qualitative traditions. We go back to Eisner’s argument that arts-based inquiry warrants different criteria. How best then to cultivate an environment of methodological tolerance and plurality than to initiate students into the “dark arts” of ABR? Perhaps we can view a pilot class project as means of active participation or even resistance to the status quo, while at the same time enriching artistic expression and appreciation of visual imagery among those students.

Designing for an Undergraduate Senior Seminar As a political science teacher and as a student in art education, I’m interested in exploring ways to integrate the use of art, particularly the photographic image, with qualitative forms of research design. I wish to find a way to “demonstrate more direct and creative ways of engaging the imagination in scholarly inquiry,” as one scholar describes the potential for ABR (McNiff 1998). This means that using images for representation, symbolic interpretation, or social critique should help identify, understand, and address contemporary global and local issues such as social justice, and should animate the researcher to creatively and effectively communicate those perspectives. Integrating ABR practice into either an undergraduate or graduate-level international relations seminar is not a far-fetched scheme. Justification for the ABR model comes from following three familiar teaching and learning principles to pursue experiential, inquiry-based, and project-based activities (Marshall and D’Adamo 2011). These pedagogical aims should have merit in most, if not all, of our political science department offerings already. Without doubt, previously enrolling in two graduate art education courses put me on this ABR track. Assigned readings in Foundations in

18

RESEARCHING & TEACHING POLITICAL …

221

Art Education, and work on a class research paper provided the foundation for learning about and appreciating art in the classroom and introduced me to this approach to research design. Subsequent readings in Curriculum in Art Education and completion of an assigned curriculum portfolio, produced the nuts-and-bolts for redesigning my political science seminar to include discussion of visual art, artmaking, and an ABR student project. The primary theme envisioned for the seminar was “spaces and places,” with the community as a site of inquiry in which perceptions of threats or potential threats to individual well-being and dignity can be made visible. Three curricular units or modules were planned: one unit devoted to student learning about human security from assigned readings, videos, and discussions; one unit to an introduction of visual studies and visual arts; and the final unit to arts-based inquiry and artmaking in context of an assigned human security research project. My aim was to equip students with new tools, and so I set about plotting execution of the newly designed course for the following year. I titled the thematic topic of the seminar, Visualizing Human Security. Fast-forward to a week before the course began—panic set in. Did I lose political science and human security among the heaps of art education and visual culture? I quickly revised the syllabus, paring down the arts-related PowerPoints to a basic minimum, switching some artmaking exercises with lab time for how-to-make-a-research-symposium-poster, and reducing the number of ABR journal readings. Then I worried about losing aspects of the artistic process. On the up side, my rallying call for bridging science and art was met with wide eyes and smiles when classes commenced. The senior seminar generally attracts only a handful of students each semester, regardless of instructor or the particular topic chosen to tackle. It was not disappointing then to find only four majors and one honors student enrolled. It was a golden opportunity to foster greater interaction. Learning Activities: The catalog description for the course specifies that a cumulative research paper be produced. Introducing artmaking practices in light of strong positivist research traditions in our department was risky. Yet I was convinced that integrating creative arts in a critical exploration of local and global social justice issues, such as addressed by my seminar topic about human security, could lend itself to developing informed, holistic, and empathetic perspectives. The literature seemed to bear this out. So in lieu of a standard political science research paper, I assigned each student to create a photo essay as

222

M. COX

a capstone. Each project would include 10–15 original photographic images created by the student, along with accompanying textual narrative about a specific dimension and interpretation of human security at our community level. Art practice as a research process was emphasized. Thus, each student began by researching major themes or big ideas they wanted to pursue in their photographs such as poverty or environmental threats, formulating a research question, and then conducting a literature review and exploratory visits through the community. Subsequent stages in the inquiry process included visual analysis/interpretation and discussion about the contribution of the study. Without exception students were cheerleaders for the unique course format and content, willing participants as we moved from one activity to the next, and good sports as adjustments were made along the way. Learning activities included studying photo essays produced by several contemporary artists who tackled human security issues such as Matt Black, Edward Burtynsky, and Fatemeh Behboudi, and then researching those artist statements and processes. At the same time, they also wrote summaries and critical analyses of assigned readings about human security and ABR practices. Other activities included two photo critiques of student work in progress. The peer review sessions gave students a chance to reflect and talk about their creative inspirations and approaches, and how the work of studied artists informed them. This prompted students to experiment further with the photographs they took or with how they subsequently edited them, and to flesh out writing their own artist statement. Learning Outcomes: My own reflection on aspects of teaching and learning in this case is gratifying. It’s noteworthy that at the start of the semester all five students said they had no art history or art studio courses while in college and had only some cursory exposure in high school. Each one further claimed they had no artistic skill whatsoever. “I’m glad we’re not drawing,” said one. Yes, they all had taken photos with their iPhones but none had used another type of camera. The photos they previously took were selfies, portraits of friends or family, or to document some events. When asked if they gave attention to artistic or aesthetic qualities of those photos, they shook their heads. “I just want my photos not to be fuzzy,” said one. When asked if they ever took photos to serve as metaphors or to challenge ideas or values, again they shook their heads. One student shared a collection of photos he took of his dog. “Is this what you mean by a photo essay?” he asked. All of this astonished and

18

RESEARCHING & TEACHING POLITICAL …

223

yet delighted me at the same time. Despite trepidations about teaching art for the first time to political science students, I was sure then that these students would learn something of value about telling stories through imagery. Two of the arts-related goals articulated in the syllabus were to apply basic knowledge of terminology and concepts utilized in visual arts and visual studies, and to produce creative works that demonstrate informed perceptions of human security threats at the community level. A number of formative assignments, such as peer photo critiques mentioned previously and response papers, demonstrated steady improvement in the way students wrote and talked about human security and related visual images drawn from media reports. Many of the images they selected to attach with their short response papers went beyond simple representation to include some that we could discuss in more sophisticated interpretive ways guided by analytical frameworks, for example by considering techniques, ambiguity, and emotive elements. As the semester progressed, I had to try to contain the craziness that would sometimes erupt from overenthusiastic competition between students when reading some images. It was as though a new vocabulary and viewing lens suddenly removed inhibitions. Admittedly, though, it was fun sometimes to join in episodes of exaggeration and absurdity for the sake of levity. I was immeasurably glad not to witness apathy or scorn in any of our sessions. Another formative assessment took shape of think-pair-share exercises when students engaged with a variety of art presented in PowerPoint slides. Most indicated at the beginning of the semester that they hated modern art, specifically abstract expressions, but by the time we looked at some samples on the slides and applied both traditional and postmodern principles to explore processes and meanings, even those cynics admitted to at least understanding the work better and appreciating the role that politics play in defining what is art. One summative assessment was completion of a poster and its presentation at our university’s annual research symposium. The poster assignment was to demonstrate work in progress on their photo essays, but also to help students improve graphic arts skills such as composition and use of color, and to give them practice telling stories through images and text. Students had by this time taken enough photographs that they could select 5–6 to include on the poster and had already done some preliminary research and writing. In my view, the poster assignment was a perfect segue to a polished final photo essay. Work on the poster consumed

224

M. COX

a great deal of time, both in and outside of class time, with student unfamiliarity with software being a stumbling block, along with an overwhelming array of design choices. The primary summative assessment of learning outcomes rested on the final photo essay. This project was graded using a rubric composed of ABR criteria found in the literature. Overall, the photo essays were remarkably strong. Without exception, albeit to varying degree, students demonstrated an ability to apply basic visual arts terms and concepts to the photographs they created. The textual portion of the essays about human security, plus the organization and subject matter and style of photos, conveyed reasonable narratives in response to their research question. Some photos served as realistic depictions of the subject matter, others showed more nuanced approaches for eliciting emotive or subjective interpretations. Without question, however, the photographs succeeded in serving as more than mere illustrations and instead demonstrated they served as data. Interpretation and critique of human security issues at the local level emerged from that data and were articulated with extensive written commentary.

Last Thoughts It was not designing a new curriculum for the senior seminar that was the most challenging. True enough, plugging learning materials and activities here and there in the schedule to address learning objectives for the three major components of the course—human security, visual culture, and artmaking—required tinkering to accommodate snow days, student interest and unanticipated delays. Nor was it taxing to cultivate and maintain student interest throughout the semester. On the contrary, I found myself shooing students out of the computer lab or classroom when the time came to leave. Not even in my most popular and longstanding courses, with some of those including the study of human security as well, was there this kind of attentiveness before. Indeed, I felt a unique connection to these five students that was undoubtedly fostered by mutual curiosity and attraction to the arts. The truly challenging aspect of this semester’s experience was instead dealing with my nagging sense of self-doubt and the accompanying anxiety that things would completely fall apart—that the experiment to bring art education into the political science classroom would fail. On paper, it looked feasible and even laudable. But in practice…? Certainly, it wasn’t a perfect execution of a perfect plan. But there was enough that succeeded to encourage future attempts

18

RESEARCHING & TEACHING POLITICAL …

225

to integrate artmaking and social science. There are some lessons learned that will carry forward to conducting another ABR-enhanced course. For one, I see much value in assigning students to work on visual process diaries and journaling from the very beginning of the inquiry process. Keeping track and reflecting on creative ideas and observations can be as productive as any field notes. For another, I will not second-guess the interest political science students will have towards visual art and so will trust they’ll be willing to invest significant time with more arts-based learning materials. In short, this chapter ultimately argues that as a site of content knowledge and lifelong learning skills, as well as fertile ground for personal and group identity formation, classrooms in higher education are enriched by visual literacy. Influences on student development are multiple and nuanced, and furthermore, subject to changing sociocultural environments. The twenty-first century has already been characterized as an information society and an exceedingly globalized one in which we will find changes in politics, economy, and culture taking place now at unprecedented pace. The language of art has become one of the most critical avenues by which we can navigate complexities of the self and the other. Arts-based research design holds much promise for enlivening and enlarging student learning experiences in political inquiry.

References Eisner, E.W. 1981. On the Differences Between Scientific and Artistic Approaches to Qualitative Research. Educational Research 10 (4): 5–9. Leavy, P. 2015. Method Meets Art: Arts-Based Research Practice, 4. New York: The Guilford Press. Marshall, J., and K. D’Adamo. 2011. Art Practice as Research in the Classroom: A New Paradigm in Art Education. Art Education 64 (5): 12–18. McNiff, S. 1998. Art-Based Research London: Jessica Kingsley Publishers. Sinner, A., C. Leggo, R. Irwin, P. Gouzouasis, and K. Grauer. 2006. Arts-Based Education Research Dissertations: Reviewing the Practices of New Scholars. Canadian Journal of Education 29 (4): 1223–1270. Smithbell, P. 2010. Arts-Based Research in Education: A Review. The Qualitative Report 15 (6): 1597–1601.

CHAPTER 19

Embedding Feminist Pedagogy in Political Science Research Design with Reflections on Critical Theory and the Social Construction of Reality J. Cherie Strachan

Starting with Positivism When I first began teaching research methods, I did not question the epistemology of positivism. Nor is it likely that I could have. Positivism was overtly taught in every undergraduate and graduate social science research methods course I took, and adherence to positivism’s criteria for achieving scientific insights were used to assess contributions to disciplinary knowledge in all of my substantive political science classes. Most of my professors were mainstream political behavioralists who proselytized the tenets of positivism, and who kept critical theory at the periphery of

J. C. Strachan (B) Department of Political Science, Virginia Commonwealth University, Richmond, VA, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_19

227

228

J. C. STRACHAN

my socialization into the discipline. When I began teaching social science research methods, I relayed the same mantra to my own students: with my guidance, due diligence, and practice, you too can conduct objective, scientific research free from the normative biases that constrain other ways of knowing. In short, I taught them that political science—as indicated by our selfchosen title—is a science, which restricts our topics of study to political phenomena that can be described and measured through empirical observation. I reiterated that what differentiates us from the humanities is that our value-free inquiries focus on establishing cause and effect and that the observations we make to prove or disprove these relationships are free from normative biases. Further, our status as an objective social science is achieved by relying on well-documented methods of observation and data collection that are as reliable, valid, and precise as possible. I firmly believed, and I taught my students to accept, that producing this type of social science data, with few further precautions beyond protecting human subjects from potential harm, was in and of itself a social good. I argued that the knowledge we produced provided people with the data that they needed to make informed and reasoned political decisions. And while I still relay much of this narrative to my students, I do so with far more concern for the attention to the role of human agency in constructing a more just, egalitarian world, and to the normative implications of all social science research endeavors. I want my students to understand that both what we as social scientists choose to study and how we choose to conduct our research have normative implications far beyond what most mainstream, positivist political scientists ever fully acknowledge.

Why My Approach Changed My own struggle with this approach emerged when a series of lived experiences, combined with sex-based differences in my research findings and in my students’ learning patterns, heightened my commitment to applied research with normative underpinnings. I wanted to do work that would make things better, and in particular that would make things better for women, both in the academic world as professors and students and in the political world as elected officials, activists, and voters. The desire to re-orient my career began when lived experience taught me hard lessons about sex-based disparities in academia. At a previous institution, my colleagues’ reactions to my pre-tenure pregnancy, for

19

EMBEDDING FEMINIST PEDAGOGY …

229

example, was a mixture of shock, disbelief, and downright horror. Given their reaction, I felt that I dare not risk their wrath by asking my chair for maternity leave, so I arranged to teach quarter classes and fulfilled all of my regularly assigned teaching hours after recovering from an emergency C-section, while still visiting my premature newborn in the neonatal intensive care unit, and while still struggling with postpartum depression. When I discussed stopping the tenure clock with my chair—a university policy that was supposed to be implemented upon request without question—I was told that I still needed to provide medical evidence to document that my pregnancy and delivery had been especially difficult compared to other women’s, thus posing a hardship to my ability to focus on work. Further, I was told that if I took that extra year, colleagues on the department, college, and university retention, tenure, and promotion committees would not treat it as a replacement for lost time, but would expect an additional year of research productivity—an additional 2–3 articles, an additional grant, and so on. Finally, even my publications and past research agenda, once deemed perfectly acceptable, were hyper-scrutinized and questioned, an experience common enough among mothers in academia that it has a label, the “maternal wall.” It occurs because the stereotypes of good scholars, who are expected to have a monastic dedication to the life of the mind, directly competes with those of good mothers, who are expected to be solely dedicated to the wellbeing of their children. People still believe it is impossible to do both. Hence having a child triggers implicit bias and heightened scrutiny, as mothers must defend their publications over and over again to prove that they are rigorous scholars with a solid academic record (Williams 2004). Part of the reason I was attracted to my next university, a teaching institution with a 3–3 load, was that I hoped to find a more family friendly and supportive environment. Imagine my dismay when my lived experience as a woman academic, listening to stories from my peers and my students, reinforced over and over again that regardless of institutional type, higher education’s preferred way to deal with egregious incidents of sexual harassment, even when these incidents affect undergraduate students, was to sweep them under the rug. At a teaching institution, I had more time to pay attention to teaching—but patterns in my pedagogy research and assessment data, along with confidences shared by students—led me to the conclusion that chilly campus climates for women at institutions across the country were still helping to kill their political ambition (Strachan et al 2019, 157–159). Further, I learned inductively

230

J. C. STRACHAN

and on the ground what large-N political science research is now beginning to recognize, which is that the type of peer-to-peer discussions most commonly encouraged in political science classrooms not only alienates women students from considering careers in public service, but from considering going on to graduate school in political science (Hersh and Krupnikov 2020; Poloni-Staudinger and Strachan 2020; Strachan 2017). The culmination of these experiences made me ready to question everything about the status quo in political science, from the way we recruit good students and promote colleagues, to the way we conduct our research. When I began my first term as director of women & gender studies, I was dissatisfied enough with my career that I was receptive to new approaches that would bring normative meaning to my academic teaching, research, and service.

Positivist Political Science is Normative My approach to many things about my career, but also to research methods, was transformed after my teaching, research, and service interests expanded to include gender and politics and after a four-year term as director of my university’s women and gender studies program. Both of these experiences led me to critical theory. As a result, I began to question positivists’ claims of objective, value-free social science—and to appreciate epistemology grounded in the social construction of reality. As I delved into this work, I learned that when Critical Theory is capitalized, it refers to the Frankfurt School—a group of like-minded scholars in 1930s Germany who developed and promoted this approach, along with their students. Specifically, scholars associated with the Frankfurt School believed that their work should identify and help to overcome all of the circumstances that limit human freedom. These circumstances, they argued, were the result of ideologies that justified economic and social oppression. They believed social scientists were responsible not only for identifying these oppressive social constructions (i.e. institutions, practices, beliefs, and norms), but also responsible for recommending strategies to change them. I learned that when critical theory is not capitalized, it can refer to the numerous philosophical traditions—neoMarxism, critical race theory, feminism, queer theory, post-colonialism, transgender theory, and so on—that do not have specific links to a scholar affiliated with the Frankfurt School, but that adopt a similar, overtly

19

EMBEDDING FEMINIST PEDAGOGY …

231

normative approach and that emerged to address the full array of reasons why people are systematically oppressed. I did not have time to provide my students with expertise in any particular genre or subgenre of critical theory. After all, I was still my department’s resident political behavioralist tasked with teaching research methods. Instead, I spend time at the beginning of the semester comparing how the underpinning assumption of critical theory, that much of the world we experience is socially constructed, challenges positivism’s easy embrace of objectivity. At the undergraduate level, I might assign a short encyclopedia entry to facilitate this conversation, but at the graduate level I expect my students to struggle through excerpts from sociologists Peter Berger and Thomas Luckmann’s (1966) seminal book The Social Construction of Reality, A Treatise on the Sociology of Knowledge. One particularly useful section explains how all human civilizations rely on the process of reification to sustain the institutional structures and cultural norms that humans themselves have created, but now experience as inevitable. Reification, the authors point out, “is the apprehension of the products of human activity as if they were something else other than human products—such as facts of nature, results of cosmic laws, or manifestations of the divine will” (Berger and Luckmann 1966, 36). Reification is essential if societies are to function smoothly. Yet when people forget their own “authorship of the natural world,” they also forget that they can change features of the world that were once purposefully designed to subordinate and oppress them (Berger and Luckmann 1966, 36). Hence long-ago inequalities in access to social status, power, and resources solidify over time, as they are reified and transformed into unquestioned features of the natural world. Moreover, most people’s collective, unquestioning participation in the world “the way it is” contributes to reality maintenance, further suppressing anyone else’s ability to see cracks in this façade, or to imagine the world “the way it could be” instead. As Berger and Luckmann (1966, 61) succinctly summarize this point: “Even while apprehending the world in reified terms, [people] continue to produce it. That is [people are] capable paradoxically of producing a reality that denies [them].” The zinger for students occurs when Berger and Luckmann take their argument to its logical conclusion with regard to academia. As hard as we may try, social scientists are always contributing to the social construction of some version of reality. Social scientists, therefore, can never be objective. Even when social scientists adhere to the tenets of positivism,

232

J. C. STRACHAN

our carefully collected empirical data is far more likely to draw attention to the way a given society is currently organized than it is to encourage criticism of the way it’s institutions and norms produce injustice. Hence Berger and Luckmann (1966, 208) conclude their work by warning us that empirical social science, especially when it is no longer anchored by critical theory’s normative agenda, “is endemically in danger of reifying social phenomenon.” For, “even if it begins by modestly assigning to its constructs merely heuristic stats, it all too frequently ends by confusing its own conceptualization with the laws of the universe” (Berger and Luckmann 1966, 208). In short, social science grounded solely in positivism at best draws our attention to the status quo, and at worst normalizes its injustices. Without critical theory, social science research might help us to make informed and well-reasoned decisions about how to succeed in the world the way it currently exists—but it suppresses imagination about how to create a different world where injustice can be overcome. My students understand this point in the abstract, but they are often still skeptical that those who produce knowledge in society—especially mainstream social scientists who authoritatively convey their findings as unbiased truths about the world—really play a powerful role in reification and reality maintenance. To help them recognize this influence within our own discipline, I move on to a series of short readings and reflective assignments designed to make political scientists’ role in these processes more transparent. The first readings explore the origins of political science, as a “home-grown” American discipline, shaped by its emergence during the Progressive Era in the late 1800s (Gunnell 2005, 2006). The academics who emerged from other disciplines to found political science overtly stated their normative agenda, which was to establish a well-regarded applied science whose research findings would ideally be used to improve democratic governance. The entire endeavor is imbued in normative assumptions about the unquestioned superiority of western-style democracy, in general, and of the United States’ version of a constitutional republic, in particular. These normative assumptions extend not only to a preferred regime type, but also to preferred activity level—as our discipline has assumed from the very beginning that “good governance” involved passing and implementing policies to effectively resolve issues of public concern. Political scientists look to a democratic government, and not to charismatic leaders, civil society, or the market, to solve collective action dilemmas. These assumptions about regime type and the role of government are grounded far more in the work of historians and

19

EMBEDDING FEMINIST PEDAGOGY …

233

political theorists who shaped American political thought, scholars such as Montesquieu, John Locke, and Alexis de Tocqueville, than they are in contemporary political science research. I ask my students to consider, without normative questions asked by those we now eschew as part of “the humanities,” would political science even exist? I ask my students whether they are comfortable with our discipline’s unspoken normative embrace of an energetic western-style democracy—a question that triggers the best discussions if libertarian, post-colonialist, or international students with different perspectives feel safe enough in my classroom to share their honest reactions. And, even if students agree with our discipline’s unspoken normative agenda, I ask them to think about why the discipline goes to such lengths to position political scientists as objective and whether our lack of transparency, purposeful or not, is problematic. To further reinforce that students should be aware that empirical social science not only describes and reflects the broader political world, but also influences it, I share examples of social scientists who went on to lament the real-world impact of their own or others work. These often include discussions of the following: ● Federalism scholar Harry N. Scheiber (1993, 67–74) criticized Morton Grodzins’ and Daniel Elazar’s claims of historical cooperative federalism as inaccurate with an “enormous potential” to foster the further centralization of power in the national government; ● In his preface to The Imperial Presidency, Arthur M. Schlesinger (1973, ix) laments the state of the American presidency, and concedes that “historians and political scientists, this writer among them, contributed to the rise of the presidential mystique;” ● Studies have shown that economics students taught that people are naturally self-interested become more motivated by self-interest and less interested in civic acts like voting than students in other majors (Ridley 1996, 260); ● Some have claimed that Richard E. Neustadt (1990, xvi–xvii) shares some responsibility for the Nixon Administration’s enthusiastic embrace of executive privilege. The purpose of this discussion is to help students fully realize that those who produce knowledge wield power even if they do not intend to do so. Hence social science research should be undertaken not only

234

J. C. STRACHAN

with positivism’s concern for scientific rigor and precision in mind, but also with critical theorists’ concern for the effect even the most accurate empirical claims might have on the world around us.

Positivist Political Science Research Protocol Reifies Inequity I revisit critical theorists’ concerns for mainstream positive research practices when I teach students about Institutional Review Boards, the Belmont report, and federally approved guidelines for preventing harm to human subjects. Here, my focus shifts to the way oppressive hierarchical cultural norms and institutional structures that critical theorists oppose are reified not only by the topics we choose to study, but also in the day-to-day way we conduct our research. Pending the level of the class, I typically assign either a short encyclopedia entry summarizing the goals of feminist epistemology or a longer article or book excerpts detailing research protocols adopted by post-colonialist and transnational feminist scholars (Falcon 2016; Fernandes 2013; Smith 1999). While still relying on empirical observations—typically gathered through qualitative interviews, field research, and participant observation—transnational feminists’ embrace of critical theory heightens their awareness that the legacy of social constructions passed down to us throughout history results in systemic inequalities, both within and across national boundaries. These systemic inequalities inevitably affect social science scholarship, as those with enough resources and status to collect data and create new knowledge typically have far more privilege and power than those they study. While this inequity exists between USfocused behavioralists and the members of the electorate they typically study via survey research, it is perhaps easiest for most US students to recognize when US scholars study political phenomena that affect indigenous women or other marginalized groups around the globe. Further, transnational feminist scholars have been incredibly explicit in their efforts undermine the imperialist features of mainstream positivist research. They point out that mainstream positivist researchers feel entitled to extract information from subjects and to use it to advance their own research agenda—which may or may not be of interest to those expected to share relevant information. Hence transnational feminists have worked hard to decolonize traditional positivist research protocol, purposefully

19

EMBEDDING FEMINIST PEDAGOGY …

235

eschewing practices that leverage mainstream scholars’ intersecting privileged identities (of geopolitical power, race, or gender, and so on). They proactively replace extractive research practices with those that prioritize reciprocity and the collective construction of knowledge instead (Falcon 2016; Fernandes 2013; Smith 1999). One simple recommended change, for example, is to re-frame participation in research as labor and the information provided as intellectual property—and to compensate people appropriately for their time and insights. Transnational feminists also recommend building an on-going relationship with a community of interest before collecting data, to facilitate more meaningful reciprocal practices. These might include using expertise and training to undertake helpful tasks (e.g. translating documents, applying for grants, facilitating meetings, and so on) at the community’s request, or even asking local activists to help shape the project’s research agenda and questions so that findings will address their immediate concerns. Transnational feminist scholars also suggest relying on an interactive interview structure featuring a dialogue of questions and answers, an interactive consent process with repeated check-ins, and a preliminary transcript that participants may review and modify, or even revoke, prior to data analysis (Falcon 2016; Fernandes 2013; Smith 1999). I use these readings to facilitate a discussion of whether standard human subject protections required by IRBs are adequate, or whether we should hold ourselves to a higher ethical standard. Should our goal be to do no overt harm to those who consent to participate in our research? Or should we try, as much as possible, to upend the hierarchical relationship between those who study and those who are studied? Students are most apt to agree that these types of precautions are essential when US scholars rely on qualitative research methods that bring them into close contact with minoritized groups around the globe. When pressed, they often agree similar care should be taken when conducting qualitative research with members of minoritized groups in the United States. To encourage them to consider the potential consequences of positivism’s hierarchical, extractive research model, even when methods of data collection minimize contact between researchers and subjects, I often share my own personal reaction to research on the effects of motherhood on women academics when I read Mary Ann Mason, Nicholas H. Wolfinger, and Marc Goulden’s (2013) book Do Babies Matter? Gender and Family in the Ivory Tower. The book relies primarily on survey

236

J. C. STRACHAN

research, but also on secondary data analysis and qualitative interviews to outline the ways implicit and explicit bias against mothers (but not fathers), along with institutionalized sexism, damage women’s academic careers, especially in science-oriented fields like political science, if they choose to have children. I share stories of how the patterns the authors describe mirror my own troubling early-career experiences, which almost led me to leave academia altogether before I decided to transition from a research-intensive university to a teaching university instead. When I read the book, I felt so “seen” that I actually wondered whether I was represented as a data point in the secondary analysis. I had a strong emotional reaction to the book, first because I realized so many other women shared my experiences, and second because I realized that few of our colleagues in the decades since the 1960s had cared enough to try to change our profession. I felt diminished and recall thinking that my lived experiences were nothing more than a data point to them and that everything I had experienced had been reduced to a statistical artifact. Finally, though, I recall feeling relieved—because the authors’ used their findings to argue for specific reforms that I agreed would be helpful. How might I have felt, I ask my students, if I had disagreed with Mason and her colleagues’ (2013) recommendations? What if I had been angry instead of relieved at the way experiences like mine were interpreted? What obligation, if any, do researchers have to think about how survey respondents, or those whose experiences are summarized in secondary data analysis, might respond to their work? Should we be concerned that the US public’s current animosity toward scientific expertise might be a reaction to the extractive nature of most human subjects research? Further, even if it is not possible to cultivate a research community for every large-N research design, are there creative ways scholars can soften our discipline’s tendency to extract data from subjects without extending reciprocity? Possibilities for discussion include writing more expansive literature reviews that include work by critical and qualitative scholars, seeking co-authors with established relationships to communities of interest, relying on community input to develop research questions and to frame findings, or increased reliance on focus groups and pilot studies as research projects are being developed.

19

EMBEDDING FEMINIST PEDAGOGY …

237

Revisiting Normative Concerns Throughout the Semester Foregrounding these concerns allows me to use critical theory as a touchstone throughout the semester. As students learn to conduct research, they often become enamored with their newfound ability to create new knowledge and need to be reminded to consider the normative implications of their work. But it also lays the foundation I need to purposefully revisit critical theorists’ concern over extractive research practices as my class learns about the full array of qualitative and quantitative research designs, from interviews and participant observation to surveys and experiments. My goal when adopting this approach is not to encourage my students to reject empiricism altogether. Rather, I hope to convey that social science research should be undertaken not only with careful concern for scientific rigor, but also with concern for our collective ability to construct a more just and equitable world.

References Berger, P.L., and T. Luckmann. 1966. The Social Construction of Reality, A Treatise in the Sociology of Knowledge. Garden City, NY: Doubleday. Hersh, E., and Y. Krupnikov. October 9, 2020. “Another political gender gap emerges.” The Hill. Retrieved from: https://thehill.com/opinion/campaign/ 520277-another-political-gender-gap-emerges. Falcon, S.M. 2016. Transnational Feminism as a Paradigm for Decolonizing the Practice of Research, Identifying Feminist Principles and Methodology Criteria for US-Based Scholars. Frontiers 27(1): 174–194. Fernandes, L. 2013. Transnational Feminism in the United States: Knowledge, Ethics, Power. New York: New York University Press. Gunnell, J.G. 2005. Political Science on the Cusp: Rediscovering a Discipline’s Past. American Political Science Review 99(4): 597–609. Gunnell, J.G. 2006. The Founding of the American Political Science Association. American Political Science Review 100(4): 479–486. Mason, M.A., N.H. Wolfinger, and M. Goulden. 2013. Do Babies Matter? Gender and Family in the Ivory Tower. New Brunswick, NJ: Rutgers University Press. Neustadt, R.E. 1990. Presidential Power and the Modern Presidents. New York: The Free Press. Poloni-Staudinger, L., and J.C. Strachan. 2020. Keynote: Democracy is More Important Than a P-Value: Embracing Political Science’s Civic Mission through Intersectional Engaged Learning. PS: Political Science & Politics 53(3): 569–574.

238

J. C. STRACHAN

Ridley, M. 1996. The Origins of Virtue: Human Instincts and the Evolution of Cooperation, 1996. New York: Penguin Books. Scheiber, H. N. 1993. The Condition of American Federalism: An Historian’s view. In American Intergovernmental Relations, 2nd ed., Laurence J. O’Toole, Jr, ed. 67–74. Washington, DC: CQ Press. Schlesinger, A. 1973. The Imperial Presidency. Boston: Houghton Mifflin. Smith, L.T. 1999. Decolonizing Methodologies: Research and Indigenous Peoples, 1999. London: Zed Books. Strachan, J.C. 2017. Deliberative pedagogy’s feminist potential: Teaching our students to cultivate a more inclusive public sphere. In Teaching civic engagement: From student to active citizen, 2nd edition, Elizabeth Bennion, Elizabeth, Alison McCartney, and Dick Simpson, eds. Washington, D.C.: American Political Science Association. Strachan, J.C., L. Poloni-Staudinger, S. Jenkins, and C.D. Ortbals. 2019. Why Don’t Women Rule the Word? Understanding Women’s Civic and Political Choices. Washington, DC: SAGE-CQ. Williams, J. 2004. Hitting the Maternal Wall. Academe. 90, 6 (Nov–Dec): 16–20.

CHAPTER 20

Black Lady Classroom Nadia E. Brown, Jasmine Jackson, Aayana Ingram, India Lenear, and Ariel D. Smith

Introduction We are a Black Lady Classroom. At a super-majority White university, we are absolutely our own version of academic #BlackGirlMagic. In the spring of 2020, a graduate level Research Methods in African American Studies course was taught by Professor Nadia E. Brown at Purdue University and consisted of 7 students, all Black women. Like Brown, a majority of the students in the class were political scientists, although students from the Humanities and STEM were also enrolled in

N. E. Brown (B) Government Department and Women’s and Gender Studies Program, Georgetown University, Washington, DC, USA e-mail: [email protected] J. Jackson · A. Ingram Department of Political Science, Purdue Univeristy, West Lafayette, IN, USA e-mail: [email protected] A. Ingram e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_20

239

240

N. E. BROWN ET AL.

the course. As such, the course heavily drew from social science methods and modes of inquiry. Brown designed the course to provide students with an overview of qualitative methodological tools such as interviewing, ethnography, and content analysis. The course centered on student-driven outcomes. The students were challenged to advance their own research project throughout the semester as we workshopped our materials during class time. However, the innovative aspect of this class was its sistah-scholar circle model. We drew from our shared experiences and social location as Black women. We are a small group of interdisciplinary Black women. The model is built off of other academic Sister Circles (Harley & the Black Women and Work Collective, 2002) of the 1970s and early 1980s when Black women first entered White academic spaces in tangible numbers. Indeed, at our university, the department of political science enjoyed the largest number of Black women students in its history during Brown’s tenure as the chair of the recruitment and retention committee. Therefore, our sistah-scholar circle is not a foregone relic of academia when Black women were largely seen as tokens in their disciplinary department. Instead, we are a building upon the Sister Circles of the late-twentieth century where Black women met regularly to engage in mutual support and to survive a hostile space. Within the academy, we are an intergenerational sistah-scholar circle that continues the legacy of our Black feminist foremothers. In this essay, we present our personal accounts of how our Black Lady Classroom became a sistah-scholar circle that advanced methodological training. The methodological training was advanced by teaching students how to review articles to provide analysis on social methods in conversation with studies in the humanities and STEM. The sistah-scholar circle provided a safe space for us to workshop our research projects. In this essay, we will share our experiences as part of the sistah-scholar circle and

I. Lenear Department of Political Science, Rutgers University, New Brunswick, NJ, USA e-mail: [email protected] A. D. Smith American Studies Program, Purdue University, West Lafayette, IN, USA e-mail: [email protected]

20

BLACK LADY CLASSROOM

241

how we leveraged this classroom space to advance our individual research projects.

The Origins of the Black Lady Classroom Dr. Brown was enthusiastic about creating a Black Lady Classroom ethos because her own methodological training as a graduate student did not prioritize cultural sensitivity. She left that classroom determined to prioritize students’ research goals and equip them with the tools to ask and answer academic questions. However, she did not know how to do this. Centering student-driven goals and outcomes is a laudable approach that often is not detailed in pedagogical materials. As such, she spent the majority of her teaching career trying to institute this philosophy in her classrooms and on her syllabi. In one painful batch of student evaluations in 2011, students shared with Dr. Brown’s department chair that “she doesn’t know what she’s doing. She’s looking for students to do the teaching.” That hurt. It was partly true. Dr. Brown wanted students to devote a considerable amount of time workshopping their papers during class time. Indeed, she found that portion of her own graduate methods training the most rewarding. But Dr. Brown vastly underestimated student insecurities that prohibit them from talking openly about challenges in their own work and who are often defensive when others critique their work. Adding to the tenuous situation are raced/gendered biases and stereotypes that prohibit students and faculty from truly communicating in a candid and supportive fashion. The Black Lady Classroom and the sistah-scholar circle became an avenue for Dr. Brown to try a pedagogical approach that prioritized student research goals and outcomes. Without the barriers of white supremacy, anti-Black racism, sexism, and patriarchy, Dr. Brown was able to create a community of graduate student scholars who could learn from and with one another to advance their individual research projects. Unlike her teaching evaluations in 2011, the evaluations from this course indicate that her approach was successful. Our course focused on improving the students’ research design and consequently, their research writing. The Black Lady Classroom was a methods course that required students to submit a research design for either their dissertation (for advanced students) or a research paper (for the first-year students).

242

N. E. BROWN ET AL.

The Importance of the Black Lady Classroom As members of the Black Lady Classroom, including Dr. Brown, we found ourselves shocked that Purdue University was the place that we found ourselves in a classroom of all Black women. Although half of the group graduated from Historically Black Colleges and Universities (HBCUs), including Dr. Brown, none of us had ever been in a room where there were only Black women present. The racial demographics of our classrooms at Purdue are predominately white, conversely, at HBCUs the vast majority of the students are Black and there are many more Black professors than at predominately white institutions. However, once at Purdue University, we quickly went from the majority to the minority in the classroom. Being a part of the Black lady classroom was essential. Our encounters in other classrooms and academic spaces at Purdue made us feel inadequate, unworthy, and both invisible and hyper-visible due to being the only Black women in them. India, a first-year political science Ph.D. student, details her experiences in other classrooms as being disheartening. She states that “I have sat in class with white men and women professors who have laughed at my critiques and questioned my ability to critically think. I was also described as a distraction because the discourse I brought into classes was considered tangential.” Jasmine, a fourth-year political science Ph.D. student, describes her relationship with the academy as one that was tumultuous from the beginning. Her earliest memory of racism, discrimination, and prejudice took place in a classroom. Despite our various experiences, we knew our Black lady classroom would be memorable. Before taking the time to discuss some of the shared experiences of the students in this classroom, we will discuss the positive influence of Dr. Brown as an instructor. While it is rare that a classroom consists of Black women, it is also rare to have a Black woman instructor while in graduate school.

The Pedagogy and Dynamics of Our Black Lady Classroom At the beginning of the semester, Dr. Brown fostered a welcoming classroom environment by explaining to us that the class and the time spent in the classroom would be dedicated to us furthering our independent research projects. Her pedagogy integrates participatory lectures with active learning. Active learning requires being able to actively engage with

20

BLACK LADY CLASSROOM

243

our required weekly readings as opposed to merely reading to regurgitate quotes in class. By understanding these readings, we were better able to incorporate either the theoretical framework or a critique of the methodology in our own research design. To help us strengthen our synthesizing skills, Dr. Brown pushed us to read with the end goal of being able to answer the following questions: 1) What is the main question the author is asking? 2) What motivates the question/why is the author asking the question? 3) What is the primary expectation, hypothesis, or claim the author seeks to test? 4) On what theory or logic is the expectation based? 5) What methods of investigation has the author used to evaluate the claim? 6) What are the primary findings? Were the expectations met? 7) How does this research advance, or contribute to, our knowledge of this topic? The syllabus contained social science articles and books, the majority of which were political science texts but also included readings within the humanities. The diversity of texts and writing styles pushed each of us to become confident in our abilities to analyze a text that may not be in our direct or cognate field. As graduate students, we evaluated our identities as emerging scholars by whether or not we understood the material. At times, the readings were challenging. At other times, they were confusing. Nevertheless, she immediately tasked us to show each other grace and kindness and stated that our class was a communal space that we would foster together. We would come together to help our fellow sistah-scholars brainstorm ideas, draft, and review papers, and just be good accountable human beings to each other. She stated that the classroom was a space for us to ask questions that we may be embarrassed to ask in other spaces and that we challenge one another to make sure we were researching what we truly wanted to know, not what we thought sounded good or would be accepted by our respective disciplines. Theoretically, this lofty goal was admirable. But with some work, we were also able to put it into practice. Early on in the semester, there was a point where we read an article that had some ambiguous points. Ariel, a fourth-year Ph.D. student in American studies, was apprehensive

244

N. E. BROWN ET AL.

to share that she didn’t understand an article we had to read in class until Jasmine said that she also didn’t understand the point of the article. Dr. Brown made sure that we took that moment of vulnerability as a teaching moment to explain that our lack of understanding is not always because we missed something; it could be a valid critique of the author’s inability to provide clarity. She pushed us to not shut down because we don’t understand but instead to investigate further into why we do not understand. Students have been socialized to quietly internalize their inability to comprehend academic materials rather than investigate these challenges as part of the learning experience. In our Black Lady Classroom, students were encouraged to openly discuss what they did not understand and why. This supportive sistah-circle then became a place for us to break down the readings as a collective to ensure that all students grasped the key points of the material. Likewise, we used this approach when discussing our own research projects. Dr. Brown dedicated the second half of the class to us workshopping our research projects with the group. We would dissect one another’s research questions, theoretical frameworks, methodological and data choices, and the ethical implications of our work, focusing on helping our fellow sistahs improve their work. Our class would be an environment where new and innovative ideas on various topics such as race relations, higher education, and food trucks could exist within the same space, be understood, and encouraged. Not only did Dr. Brown create a space for us to discuss our research topics, but she also opened up our class as a platform to other Black women scholars at Purdue. Several class periods were dedicated to research presentations by these scholars. Following these presentations, it was then our turn to apply what we were taught by Dr. Brown by asking thought-provoking questions about their research. Very quickly in the semester, she challenged our imposter syndromes by pushing us to embrace moments of confusion as opportunities for inquiry and deeper analysis. In the following sections, we delve further into this moment of awakening from imposter syndrome and how this candid classroom dialogue led us to be able solidify our research topics and form a sistah-scholar network that has continued beyond the course and despite the coronavirus pandemic.

20

BLACK LADY CLASSROOM

245

Combating Imposter Syndrome Aayana, a first-year graduate student in political science, stated that “as a graduate student, the pressures of being ‘perfect’ can lead to anxiety and a lack of confidence in one’s abilities.” This is a feeling that everyone within our class could relate to; many refer to this lack of confidence as imposter syndrome. Imposter syndrome is referred to as “a phenomenon in which a person feels intellectually inferior despite demonstrated competence” (Clance and Imes 1978; Cowie et. al 2018). However, overcoming imposter syndrome is difficult, particularly as a Black woman in a graduate program at a predominately white institution. India, who is in the same cohort as Aayana, shared how hostile classroom environments can exacerbate imposter syndrome. Nevertheless, the Black Lady Classroom was a place where India did not have these same barriers. At first, it was uncomfortable and unnatural for her to share her inputs and critiques with her classmates. The Black Lady Classroom was a place where we would not be mocked and scoffed at, instead, we had the freedom to bring our full selves into this academic space without fear of race/gender-based discrimination or stereotypes. When any of us offered our critiques of the literature, we were heard and seen. No one had to justify why their critique of literature or when articles merely used race as a variable but did not systemically include a racialized analysis of the findings. We were empowered to call out the outsized role of White supremacy in social science. Furthermore, we were pushed to critique how systemic biases are present in research and have adverse implications on societal outcomes. In this space, we felt the opportunity to breathe and let down our guards. We did not feel as if we needed to forcibly demonstrate why some social science relies on flawed assumptions about historically marginalized groups, nor did we have to pretend as if we had the answers to fix these systemic biases. Instead, we had the opportunity to simply be students who were learning how to move past being consumers of knowledge to learn how to be producers of knowledge that is reflective of the communities from which we come. Because we all grapple with imposter syndrome, we discussed our various approaches to cope with this feeling. We drew on each other’s unique strengths and it helped each of us become better scholars. The Black Lady Classroom gave us a place to fine-tune our academic skills and increase our confidence. This classroom was a space where we were able to enjoy not having to be the Black or woman representative in a

246

N. E. BROWN ET AL.

majority white male space. Ariel describes her experience in the class as uncommon from what she encountered in other classrooms. She enjoyed not having to represent her entire race in class. Here, students and the professor did not look to her alone to explain or speak for “the Black experience.” In the Black Lady Classroom, she did not have to mentally prepare herself for a question or comment from a professor or peer that would try to “uppercut my humanity as a Black woman,” instead, she found support and mentorship.

Gaining Clarity for Research Foci Even after diminishing the effect of imposter syndrome, we still needed to solidify our methodologies for our research projects. To achieve this, we first had to clarify our research topic. We quickly found that there was often a disconnect to what we were vocalizing as our research projects and how we were choosing to measure, conceptualize, and operationalize said topic. Dr. Brown pushed us to understand the importance of our research questions. These questions would guide our research, and in turn, directly affect the choices we made methodologically. For example, Jasmine had become accustomed to other faculty and peers asking about the relevance of why she studied racial gaps in political knowledge. As a way of insulating herself from critiques, she chose to use a commonly used methodology as a way of making her research more “acceptable.” This strategy is what Dr. Brown refers to as the cake mix approach, “add Blacks and stir.” This means that she follows exactly what other scholars have done in the past and only changes one aspect—here, the race of the population that one is interested in analyzing. This approach incorrectly negates the unique historical, economic, cultural, and political factors that shape group identity. These important factors help to explain why and how Blacks behave differently in American politics. With the help of her classmates, Jasmine was able to revise her research to include methods that are better suited to contextualize how Black Americans think about politics.

The Long-Lasting Effect of the Black Lady Classroom The Black Lady Classroom served as a place of healing, growth, and support. It was a community that we were able to actively mold and shape, allowing it to become a community that we needed to thrive at

20

BLACK LADY CLASSROOM

247

our institution. In this space, we were able to recharge before reentering a space where racism and discrimination is a part of our daily experience in and out of the classroom. We will forever cherish the lessons and environment cultivated from this experience. In our classroom, we were equipped with the tools necessary to articulate and defend our research in unsupportive spaces. These tactics will be useful for us going forward and can be incorporated into our own teaching. The Black lady Classroom also taught us much about mentorship and community. We learned that mentorship is not only hierarchical, but it can also be peer to peer. Dr. Brown served as a mentor to all of us, but one of the most important things about this sistah-circle network was that she taught us how to advise and be there for one another inside and outside of the Black Lady Classroom. The first-year students were able to see Ariel and Jasmine achieve milestones in their Ph.D. program. Both scholars passed their comprehensive exams as members of our Black Lady Classroom. The space was able to serve as a support system through this tumultuous time and give advice to the younger graduate students on how to prepare for these moments. The relationships we formed during our time together transcended the academic semester and will continue to grow. In sum, the Black Lady Classroom was a unique pedagogical approach to learning social science methods. For others who may want to emulate this approach, we recommend that you follow three tactics that helped to make our sistah-scholar circle a success. First, we prioritized a sense of community among the group. While everyone does not have to be friends, we fostered a friendly and collegial relationship which made it possible for this teaching style to be effective. The professor and students respected one another as equal contributors within the knowledge production process. Next, we also acknowledged differences among ourselves as Black women. We did not assume that we all had similar experiences or perspectives based on our shared raced/gendered identities. We embraced these differences by talking through them and through learning more about how to support each other. Our goals were to actively listen to one another first, ask probing questions, and then respectfully engage in scholarly dialogue. And lastly, we recognized that our unique classroom may never occur again. We were cognizant of this possibility and as such embraced each class period—even those that were forced to move to Zoom as part of the social distancing measures put in place because of COVID-19—that we would honor the specialness of our time together.

248

N. E. BROWN ET AL.

This meant that we often took pictures (and later posted them to our own social media pages), shared words of affirmation with our fellow sistahscholars, looked for ways to remain in scholarly contact after the course ended, and connected socially outside of the classroom. We cherished our time together and were deliberate in our actions to show this appreciation for one another. We are forever indebted to the Black Lady classroom!

References Clance, P.R., and S.A Imes. 1978. The Imposter Phenomenon in High Achieving Women: Dynamics and Therapeutic Intervention. Psychotherapy: Theory, Research & Practice 15(3): 241–247. Cowie, M.E., et al. 2018. Perfectionism and Academic Difficulties in Graduate Students: Testing Incremental Prediction and Gender Moderation. Personality and Individual Differences 123: 223–228. Harley, S. 2002. The Black Women and Work Collective, eds. Sister Circle: Black Women and Work.

CHAPTER 21

How the Research Design Can Be a Structure, a Process, and a Product for Learning Political Science Erik Cleven

Introduction I started my college career as a physics major. My goal was to become an astronomer and I read everything I could get my hands on about astronomy. As a freshman in college I knew quite a bit about the universe, and everything I knew was also connected to how it was known. There was no book on astronomy that just reported the distance from the earth to the sun or the size or age of the universe. Most books also explained how those things were measured. Understanding how to make a measurement like that was just as fascinating as knowing what the measurement actually was. Other subjects were often taught to teach content, not how the content was known.

E. Cleven (B) Department of Politics, Saint Anselm College, Manchester, NH, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_21

249

250

E. CLEVEN

When I did go to graduate school, it was in political science. As a graduate student at a large R1 research university the first class I taught was international relations. There were no incentives for me to do anything more than lecture and provide a midterm and a final exam. There was course content, but whether that content had been learned was evaluated through a series of multiple-choice questions on an exam. This meant that students mostly had to be able to define terms or at most to identify the main point an author had made in a reading. I taught research methods the same way. It is easy enough to present a series of concepts that relate to research methods that students need to know and memorize. The terms reliability, validity, hypothesis, dispersion, levels of measurement can all be explained in lectures and then tested as multiple-choice questions. Certainly, I illustrated concepts using my own research hoping to make the concepts concrete. But the experience was a bit like attending a series of lectures on skiing where you learn about what the snow plow turn is, but have no idea how to do it and never attempt it.1 The result was that many students got exam questions right, but soon forgot what they had “learned.” They certainly would not be able to apply any of the concepts they were learning and it did not change how they viewed scientific knowledge. People encounter data and research in many parts of their lives, both as citizens and professionals. They also encounter numerous assertions that purport to be based on data and analysis. At some level we all need to be able to evaluate such claims and the evidence they are based on. Students who are interested in graduate school obviously also need basic research skills. The lecture style of teaching treats knowledge like information. The difference between knowledge and information, and the social production of knowledge, only becomes visible when students participate in their own research. Only by doing research do students really understand what concepts like validity and reliability really mean, and only by doing research can it possibly matter for students to know these concepts. So, what do students need to learn and how can we best teach them? I argue that whether students are going to attend a graduate program or not some of the key research skills they need are the ability to: ● ● ● ●

read and review scholarly literature think theoretically develop good research questions conceptualize key variables and operationalize them

21

HOW THE RESEARCH DESIGN CAN BE A STRUCTURE …

251

● conduct basic bivariate data analysis ● interpret their research results and communicate these to an audience of their peers Learning these skills also means that students can critically evaluate the research of others that they encounter professionally or in the media, whether it is pandemic statistics, unemployment data, or presidential polling. I will argue that it is possible for students to learn these skills and that they are best learned when students do their own research. In the following I review how I have developed my teaching of research methods and lessons that can be drawn from this experience, whether you are teaching at a small liberal arts college or a large research university.

Teaching How, not just What The type of teaching described above was not really in line with my pedagogical values. In my teaching statement I wrote that, “My teaching is based on making it possible for students to participate in their own learning and to connect course topics to their own experience in concrete ways.” At Saint Anselm College I not only could, but had to, think differently about teaching methods. One reason for this is that the research methods course is not just a free-standing course, but a preparation for writing a senior thesis. Students take research methods and develop a research proposal as their final project. This research proposal is then the starting point for the research they do in Senior Seminar, which they carry out and write up as a thesis. Every major in our department is required to write a thesis consisting of original research that is at least 30 pages long and that builds on a minimum of 25 peer reviewed sources. I could no longer just teach a series of concepts about methods. I needed to teach students how to do their own research, and as I did so, I realized how little they had really understood about certain concepts. For example, I had a lecture about the importance of survey question wording. But when I divided the class into small groups to have them write survey questions that they could ask other students at the college, they would write things like, “Question about favorability.” As I went around the classroom, I would then ask students how exactly they were going to ask the question. Usually they would say something like “Well, of course we will ask whether or not the respondent has a favorable opinion of the president.” Then I would ask members of the group whether they

252

E. CLEVEN

were all going to be asking the question in the same way, and only then would some lights come on, and students would realize that this was what I had been talking about in the lecture on question wording. In another case, students set up a question as a “feeling thermometer,” i.e. a question where answers were continuous. When they tried to analyze the responses using a cross tabulation, it suddenly became clear that that only works when the level of measurement is nominal or ordinal. I had told them this in lecture, but they needed to grapple with actually doing it before they began to understand it. I began to design more exercises that would help students go beyond learning definitions to grasping the significance of concepts and ideas. My methods class evolved to consist of not just a series of participatory but independent exercises like the ones described above. Instead, the exercises and activities were structured by the research design. The research design is both a product and a process and my teaching builds on three premises that form the basis for the research design and senior thesis project at the college. In the following I describe these premises and then show how the research design makes it possible to build on them so that undergraduate students can conduct their own research. In the end, students not only learn research skills, they also develop a more mature understanding of how we know things, and the provisional and social nature of the production of knowledge.

Three Premises My teaching of research methods is based on three premises that ground my pedagogical approach. The first of these is a belief in the capacity of each student to do research. It is rare that a student does not complete a thesis, even if the quality of the work varies. Some theses are of publishable quality (see, e.g. Bisbee 2019; Krusemark and Cleven 2014; Nolan 2019), while others manage to test a simple hypothesis. But each student has the ability to wonder about something, and if you give them the opportunity to start talking about what really interests them about politics, there is usually something. It might be a topic that piqued their interest in an introductory course they took; students after all take research methods their junior year, so they have had two years of course work to survey the discipline. Conducting research on their topic of interest would be impossible if students were simply told to do research. Instead, the process is broken

21

HOW THE RESEARCH DESIGN CAN BE A STRUCTURE …

253

down into stages consisting of shorter assignments that are given in parallel with classroom instruction, where students are taught how to carry out each step of the research process. The starting point, however, is each student’s interest or passion. The only limitations on the choice of topic are that it must be connected to politics or policy, and it must be researchable for an undergraduate in the time available. The second premise is that students learn best when they grapple with a problem themselves. As we saw above, if a student learns about measurement validity in lecture, even when they answer an exam question correctly, they do not necessarily understand the concept or have the ability to apply what they have learned. Only when they have designed a survey question with particular answer options, and realized how this connects to a chosen method of bivariate analysis, does it really become clear to them. What I have found is that as students work on their projects, they gradually begin to think like political scientists. I have often overheard students talking outside faculty offices asking each other questions like, “What is your dependent variable?” or “How are you measuring that?” They do not just know concepts; they use them like social scientists. The third premise is that any pedagogical process needs to start where the students are. A journey needs to start from where you are when you begin. This may seem trivial and obvious, but it is important. By the time students take research methods they have taken a number of political science courses. They have learned content, but they have usually not learned much about how we know what we know (but see Clark, Golder and Golder 2018). Students also come to college without many basic research skills. When I started teaching I thought I could assume that students knew what a card catalogue was and how to use it, but some of them have barely set foot in a library when they start research methods, and many do not know the difference between an article, a journal, and a book. I now include a full class session in my introductory international relations and comparative politics courses on how to use the library webpage and what peer reviewed sources are so that students will be better prepared. Many of my colleagues are doing the same and are adding shorter research assignments to their courses.

254

E. CLEVEN

Structuring the Learning Process The thing that gives structure to learning about research is the research design. It is both a process and a product. The research design has six parts. Table 21.1 shows these parts and how they are connected to the research process, class sessions, and the research skills that students learn. Throughout this process I meet regularly with each student to give them Table 21.1 Research Design as Structure, Process, and Product for Learning Product

Process

Introduction Literature review

General topic Search for and read literature

Research question and hypothesis

Develop a good research question Develop strategy for answering research question Conceptualization

Data and Conceptualization

Operationalization Measuring key concepts Finding or collecting data

Method and analysis Finding out how to analyze your data

Lectures and labs

Research skills

What is peer reviewed research? How to find library resources? What is a literature review and Why do we write them? What is a good research question? What is a hypothesis?

Reviewing scholarly literature

What is a good theory? What is conceptualization? Measurement, Levels of measurement Discovering existing data sets Designing surveys Semi-structured interviews Descriptive statistics

Conceptualizing variables Measuring variables

Thinking theoretically

Analyzing data

Bivariate data analysis Content analysis Conclusion

Interpreting results Relating results to discussion of scholarship in the literature review

Interpreting results Presenting findings

21

HOW THE RESEARCH DESIGN CAN BE A STRUCTURE …

255

one-on-one mentoring that is individualized to their specific research project. These conversations are often as much brainstorming with the student as they are advice giving. As I show below, asking questions of the students in a one-on-one conversation can also help clarify for the student what they know and what they still need to figure out. The senior thesis they write the following year has much the same structure, but, of course, with a results and discussion section added. The literature review is the overview of the scholarly conversation on the topic. Without doing research on the existing literature it is not possible for students to know whether a research question is a good one or how their research will fit into the larger scholarship on the topic. In order to write a literature review, students need to know how to locate peer reviewed articles and books, they need to be able to read them, and finally, they have to present an interpretation of the state of scholarship on their topic. Is there debate? Consensus? To write a literature review, students of course need a topic. That is not the same as a research question. The research question comes later. Students first hand in a topic, then they have to find eight peer reviewed sources on this topic. Next, students write an annotated bibliography. This is kept simple. The student has to write three sentences, no more and no less, on each article or book they are annotating. The first sentence must describe what the book is about. The second describes the author’s main argument. Finally, the third sentence explains why the article or book is important to the student’s interest in the topic. The annotated bibliography serves as a basis for writing the literature review. Note that although the final thesis consists of 25 peer reviewed sources, they get there a little at a time. When I first started teaching methods, I assigned students to write a research question very early in the semester. Most of them changed their question after writing their literature reviews, because they learned that either the question had already been answered or it wasn’t a very good question. I now ask students to hand in a research question and hypothesis only after they have written their literature review. Once students have some idea of the scholarship on their topic and have formulated a good research question and at least one hypothesis, the next step is developing a research design that will help them answer their question. There are basically three choices here: an experimental research design, an observational research design, or a subcategory of the latter, the comparative case study. In order to understand these research

256

E. CLEVEN

designs, we talk about causality in class. The fundamental problem of causal inference is that we cannot know the outcome of a process and its counterfactual (King, Keohane, and Verba 1994). The three types of research design are basically work arounds to this problem. The next step is clearly conceptualizing variables and figuring out how to operationalize or measure them. This can be a challenging step for students because they don’t see their concepts as problematic. After all, they know what they mean by them, or so they think. But they do not understand that they need to find or collect data to measure the concepts. This is one place where one-on-one conversations with students are helpful. I can ask, “When you say democracy, what exactly do you mean?” or “What about democracy is important to your theory?” And as the student and I discuss the concept, we can start to crystallize what the student is really talking about and the student can get practice expressing her ideas and concepts more clearly. Finally, students have to figure out how they will analyze their data. This is challenging because there is limited time during the semester to cover so many topics, and bivariate analysis comes towards the end of the semester. Students cannot determine what their method of analysis will be until they know what the possible choices are. In addition, I have to acknowledge that my research methods course gives priority to empirical research of a particular kind. More class sessions are spent learning the logic of hypothesis testing and basic statistical analysis than research in political theory. Nonetheless, students have written theses on topics ranging from Machiavelli’s concept of citizenship to critical security studies. Students have interviewed town officials about alternative energy sources and conducted content analysis of English language Pakistani newspapers. This is possible through one-on-one mentoring and by connecting students with faculty members whose research is closest to their interest. Students learn best by doing and learning about research methods is very effective when students work their way through a whole research project. This helps them connect the various parts of the research process and see how they fit together. From the outside, it may seem like this process would only work at a college with small class sizes. But these ideas can also be applied by instructors at large universities. The key thing is to give students assignments where ideas have to be applied and implemented. Only then do they understand what they don’t know or understand, and this is the first step to real learning.

21

HOW THE RESEARCH DESIGN CAN BE A STRUCTURE …

257

If you have too many students to give one-on-one mentoring, much can nonetheless be achieved through in-class group work or assignments where students have to conduct one aspect of the research process. Research methods is a class where it can also be advantageous to “flip the classroom,” letting students watch recorded lectures at home, allowing for class time to be spent on activities where students have to engage one aspect of the research process. For example, rather than come to class for a lecture on literature reviews, students can watch a short presentation on the topic at home and instead use class time to work in groups on a “reverse outline” exercise. Students are given a literature review from a journal article and in groups need to make an outline of it. This helps students understand the function of a literature review in an article and how one is structured. Another way to teach research skills is to present students with a puzzle and assign them to develop a theory that explains the puzzle. If they think about the main independent or intervening variables, and the causal mechanisms at work, they are learning to think theoretically for themselves. My colleague at Saint Anselm College, Jennifer Lucas, has developed another activity that can be done in or out of class. Students get a data set and code book (e.g. the American National Election Study) and pick two variables. They brainstorm a hypothesis that would connect the two and ascertain the level of measurement of the variables. They then choose a bivariate test of their hypothesis (e.g. comparison of means or cross tabulation) and run the test. With step by step hand-outs that explain the SPSS process, students can learn another aspect of the research process. As the above examples show, there are many things students learn along the way as they do research. They are learning basic statistics and the logic of hypothesis testing. They are learning how to find and access information and data and they are learning the basics of critical thinking and, perhaps for the first time, learning to think theoretically. All of this contributes to numeracy and information literacy, but in addition I argue that it also contributes to two major epistemological goals. First, it teaches students that everything we know about politics is connected to how we know it. Second, they understand the provisional and social nature of the production of knowledge.

258

E. CLEVEN

How We Know What We Know Students are used to searching for information online or reading about a topic in a textbook and taking the information as “truth.” They don’t always differentiate between information and what I will call knowledge. One result of the internet revolution is that many young people do not understand why they should read a book about something when they can just look up the information online. This attitude is of course based on the assumption that facts or propositions about the world are given. It ignores the possibility that the reason for reading a book might not be to get facts or information, but rather to follow a complex argument, to get a particular perspective on a problem, or to see evidence presented to test propositions or theories. You might be able to look up the level of voter turnout in the last presidential election online, but how do we know why some people are more likely to vote than others? Is there more than one way of measuring voter turnout? These questions are not answered by internet searches, but by clear conceptualization, carefully designed research strategies, theoretical arguments, and sophisticated analysis. In fact, they are also never answered by one scholar. Instead, as you discover when writing a literature review, the answers to these questions become clear only when many scholars have applied their efforts to answering them. When students come to college, they probably think that to know something is to be certain. Knowledge is indisputable. As you learn more, and especially as you learn how we know things, you also come to realize that scientific claims are usually not accompanied by certainty, but rather with an estimate of our level of confidence. As you learn more, you also understand that the questions that are worth answering may have more than one answer and that the answers are contested. When students have had to work hard on their own research, when their knowledge has been hard won through critical research, they know what this means and they understand the provisional nature of scientific knowledge. At the end of the process, students have learned something about what it means to produce new knowledge. They realize that they are part of a community of scholars that are working to understand complex political processes and that what we know is a result of such cumulative efforts. Our understanding can change as new ideas are developed, as new research is done, or as the world itself changes. To understand this is to be a mature scholar.

21

HOW THE RESEARCH DESIGN CAN BE A STRUCTURE …

259

Interestingly, there are students who do not always do well in lecture or content-based courses that do very well on their own research. Doing their own research engages students in ways that lecture-based courses do not. One of the most satisfying teaching experiences I have had is seeing a student who was a B−/C+student getting a high pass on his senior thesis. This was because the student was genuinely engaged in the research he was doing and really embraced the process of answering an original question that he cared about. The research design helped the student take his interest and passion and structure it into original research.

Note 1. In fact, there are ski schools that consist of YouTube videos.

References Bisbee, J.K. 2019. Hitler’s Hand in the 21st Century: How Holocaust Collaboration in the 1940s Shapes Modern Anti-Semitism. Pi Sigma Alpha Undergraduate Journal of Politics, XIX, 2 (Fall): 42–61. Clark, W.R., M. Golder, and S.Nadenichek Golder. 2018. Principles of Comparative Politics. CQ Press, 3rd edition. King, G., R. Keohane, and S. Verba. 1994. Designing Social Inquiry: Scientific Inference in Qualitative Research. Princeton, N.J.: Princeton University Press. Krusemark, A., and E. Cleven. 2014. Sex and Drugs (But Not Rock and Roll): The Variation in HIV- Related Restrictions on the Entry, Stay, and Residence of Seropositive Foreigners in the Middle East and North Africa. Risk, Hazards & Crisis in Public Policy 5 (3): 279–294. Nolan, A. 2019. Confident Women, Compassionate Leaders: The Effects of Single-Sex Education on Female Empowerment in Uganda. Pi Sigma Alpha Undergraduate Journal of Politics,XIX, 1 (Spring): 57–71.

CHAPTER 22

The Success of Research Methods at the Department Level Neil Chaturvedi and Mario Guerrero

The academy often subjects scholars to a version of this story: an academic department divided by methodological differences. Usually, the details are only dared to be whispered in the hallways, but these fissures run deep in every aspect of how the department conducts business. Over time, these differences lead to factions. Perhaps one group of faculty takes ownership of the graduate program and implements a methods sequence in their vision of the discipline. Maybe a recently tenured professor has the ear of the department’s lead graduate student, who might incidentally encourage a group of graduate students to apply for department funding to a summer training program. And then there is the faction

N. Chaturvedi (B) · M. Guerrero Department of Political Science, Cal Poly Pomona, Pomona, CA, USA e-mail: [email protected] M. Guerrero e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_22

261

262

N. CHATURVEDI AND M. GUERRERO

of faculty with extensive knowledge of university procedures, who regularly undercut systematic changes to the curriculum by subverting the department’s efforts at the college level. These divisions, too often, serve as the norm in many academic departments. While the details might change, environments like these socialize scholars to replicate these patterns throughout their careers. It is not uncommon for PhD students to navigate mentoring relationships by tiptoeing around methodological divisions and turf wars. While on the market, candidates often are told to avoid making strong statements regarding methodological preferences in order to not upset the department’s balance. Yet despite this almost inevitable division, departments must reconcile differences like these to effectively educate their students and prepare them in the social sciences. The political science department at California State Polytechnic University, Pomona serves approximately 450 undergraduate majors and 50 students in a Masters of Public Administration program. Cal Poly Pomona is a public, Hispanic-serving institution that serves over 29,000 undergraduates in Southern California. Over the last ten years, we have hired ten new tenure-track faculty while undergoing substantial growth by doubling our majors. This growth was a driving force for the department to assess our pedagogies through the teaching and research we do. This chapter explores how our department has explicitly used assessment conversations, heavily based on research methods and pedagogy, to structure our core curriculum. All academic departments are responsible for fostering the development of its faculty as instructors, but this collaborative, shared effort has culminated in a core political science curriculum that is connected through research methodology. In particular, our department’s experience highlights how a department has updated its methodology curriculum to better reflect recent developments in political science while mindfully incorporating the pedagogies of each faculty member in the department.

Assessing a Political Science Program Our department holds the view that the skills imparted in a methodology curriculum have lasting effects on students, both undergraduate and graduate. However, departments do their students a disservice if they avoid viewing their methodology curriculum as constantly evolving. Departments too often relegate ownership of the methodology curriculum to a

22

THE SUCCESS OF RESEARCH METHODS AT THE DEPARTMENT LEVEL

263

small number of faculty. Instead, a department’s methodology curriculum should be viewed as a fundamental cornerstone to the construction and success of all the other courses contained in a political science department. Indeed, while many departments build their methodology sequence around their department’s strength in either qualitative or quantitative methodology, every faculty member should have a hand in actively cultivating the objectives and topics in the methodology course(s), regardless of the approach. One way we have fostered an environment that includes every faculty member’s input is through collaborative assessment. A program assessment activities should occur each academic year, leading to regular discussions about the knowledge and skills that students learn within a program. Although many departments dread program assessment or avoid assessment altogether, assessing curriculum presents a unique and rewarding opportunity to have conversations about shared pedagogies. Our department has always taken assessment as a serious opportunity to reflect upon the curriculum. However, although many view assessment as a university-instituted, formal requirement at the end of each academic year, our department uses assessment as a starting point to have a conversation about teaching. As our department has evolved and grown in the last decade, we believe that the curriculum should reflect both the students we serve and our respective expertise. In fact, by design, we have learned to bring together our respective pedagogies through a set of five core courses that cut across the subfields: an introductory course to the major, a year-long research methods sequence, and a senior capstone year-long sequence. The connecting thread in these five courses is undergraduate research. Universities have increasingly allowed more and more undergraduates to conduct their own research. At some institutions, undergraduates directly assist with faculty’s research. The number of undergraduate research conferences has substantially grown in the past few decades (Corbett and Rosen 2020). Some might criticize the growth of undergraduate research as less than one percent of the population end up enrolling in post-baccalaureate programs, even as college enrollment continues to grow (Hussar et al. 2020). Outside of the university, some argue that graduates might not have much use for the enterprise of academic research. Still, while undergraduates may not regularly engage with academic scholarship after graduation, the skills necessary to conduct good research are marketable in today’s job market. In addition, research

264

N. CHATURVEDI AND M. GUERRERO

is the single commonality among the different subfields and course offerings in any academic department. As such, we view research methods as the connective tissue, not only in departmental course offerings but more importantly, in a departmental pedagogy.

Research Methods as a Department’s Connective Tissue As elucidated upon in the examples that began this chapter, academics sometimes use methodological differences to make distinct choices about department governance. Shared governance can be threatened by methodological preferences when they lead to deep divisions and factions, sometimes lasting decades. These divisions and factions can have a substantial impact on a department’s curriculum and student body. In reality, departments with faculty who insist upon staking methodological territory are doing the discipline a massive disservice. In practice, most political science departments are populated with faculty with a wide variety of training. The scope of political science methodology sees substantial change as time goes on, although such a trajectory is difficult to track. However, by surveying the variety of methods used in the discipline’s top journals, it is clear that political science methods have grown in scope (Bennet, Barth, and Rutherford 2003). Because of this, recent doctorates are likely to begin a tenuretrack position with a distinct methodological perspective from those who earned their doctorates ten, twenty, or thirty years ago. Indeed, in our own experience, we have found our own methodological approaches to differ vastly from that of our more senior colleagues. Even still, as career incentives change with hiring and promotion, scholars are likely to shift their methodological perspectives as they move forward in their individual careers (Poteete, Janssen, and Ostrom 2010). Thus, broad claims about methodological preferences in the curriculum are short-sighted as the discipline and careers are fluid and ever-evolving. Cal Poly Pomona founded the political science department in 1969. Its early course offerings were sparse and probably were in accordance with the faculty’s expertise who founded the department. However, by the mid-1970s, the department began to include an “Introduction to Political Science” course and a “Research Technology in Political Science” course as part of its core offerings. By the 1990s, we renamed the Research Technology course, “Research Methodology in Political Science.” As is

22

THE SUCCESS OF RESEARCH METHODS AT THE DEPARTMENT LEVEL

265

the case in most political science departments, that research methods course was almost exclusively taught by a single faculty member, the “resident methodologist,” until the 2010s. As part of the department’s core offerings, majors were required to take both the introductory and methodology courses in order to graduate with a political science degree. When the department began hiring tenure-track faculty once again in 2011, the non-subfield-specific major core then consisted of: an introductory political science course, a single research methods course, and the two-semester-long senior capstone. However, these core courses were not necessarily taught in conjunction with one another, primarily because the assigned instructors for each of the courses frequently changed and were frequently taught by adjunct faculty. These core courses also came into being at different periods in the department’s history, thus they were not thought of as a coherent, sequential set. However, as the department’s tenure-track faculty grew in size, assessment discussions also grew, and this fostered a couple of key developments that cemented these courses as a crucial component of the department’s shared pedagogy. This was realized in our department’s use of our senior capstone requirement. In the mid-2000s, the department instituted a senior capstone requirement. The capstone requirement began as a mechanism to assess the department curriculum. The department had no consistent way to directly assess the curriculum, thus a year-long senior project would help to identify the strengths and weaknesses of students who completed the degree program. In our capstone, students opt to complete a senior thesis or internship, but the experience’s main deliverable is a 20–40-page paper in which the department then uses to assess our department objectives. At the end of each academic year, tenure-track faculty share responsibilities in grading the senior capstone papers using a rubric designed to measure outcomes based on the department’s learning objectives. While our students write their capstone papers under the supervision of a single faculty member, the department emphasizes the independent nature of the capstone paper. Still, though only five of our tenure-track faculty actually advise the senior theses, we have the entire department’s tenured and tenure-track faculty review the thesis projects using the aforementioned rubric. That is to say, rather than merely assigning final grades, grading senior capstone papers occurs as a department discussion. These discussions have been illuminating for curriculum purposes: we review our learning outcomes but we have specifically instituted changes in the curriculum as a result of the capstone conversations. In addition, the

266

N. CHATURVEDI AND M. GUERRERO

grading discussions have encouraged faculty to be more thoughtful about our teaching’s effectiveness, not only in our methods courses, but in our respective subfield-specific courses as well. The capstone is meant to be an indicator of the skills that the students have learned during their entire experience as a political science major. While we meant for the senior capstone project to provide our department with useful assessment data, it is useful to acknowledge that departmental discussions focused squarely on pedagogy are difficult, primarily for some of the reasons we have described earlier in the chapter. It is rare to share course assignments with someone else in your department, as many faculty are uncomfortable with having their students assessed by a colleague. Thus, it is difficult to find commonalities in pedagogy if the focus is simply on the courses that you teach. Our pedagogy is informed first by the courses that we teach, but to find common ground with departmental colleagues across subfields is to find the shared threads among what we all teach. Incidentally, our department stumbled upon our shared thread by discovering that our core set of courses should be focused on research methods, especially as our department grew in both faculty and diversity of viewpoints. As the department began hiring new faculty in 2011, the university also increased enrollments across the campus. Because of the increase in the number of students, junior faculty began to advise senior capstone projects. Capstone advising is an unfamiliar task as junior faculty members typically have little to no experience advising undergraduate research. However, since new assistant professors begin their time in an academic department having just completed a doctoral dissertation themselves, junior faculty are nicely situated to help undergraduates troubleshoot research design. In fact, in our advising discussions, the junior faculty were especially helpful, as they were able to highlight concerns about the soundness of the methodology found in the capstone papers. Although department faculty understood the application of research methodology to be a higher-level learning outcome, there was often a wide variety of knowledge, understanding, and mastery of research design in the capstone papers. Even while the capstone papers demonstrated a wide variety of success in devising their own research methodology, faculty were hesitant to expand on the major’s ability to conduct their own research because undergraduates will rarely utilize academic scholarship in their careers and lives after graduation. However, while many political science graduates will rarely conduct their own research outside the

22

THE SUCCESS OF RESEARCH METHODS AT THE DEPARTMENT LEVEL

267

university, undergraduate research is known to increase student engagement and academic performance (Fechheimer, Webber, and Kleiber 2011). Undergraduate research might result in higher satisfaction and higher GPAs, but generally, engaging students in undergraduate research should make teaching in a political science program easier. Whereas a political science curriculum can cut across anywhere from four to eight subfields, it can be difficult for students to ascertain the programmatic connections in a political science degree, across subfields. In our department, more than half of our learning outcomes are connected to research design and methodology. Yet, like most political science programs, the undergraduate research design course was relegated to the tutelage of only one faculty member and others would rarely interact with the course whatsoever. For decades, the course had been taught as a quantitative analysis class, where students would learn about inferential statistics. Qualitative analysis was relegated to a brief discussion for a week in the term. Given the behavioral revolution and shift to quantitative analysis across subfields, political science students do indeed have a distinct need to understand quantitative statistics. However, in practice, the research methods course carried very little context for how students understood their other courses or even completed their senior capstone project. After about five years of hiring new faculty and growing enrollments, the department decided to make two distinct changes to the curriculum core. First, the major core sequence that we earlier identified would be taught by emphasizing research and scholarship. This meant that in the introductory course, students would begin to learn and develop knowledge about the practice of academic scholarship. For freshmen students especially, learning about research as a systematic attempt to build expertise is an important component of understanding the major coursework that follows over the next four years. In the research methodology course, students would begin to apply the basic tenets of research design to analyze both quantitative and qualitative data. Ideally, one of the goals is to perform quantitative data analysis at the end of the research methodology course. And lastly, in the capstone courses, students would be expected to develop and create their own arguments, dealing with the principles and substance of the courses they completed in the preceding four years. Although one of the major goals of the research methodology course was for students to be able to analyze quantitative data, we realized that

268

N. CHATURVEDI AND M. GUERRERO

Table 22.1 How the Methods Sequence Evolved Over Time at CPP Number of Methods Courses

Course Titles

1970s–1990s

1

1990s–2011

2

2011–2016

3

2016–present

4

1. Research Technology in Political Science 1. Introduction to Political Science 2. Research Methodology in Political Science 1. Introduction to Political Science 2. Research Methodology in Political Science 3. Senior Thesis* 1. Introduction to Political Science 2. Introduction to Research Methods 3. Advanced Research Methods 4. Senior Thesis*

* Senior Thesis is used to assess our department’s pedagogy, including methodology

by exclusively focusing on quantitative analysis, the department was shortchanging our students. In addition, much of the scholarship that faculty assigned in upper-division courses utilized more than just quantitative analysis. In fact, a sizable contingent of the faculty primarily relied on qualitative analysis in their own scholarship. Thus, as a second change to the curriculum core, we expanded the research methods course to a year-long, two-semester sequence. Given that we were asking students to make their own arguments in a senior capstone paper, it was important for students to receive a year’s worth of methodology training. Table 22.1 summarizes the changes our department made to the methods sequence over time.

Research Methods as a year-long Endeavor Given that the department had made a distinct choice to emphasize research methods in the core and expand the research methodology courses in a two-semester sequence, the department committed to base the entire curriculum in research methodology. As such, we created or reframed the program into three distinct courses. This is summarized in Table 22.2.

Course Name

Introduction to Political Science

Introduction to Research Methods

Course

Political Science 1010

Political Science 2051

Table 22.2 Summary of New Methods Sequence

-Research Design -Qualitative Methods

-Introduction to the subfields -Literature reviews

Topics

(continued)

1. Understand the subfields in political science 2. Comprehend how knowledge is built through research 3. Learn why we use literature reviews and how we write them 1. Understand the science behind political science 2. Use to scientific method 3. Write a research design 4. Form hypotheses 5. Explain the basic forms of political science methodology

Learning Outcomes

22 THE SUCCESS OF RESEARCH METHODS AT THE DEPARTMENT LEVEL

269

Course Name

Advanced Research Methods

Course

Political Science 3055

Table 22.2 (continued)

-Quantitative Methods -Measures of variance to Multiple regression

Topics

1. Understand how inquiry is conducted in the social sciences 2. Explain how quantitative analysis in a scholarly paper supports, or fails to support, the author’s conclusions. 3. Apply statistical concepts to a systematic inquiry regarding factors that influence public opinion and/or policy outcomes 4. Able to design, conduct and critique survey research methods, univariate and bivariate statistics, experimental and quasi experimental research 5. Able to design, conduct and critique strengths and weakness of qualitative methods such as focus groups, content analysis, and field observation methods

Learning Outcomes

270 N. CHATURVEDI AND M. GUERRERO

22

THE SUCCESS OF RESEARCH METHODS AT THE DEPARTMENT LEVEL

271

The first, Political Science 1011, serves as a freshmen-level course, introduction to political science, where students are introduced to the subfields in the discipline (as well as public administration), and are instructed on the basic building blocks of social science, including the literature review. The year-long methods sequence begins with the second course, Political Science 2051, a research design and qualitative methods course. In this course, students learn about how to formulate and develop research questions, conduct a full literature review on their own, develop hypotheses (if appropriate) and recognize the appropriate method of analysis. The course then introduces various methods of qualitative analysis, including but not limited to ethnographies, content analysis, case studies, and interviews. The benefit of adding this course was twofold: it provided students with a more comprehensive toolkit for their capstone projects but it also allowed students to better understand the methodology utilized in scholarship found in the upper-division courses. Finally, our third course, Political Science 3055 covers advanced research methods and focuses primarily on quantitative political science methods. The course covers basic statistical methods beginning with central tendency and dispersion and levels of measurement to hypothesis testing using chi-square, t-tests, and linear regressions. The idea behind this year-long methods sequence was to help students build skills to help them write their capstone senior thesis paper and have the ability to engage and understand social science research. In introducing a research design aspect into the year-long sequence, we have thus far been able to increase the students’ ability to understand the purpose of literature reviews and how they motivate hypotheses, and better prepare them for the various methodologies available to them to test their hypotheses, be it qualitative or quantitative. Using a broadly accepted form assessment, our department has facilitated a collaborative effort in developing a research methods sequence that has helped our students develop applicable skills that are realized through their own research projects. As departments seek to establish or evaluate and change their research methods programs, we recognize that the process by which one starts this discussion with their department is a difficult and often uncomfortable conversation. Based on our experiences, we have a few tips on how to get the conversation started. First, we believe that the conversation is moot without a set of department learning outcomes. Our department’s learning outcomes provided us an objective

272

N. CHATURVEDI AND M. GUERRERO

benchmark to measure the success of our methods sequence. Similarly, we believe in the importance of a collaborative assessment assignment. Our senior thesis class is an independent study, guided by a faculty member in which the projects serve as a measure of the department’s effectiveness in achieving the learning outcomes. It is important for the assignment to be a collaborative one, as this assures buy-in from all faculty and the inclusion of diverse viewpoints. This assessment should lead to a discussion of the curriculum. We also strongly believe that the department should simultaneously be assessing the field of political science as well, as we mentioned above that methods are in a constant state of evolution. While this process is still a difficult one in terms of self-evaluation and reflection of long-held values in regards to political science methodology, having a collaborative effort from the establishment of learning outcomes to assessment helps foster an environment that is open to changes to the curriculum. Although departments might have factions nestled in methodological turf wars, this chapter presents a vision of a political science department that is brought together by research methodology.

References Bennett, A., A. Barth, and K.R Rutherford. 2003. Do We Preach What We Practice? A Survey of Methods in Political Science Journals and Curricula. PS: Political Science and Politics 36(3): 373–378. Corbett, P., and J.R. Rosen. 2020. Supporting Twenty-First-Century Students with an Across-the-Curriculum Approach to Undergraduate Research. Scholarship and Practice of Undergraduate Research 3(3): 4–13. Hussar, B., J. Zhang, S. Hein, K. Wang,. A. Roberts, J. Cui, M. Smith, F. Bullock Mann, A. Barmer, and R. Dilig. 2020. The Condition of Education 2020. Fechheimer, M., K. Webber, and P.B. Kleiber. 2011. How Well Do Undergraduate Research Programs Promote Engagement and Success of Students? CBE—Life Sciences Education 10(2): 156–163. Poteete, A.R., M.A. Janssen, and E. Ostrom. 2010. Working Together: Collective Action, the Commons, and Multiple Methods in Practice. Princeton University Press.

PART III

Teaching Research Methods

CHAPTER 23

Traveling Along with an Accidental Academic: Doing and Teaching Research John A. Garcia

Becoming an Accidental Academic Midway through my academic career, I was asked to address Ford Fellow recipients at their Annual Conference to reflect on the underlying “drivers” that had influenced my professional choices. Largely, several seemingly random or “accidental” decisions led to the completion of a 1971 Ph.D. in Political Science. My interests lay with critical inquiries, intellectual and pragmatic relevance, pursuing curiosity in both creative and systematic manners, and building upon and adding to knowledge bases. After completing my doctorate, I embarked on a post-doctorate at Northwestern University’s Center for Urban Affairs. Rather than refining my dissertation, I decided to explore Chicago’s marginalized communities. I took this time to become involved analytically and pragmatically on matters of marginalized communities, empowerment, and critical

J. A. Garcia (B) University of Michigan, Ann Arbor, MI, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_23

275

276

J. A. GARCIA

issues of people’s daily lives through exploration of community-based free health clinics. These clinics were operated by politicized community organizations that were trying to empower their respective communities and address real needs. Although much of my formal research training consisted of empirically-based research techniques, my probes took the form of participant/observer. My conviction in support for change, empowerment, and pragmatic relevance, as well as my curiosity, propelled me to develop a way to explore, understand, and hopefully impact the free clinic “movement.” Using a participant/observer approach, intermeshed with transparency and reciprocity, I worked at several clinics weekly in a “basic lab setup.” The different clinics allowed me firsthand observation of each organization, its membership, patients and their families, and their network with health-related organizations and city government. Utilizing my analytical skills, these relationships allowed me to compare and communicate observations, and to serve as a liaison, between the clinics. My career in teaching and research was greatly impacted by my interests in health, local community organization, and empowerment. My development as a researcher had a direct bearing on teaching political science courses. I received no formal training in graduate school in pedagogy, teaching approaches, course preparation, or organization. In many respects, appreciation and acknowledgment of my former teachers and their effectiveness informed my approach to teaching. I soon realized I was motivated to learn more about empirical research and methods, so I took graduate statistics courses in the Departments of Psychology and Statistics. I participated in the ICPSR Summer Program, taking a sampling workshop and advanced casual analysis. As I was developing more research skills, I was also accumulating valuable experience from a variety of instructors; their persona, communication techniques, and the underlying theorems, mathematics, and logics of research techniques. Since 1978, I have been involved in at least six major national/regional studies with other researchers and graduate assistants from different social science disciplines The studies included identifying and contacting underrepresented populations (communities of color, women, sub-groups such as widows, etc.), adding to our knowledge base beyond replication, developing linguistic versions of survey instruments, use of focus groups, collaboration with a research team and outside expertise, evaluating modes of data collection, and assessing the extant research literature. Often these were less-charted “waters,” so my systematic training was

23

TRAVELING ALONG WITH AN ACCIDENTAL …

277

enhanced by experiences, insights, intuitiveness, and creativity. I believe teaching research methodology reflects this combination of training and what the researcher personally brings to systematic inquiry. An example: active participation in devising sampling frames with under-represented populations that would include race/ethnicity, national origin, nativity, language use and multi-generational households. Constructing survey sampling frames requires consideration of geographic distribution, household and family “types,” migration patterns, concentrated and dispersed community members, sample weights, and design effects. While technical in nature, the consideration of both “demographics” and representativeness of screened and sampled respondents can result in richer and more useful data. Another example lies with exploration of social identity and its linkages to behaviors and other important attitudes. Contextually, some of these late 70’s and 80’s projects represented attempts to investigate the nature and “centrality” of how persons see and categorize themselves. The dimensions of situational factors and dynamic character of identity were part of research designs with the idea of capturing “urban widows of color,” minority-elected officials at all levels of government, and such focused topics as equality, discrimination, mental health and help-seeking, being an immigrant, political engagements, and multiple dimensions of race and ethnicity.

Intersectionality of Researcher and Teacher My foundational beliefs, values, and research experiences were reinforced during undergraduate and graduate courses and by faculty advisers by integrating the theories and results of systematic research to understand, question, and extend beyond current knowledge. I use the term intersectionality, the confluence of experiences, perspectives, critical thinking, and knowledge, to inspire students to become active researchers. Overall, the development of my pedagogical approach to teaching coursework and research gradually “emerged.” At the University of Arizona, I occasionally taught the undergraduate scope and methods course and I incorporated many of those research dimensions into my courses on Minority Politics, especially Latino Politics, and Public Policy. Later in my career, pedagogy was more at the forefront as I taught our graduate seminar, “Teaching Political Science.” I read more systematically about learning theories, learning styles, communication styles, visualization, nature of content and topics, teaching approaches, and clarity in

278

J. A. GARCIA

course goals and objectives. This added knowledge base assisted me in approaching courses exclusively devoted to the research process. In the 1980s, my courses on the research process took a dramatic turn. A project by the Social Science and Research Council (SSRC) and the Inter-University Program for Latino Research (IUPLR) created a fourweek research seminar “targeting” scholars who were graduate students and current faculty (of all ranks), from any of the social sciences, who expressed interest and involvement with Latinos. I was approached to organize this seminar with nearly carte blanche discretion as to its design. The general goal was to “equip” research scholars with the skills and understanding to conduct Latino research. After considerable thought, I agreed to assume the role of instructor. Given the diversity of scholars from a range of disciplines, I developed a three-principle approach with an interdisciplinary scope, incorporating materials, techniques, and purview of particular social science dimensions into the seminar structure. The first principle of de-partitioning discipline boundaries required participants to communicate across discipline borders. As I had a penchant to read a variety of materials that fell outside Political Science, I was motivated to explore and understand a diverse range of topics, groups, etc. This pursuing and discovery of knowledge entailed the expanded use of tools from different social science disciplines to discover alternative explanations for topics or issues. I have sustained this orientation toward interdisciplinary research throughout my career. I place greater value in trans-disciplinary research, conducted by investigators from different disciplines AND working jointly to create new conceptual, theoretical, methodological, and translational innovations, integrating and moving beyond discipline-specific approaches, than in multidisciplinary research, which usually involves individuals from different disciplines staying in their “lanes” while sharing their disciplinary knowledge. The overall research experience may not be integrative, but interdisciplinary research modes should integrate information, data, techniques, tools, perspectives, concepts, and/or theories from multiple disciplines advancing fundamental understanding or solving of problems whose solutions are beyond the scope of a single discipline. The second principle was the important linkage between theories and concepts with analytical tools and interpretation. If research is about developing ideas and theoretical frames to explore and comprehend social phenomena, then the research process commands linking theories and their components to operationalization, measurement,

23

TRAVELING ALONG WITH AN ACCIDENTAL …

279

data, and appropriate analytical techniques with informed interpretation. Thus, a key organizing “theme” of the seminar was identifying critical concepts central to Latino research. These included social identity, group consciousness, assimilation and acculturation, family and culture, power and engagement, mobilization, community, race and ethnicity, disparity, gender/sexuality, equality/equity, nativity/foreigner, and social justice. Examination of these pertinent concepts would advance our understanding of Latino communities and their relationships to social structures, social groups, and public policies. Using the conventional theories and concepts of research design, matters of delineating relationships, measures and indicators, specification of models, analytical approaches and techniques, and interpretation should “produce” contributing research. My third principle consisted of the collaborative and interactive elements of conducting research, not necessarily involving working jointly on a research project, but rather providing feedback and suggestions to fellow researchers engaged in same/similar questions. Since the research enterprise is about knowledge-building, curiosity, understanding, and applications, this principle of sharing research with others who were not “vested” in one’s content area afforded the opportunity to expand perspectives, analytical breadth, and to consider unexpected questions. I placed emphasis in ongoing communication among fellow scholars regarding their basic ideas, theoretical frames, data choices, and so forth because the ability to communicate one’s work can provide clarity in what one is doing.

Principles and Seminar Practices My preparation for the series of SSRC/IUPLR seminars (1989–1994) followed these three principles, adjusting to the personalities and interests of each participant. The format consisted of daily 2 ½-hour sessions over a four weeks period. Assignments required access to computer labs and assistance with statistical packages. Since the seminar was part of the ICPSR Summer Program for Quantitative Research, I was able to identify teaching assistants from various social science departments at the University of Michigan. Participants, many of whom took several ICPSR offerings, wanted to develop specific analytical skills. Since most participants came from many different departments and disciplines, living away from their “home base” fostered interactions outside the course

280

J. A. GARCIA

setting, creating a camaraderie that contributed to the collaborative and interactive principle. I organized the syllabus around the key concepts discussed earlier and conducted an extensive literature review to develop the seminar bibliography. The readings, assigned daily, reflecting theoretical works as well as testing theories and concepts, were discussed in the context of Latino populations. Initially, each participant felt more comfortable with theoretical literature in their own field, with each discipline having slightly different “professional jargon,” so finding and using common language became part of the learning process. It also helped us thoroughly discuss the meaning and bases of our theoretical concepts. I also included articles on specific analytical techniques. On the first day of the seminar, I asked each participant to talk about their research interests and projects so that everyone had a working knowledge of the range of interests among this group and where some overlap existed. Subsequent discussions involved relating our concepts of Latinos to individual’s research interests. Applicability, challenges, adjustments, and using our concept to carry out analytical research was the goal. The mixture of social science disciplines served to interject perspectives and questions that many participants had not encountered within their disciplinary “silos.” Presentations of specific analytical techniques were introduced; the underlying assumptions about the data, requisite conditions for application of the technique, the statistical/mathematical representation of the technique, and the bases for interpretation. Aside from homework assignments (use of software packages, research design of proposed projects, etc.), each participant was required to submit a research project to be completed for class presentation at the end of the fourth week. Everyone was familiarized with the structure, design, and objectives of the studies and all relevant documentation. I selected 1–2 data sets from which the research hypotheses/questions would be tested and analyzed, uploading them to the course homepage, with the instructor and TA’s serving as resources. Designating existing available data sets afforded participants to assess and understand the study (its organization, skip patterns, set response categories, sample weights, etc.), enabling them to use the data sets for their own research queries. Since working with another’s study usually differs from your own line of inquiry, question format, questions, and response categories, conforming it to your own research investigation involves creativity, generating surrogate (indirect) measures, scales, etc. From an instructor’s perspective,

23

TRAVELING ALONG WITH AN ACCIDENTAL …

281

my familiarity with the selected data sets allowed me to give advice and possible direction for each project, meeting personally with each participant, reviewing and commenting on their research design. During the last week of the seminar, each presented their research project and findings within a conventional conference format. A Q&A session followed each presentation allowing conversation and input from their fellow participants. As the sole instructor of this seminar for six years, it was not uncommon to see participants further develop and refine their analysis for successful journal submission. In addition, the research presentations often resulted in career-long focal areas of research. One example involved two sociologists, previously unknown to each other, who began discussing mutual interests and curiosities about the role of skin color on opportunities and obstacles. They collaborated on a joint research endeavor, going on to commit their respective energies to continuing research of skin color and its consequences. Other researchers, maintaining communication with each other’s research foci after the seminar, resulted in interactive feedback on their writings, ideas and, at times, formal collaborations. Attending the ICPSR Summer Program meant that participants were away from their home institutions and engaging in research and analytical techniques almost exclusively. Several of the participants shared other courses, interacted outside of the course setting, and set up informal discussion groups. These interactions afforded them opportunities to talk about their interests, projects, and personal lives and experiences, creating a long-lasting foundation of collegiality. Over the period I served as the principal instructor (approximately 130 participants took this seminar over five years), the composition of the participants changed a bit. While the range of social sciences continued to be well represented, the mix of faculty and graduate students changed; the ratio of graduate student to current faculty increased to almost a 3:1 ratio. It was important that each person had my contact information; a good part of my office hours involved long conversations with individual participants about their works in progress, ideas, or drafts of working papers. My inter-disciplinary orientation and familiarity with a broad range of literature and concepts enabled me to provide advice, critiques, and additional sources. Although labor-intensive and demanding, my commitment was to engage fully in this seminar; availability and receptivity was a clear dynamic between the participants and myself. As is usually the case, the

282

J. A. GARCIA

exchanges were mutually beneficial as I learned from our conversations and hopefully expanded the scope and perspective of my own research.

Do Research. Version 2 While my direct involvement with Latino research ended in 1994, a “version two” of the previous seminar structure later became part of my teaching research experience. In 2006, the ICPSR Summer Program initiated a more substantively oriented four-week seminar. Over the course of its 50-plus year history, the Summer Program had offered courses encompassing major social science studies or series of studies, organized around such policy areas or major concepts as health disparities, political engagement, social mobility, etc. This new seminar, entitled Methodological Issues in Quantitative Research on Race and Ethnicity (MIQRRE), was taught by Prof. Philip Bowman (Social Psychologist) with three other researchers, including myself, as part of the team. The initial course description involved the examination of research design, measurement, and analytical issues in the quantitative world of race and ethnicity research. The exploration of methodological issues associated with race and ethnicity included reliable and valid measurement of racial and ethnic variables in surveys, sampling strategies for diverse populations, race related measurement errors to data collection, analytical techniques, and trends in research topics and approaches. The seminar bridged inter- and cross- disciplinary perspectives with a policy-relevant focus on race/ethnic disparities. During the first two seminars, my primary responsibilities were covering one week of instruction, concentrating on Latino populations and research trends. Clearly, those participants whose research interests overlapped my own, being away from their home institutions and focused on research training, resulted in many informal, often spontaneous evening sessions and “get-togethers.” The ICPSR participant “portal” outlined the benefits of having some prior quantitative “statistics” and introductory research design courses. Being less explicitly mentioned in the description of Research on Latino Issues resulted in having a few earlier participants whose background was in History and Philosophy of Science. These participants shared a strong interest in Latino issues and were open to discovering new approaches and techniques, adding different perspectives and understandings. The MIQRRE content took the form of an augmented statistics

23

TRAVELING ALONG WITH AN ACCIDENTAL …

283

course with attention to multivariate analysis, with modules on sampling, causal and mathematical models, and experimental and longitudinal analysis. Embedded in the daily schedule was attention to secondary data sources, “hands-on” assignments with data files, data searches, data collection, and methodological strategies. The substantive areas related to race and ethnicity embodied the disparities; political, educational, health, and economic. The four instructors used their disciplinary expertise to cover each of these areas with a policy link to research findings and interpretation. Within two summers, I assumed the role of principal instructor for MIQRRE. While retaining much of my guiding principles and objectives, there were some important considerations I had to incorporate dealing with both content and the composition of the seminar participants. Over a four-year period, a total of over 170 participants took MIQRRE, ranging from 24 to 46 per session. While MIQRRE covered all communities of color with participants coming from all the Social Sciences, I discovered that participants were interested primarily in one minority group. Therefore, the scope of the social sciences was expanded to include public health, public policy, anthropology, and interdisciplinary and racial/ethnic studies. The size of the seminar, especially when it was over 30, posed challenges with my responsiveness to individuals’ research agendas. Toward the latter years, the seminar attracted international researchers from Russia, Ukraine, Europe, and the Middle East. As a result, communication across disciplinary “boundaries,” organizing with different levels of research preparation and experiences and developing a foundational conceptual base to build upon was critical. The ratio of graduate students to Ph.D.s was almost 3:1. This reconstruction of race/ethnicity and empirical issues incorporated the intersection of theories/concepts and analysis. I decided to concentrate on race and ethnicity as the core of the seminar, including the questions of what race is, what ethnicity is, how they are measured and operationalized, what the inter-relatedness of race and ethnicity is, and assessment of the US binary racial system. Additional concepts included assimilation/acculturation, discrimination, social identity, group consciousness, colorism, multi-racialness, racialization, systemic racism, and racialization. Attention was placed on thinking of race as a multidimensional concept, with requisite demands for clarity, specification, reliable and tested measures, and empirical modeling. We discussed the translations of conceptual clarity and parsimony into research-based

284

J. A. GARCIA

hypotheses, reliable and valid measures, and appropriate multivariate techniques. Drawing from multidisciplinary research literature, “required” readings included variations in the statistical techniques. I selected TA’s who were ABDs with extensive methodological training. They served as both resources to me and to the participants. As I became more confident in their methodological skills, they were given responsibility for class “methods” presentations with topics involving hierarchical linear models, mediation-moderation models, experimental design, and times series models. We also spent time on scaling techniques and the use of focus groups for instrument development and concept clarification. As social surveys served as the primary data collection approach, we examined interviewer effects, item validity and reliability, and item construction. The diversity of backgrounds of the participants’ demographics, interests, and disciplines did present a challenge, so I initiated a questionnaire for them prior to the start of the seminar. I sought more detailed information as to their coursework, research interests, background experiences, expectations for this seminar, and papers written. With my decision to devote significant time and attention to exploring race and ethnicity, and with emphasis on foundational bases to pursue race/ethnicity analytically, I found having more knowledge about the participants was very useful. It allowed me to identify connectors to this more general examination of race/ethnicity sensitive to the specific research of the participants. A central focus of this seminar was the important distinctions between race and ethnicity and the distinct differences in the inter-related or overlapping meanings. That is, both concepts share common notions, and are sometimes treated as surrogates, being viewed as separate but related. Additional concepts included social class, nativity, legal status in the U.S., language use, phenotypical traits, and culture. I introduced the contextual and situational factors that might produce variations within minority social groupings. Such discussions were linked to group solidarity, common interests, life experiences, and collective behaviors. The scope of MIQRRE, the characteristics and interests of the participants, the size of the seminar, and operationalization of research goals in understanding race and ethnicity with empirical and methodological issues, affected how I developed this seminar. My guiding principles, research as an inter-relationship between theory/concepts and empiricism, transdisciplinary research, and collaborative interactions, helped me redesign this seminar. I had many one-on-one meetings with participants

23

TRAVELING ALONG WITH AN ACCIDENTAL …

285

about their ongoing projects, providing feedback and additional considerations. My TA’s were valuable resources in evaluating assignments, expanding on statistical techniques, and providing technical assistance on statistical packages. One rewarding benefit from these collaborative interactions was that all the TA’s were able to continue their own research projects successfully, with one succeeding me as an instructor. MIQRRE is still offered as part of the ICPSR summer program.

My Next Research Seminar: 3.0 After turning over the MIQRRE seminar, I moved to the ISR at the University of Michigan with appointments at ICPSR and Center for Political Studies. At ICPSR, I helped develop the Resource Center for Minority Data (RCMD) data archive. My primary mission was to augment the archive with more studies that incorporated minority populations and issues, encouraging researchers to share their data by depositing their studies, increasing knowledge about minority-related research, location of resources, and training workshops. In the case of the latter, we worked with the Robert Wood Johnson Foundation (RWJ), offering training workshops on health disparities for communities of color. We were funded to organize four seminars on health disparities with vulnerable populations. These one-week workshops, operating from 8–5 for five days, meant a different design was required. This deep immersion raised challenges of meaningful coverage of the research topic, integrating research content and analytical techniques and approaches, insuring “hands-on” experience, reinforcing the trans-disciplinary nature of research, and connecting individuals’ research interests with the scope of the seminar. RWJ funding covered fees for the workshop so we instituted an application process for potential participants that included a letter of interest in the research area, their relevant background, curriculum vitae, and, if graduate students, a letter of recommendation from their graduate adviser. Over the four years of this workshop, we averaged about 80 applications. The workshop was held in a data laboratory setting with a capacity for 24 participants, most of whom came from traditional social science disciplines with a health/medical and policy emphasis, including medical researchers, physicians, health economists, and health policy specialists, who were at various stages of their professional careers. Selection was driven by a mixture

286

J. A. GARCIA

of stages of career, disciplines, specific areas of interest in the health disparities, and their focus on particular populations. The design of the workshops took into consideration the expected interactions among the participants, in- and outside, of the all-day seminar. I also exposed the participants to active researchers, both well-established and others with a record of thoughtful and innovative research. Having worked in health behavior and policy areas for a considerable number of years, one consequence was a compendium of research literature to be shared with the participants. Also, my familiarity of the field helped me identify active researchers as instructors. The format was that each instructor would be responsible for a full day, delineating the scope of their presentations and research. My staff and I augmented these presentations with data resources and examining complex data sets. As each instructor identified essential readings for participants, I discussed with them, both formally and informally, their projects, data issues, and analytical approaches. While this made for long days, my commitment to my underlying principles was still operable. Although the structure remained consistent with my research teaching experiences (I knew the necessity of “upfront” preparation such as posting an early “dropbox” containing all presentations, readings, research bibliographies, contact information, data related presentations, and data files), the focus for each year’s seminar changed. While year one concentrated on health disparities among minority groups, subsequent years focused on other vulnerable populations such as immigrants, LGBTQ individuals/groups, and inter-group comparison models. Despite long, intense sessions, the participants’ passion for the human dimensions of health disparities resulted in dinners and informal sessions about individual’s ongoing research projects. Over time, the principle of interactions and collaborations was clear.

Postscript: Research, Teaching, and Pedagogy My career has now spanned over 45 years and I am still teaching five years past my second retirement and actively researching. My approach to teaching research was to link personal curiosities and my desire to understand and explore with immersion into building skills for analytical development. My inter-disciplinary orientations broadened the scope of my knowledge; encountering differing perspectives, and interactions with students and scholars across many social science fields helped me to

23

TRAVELING ALONG WITH AN ACCIDENTAL …

287

develop underlying principles to teach research design and methods in a variety of settings and audiences. In closing, I would like to briefly highlight one more critical element that influences my pedagogical approaches to research and instruction. Demarcating the intricacies of race and ethnicity, including all the ramifications of developing meanings, measures, and relationships to attitudes, behaviors, and institutions, is a complex area and an overwhelming task. A high degree of complexity involves substantial webs of components, interrelations, and promising properties that are not immediately investigated or conceptualized. Exploring race/ethnicity as a complex phenomenon increases the number of constructs that need to incorporate, accounting for innumerable interactions, interfaces, feedback loops, and “the impact of presumed causes on one another” (Gear et. al. 2018; Page 2015). To reduce the challenges and demands of the research enterprise, we tend to simplify our works. Giving my guiding principles, embracing complexity aids a researcher by affecting conceptualization and theory development to identify additional relevant variables and interactions, multi-dimensional measures and specific domains. In a very real way, I am driven to accept the uncertainty and sketchiness of our scientific knowledge of complex topics. Acknowledging the limitations of theory development imposed by the complexities of race/ethnicity requires me to pace complexity in the research enterprise. Personally, embracing complexity has sustained me over some fifty years of performing and teaching research.

References Gear, C., E. Eppel, and J. Kozoil-McClain. 2018. Advancing Complex Theory as a Qualitative Research Method. International Journal of Quantitative Methods 17: 1–10. Page, S. 2015. What Sociologists Should Know about Complexity. Annual Review of Sociology 41: 21–41.

CHAPTER 24

Statistical Skills for the Workplace: A Practical Approach to Teaching Methods with Excel Lisa A. Bryant

As an undergraduate I was required to take two statistics courses, one for my major and one as part of my general education requirements. My first course was an introduction to statistics course taught by the math and statistics department. The class was quite focused on integrals, matrices, and the math behind the statistics. There were few, if any, applied examples. The second course was taught in the political science department. I loved the course. We learned the difference between descriptive and inferential statistics, we learned SPSS, we were encouraged to collect our own data, and we were required to write a research paper using the skills we learned throughout the semester. This difference between the two classes was so stark that I remember thinking one of these courses must surely have been misnamed. In graduate school, I chose methods as one of my subfields and took classes across multiple disciplines–political science, economics, sociology,

L. A. Bryant (B) Department of Political Science, California State University, Fresno, CA, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_24

289

290

L. A. BRYANT

psychology, and education. Again, the approach one took to teaching methods made all the difference to my comprehension and enjoyment of the class. As an advanced graduate student, I taught the graduate stats lab and I taught undergraduate methods, though my mentor shared her syllabi and exercises with me, absolving me of having to invest too much thought into course design. These experiences in graduate school, along with having methods as a subfield, meant I went on the market touting my ability to teach undergraduate and graduate methods.

From Student to Teacher As junior faculty, I was asked to take on an undergraduate methods course for my department at a large, public, regional 4-year institution starting my third year. Looking forward to the course, I started planning by asking our majors what the methods course in the department had included when they took it. Many of them honestly couldn’t tell me. I also heard horror stories about how one professor would tell them statistics were not really necessary for political science and how they simply completed some basic exercises with very little explanation as to how or why they might be used in political science research. In the vein of many junior faculty, I decided that I was going to design a course similar to my first-year graduate experience so that my students would gain a deep understanding of and appreciation for statistics. In hindsight, I may have been a bit overzealous in my approach. The first year I taught our quantitative methods course, I used Statistical Package for Social Sciences (SPSS). Our university did not have Stata available in the computer labs, so SPSS was the only “real” option as I saw it. I had taught undergraduate and graduate methods using SPSS while I was a graduate student, so this would be easy, I thought. The semester went pretty well, but I ran into the typical issues that most people do when they teach with a particular piece of software. Students didn’t have time to finish assignments in class, and there were limited computers with SPSS available in campus computer labs outside of the classroom labs. Classroom labs were often booked with classes for the entire day, so access to the college labs was limited. Students didn’t have access to an SPSS license for their personal computers without paying for it out of pocket. I also didn’t have a teaching assistant, so walking around to help around 30 students, or even 15 pairs of students, to help them with their SPSS issues during class meetings was nearly impossible. These challenges

24

STATISTICAL SKILLS FOR THE WORKPLACE …

291

might be easily dismissed on some campuses where teaching assistants and computer labs are plentiful or licenses are provided for all students, but the vast majority of our students work at least part-time, leaving limited time to be on campus, and many are first-generation students with few resources. It became clear that using SPSS was placing an additional burden on my undergraduate students. The following year, I was asked to add our graduate methods course to my teaching responsibilities in addition to the undergraduate course. Our department has a Master’s in Public Administration program and most of our students are working adults, as is the case in many Masters of Public Administration, Public Policy, or Political Science programs. The majority of our graduate students are employed in state, local, or federal government offices, non-profit or community benefit organizations, and law enforcement. When deciding how to approach the course, I knew I needed to consider the purpose of including a quantitative analysis course in our curriculum and how it would best benefit our diverse group of students.

Understanding the Students’ Needs As I designed the class, I took into consideration that the majority of my undergraduate students were unlikely to attend graduate school but were likely to have professional careers that would require some basic statistical literacy, and most likely would be required to use or create reports that included basic descriptive statistics. I also considered that most of my graduate students were not likely to go on to PhD programs, and really had no need to learn complex statistical software packages just for the sake of learning them. Many of them already had careers, and were seeking a graduate degree for career development or to improve their eligibility and competitiveness for advancement. I decided to switch from SPSS to Microsoft Excel and to refocus my classes with an eye toward the use of statistics in the workplace. When I really thought about it, it made perfect sense. Almost every computer in office buildings throughout the country has Excel available and many of our graduate students already used Excel at work. Additionally, almost all students already have Excel on their computer or can easily obtain a no-cost copy through the university. As a worst-case scenario, students can purchase Office 365 online for a small fee.1 In sum, Excel is easily accessible and practical.

292

L. A. BRYANT

As I designed lessons, I would ask myself: Why am I asking students to learn this information? Is this information generally useful? Is this something one might use in the workplace? Does it help them read and understand news stories, policy reports, or other materials better? Will it help them communicate more effectively? Most importantly, will they use this? While we cannot know every skill a student might need in their careers, by reading public reports, policy briefs, and other government documents, we can identify the most common or most important skills required to create such documents.2 Teaching students how to understand, conduct, and report some basic analysis in an accessible program is more important than exposing them to complex statistical software they may never use.

The Benefits and Drawbacks of Teaching with Excel When I decided to move to Excel, I tried to take into consideration the benefits and drawbacks of the program and let those inform my choices. Over the course of teaching with Excel for the last four years, I have identified even more pros and cons of the software, so I will share some of those here, along with how I use the benefits to enhance the learning experience and address the drawbacks, if possible. Benefits Low-Cost/Easily Accessible I mentioned this previously, but one of the biggest benefits of using Excel is that it is easily accessible since many students already have it on their laptop or can get free or low-cost access to the program. Universities often have subscriptions to Microsoft Office 365, which allows students to download a full version to their computers to use offline. This is highly recommended because it allows students to use the data analysis tool pack add-on, which is needed to perform some of the advanced methods, like difference of means tests, regression, and even offers a simple way to make histograms. To make access easier for those students who do not already have Excel on their computers, I include a link to our university Information Technology webpage where students can access Office 365 and download a free copy for their computers. I also walk through accessing

24

STATISTICAL SKILLS FOR THE WORKPLACE …

293

add-ons and installing them in the program during class. I do this on both a PC and a Mac, because the steps are somewhat different. (This point is discussed in more detail below.) Versatile and Diverse Uses Most people, including students, know of Excel as a spreadsheet program and have used it to run calculations, organize data, make budgets, and a variety of other basic tasks, but Excel has over 400 different builtin commands and functions (far exceeding Google Sheets or Apple’s Numbers programs), allowing many complex mathematical, financial, and, of course, statistical applications. Excel can carry out all basic introductory level statistics, including ordinary least squares regression, logistic regression, and time series analysis, but also offers many features beyond computations which can be useful in research and in the workplace. For instance, it can create interactive data dashboards similar to those in Tableau, which are becoming more and more common. It has several built-in maps, so one can easily show data by country, state, or any geographic location. 3D charts or dynamic data can be added to maps, as well, offering good visualizations without having to learn fairly complicated software like ArcMap or R. When I first starting teaching with Excel, I worried that these additional features would be distracting or take too much time, but now I routinely incorporate some of these activities into my classes. Making a map is as simple as having a data exercise that includes a variable by state and students are often excited to see how easy it is for them to create maps that are useful and informative. Low-Cost Textbooks Available Statistics textbooks are often quite expensive. There are numerous affordable textbooks available for teaching statistics with Excel and many of them are even free, which is attractive as more and more universities are pushing for open source, low, or no-cost materials. One drawback is that many Excel textbooks lack any real theoretical or applied discussion about statistics. They are often written as “how-to” guides, which are wonderful for students as they learn a new program, but fail to demonstrate how or why this particular measure or statistic might be useful or appropriate for a given research question. To remedy that, I supplement the textbook with one that discusses statistics in applied research or real-world applications in addition to addressing these issues in my lectures.3

294

L. A. BRYANT

Lots of No-Cost, Quality Online Resources One huge benefit to using Excel is that there are endless online resources available, including Microsoft help, websites, and YouTube videos that demonstrate how to perform almost any task one can think of. While I walk through exercises in class, and often record the examples and make them available on our class learning management system (something I have found helps students tremendously, as they often have difficulty recalling exactly what we did during the class session), I have also started to include links to some of the best examples on YouTube over the years. These can be a time-saver for instructors and a helpful aid for students, especially outside of classroom hours. Drawbacks Data Errors The most concerning drawback to teaching with Excel is that data errors are common. It is easy to begin typing in a cell and inadvertently write over existing data, accidentally miss entire rows or columns of data when creating a formula, or change a column type format and change the values of the data within the cells. These types of errors are common when teaching, but they are also somewhat common in the workplace and research. In fact, there have been famous studies in economics and genetics research that have drawn attention to both the issue of accidentally deleting data and incorrect cell formatting leading to incorrect conclusions (Ingraham 2016), and scientists have even taken to renaming some human genes in an effort to get Excel to stop misreading them and auto converting them to dates (Vincent 2020). Studies of employers show that data entry errors in spreadsheets rank highest among employer concerns about using Excel, though most are not prepared to tackle the problem (Forrester Consulting 2019, p. 7). Making students aware of the frequency of these types of errors and using previous studies as examples is a good way to drive home the importance of careful data entry, which will serve them well in the classroom and beyond.4 No Syntax or Easy Way to Save Work for Replication Compared to the more traditional statistics software packages used in social sciences, the lack of a.do file, syntax file, or an output file is easily among the biggest drawbacks of Excel.5 Not only does this make it difficult for students to document or replicate exactly what they did, but it

24

STATISTICAL SKILLS FOR THE WORKPLACE …

295

can make grading more cumbersome than with other programs. I have students submit their entire Excel file and if answers do not come out correctly, I often must look for the error in their formulas so that I can let them know how to correct the error. Fortunately, these errors become easy to identify over time, as students tend to make the same mistakes, but it does add time to grading. Cleaning, Recoding, and Labeling Can Be Cumbersome Unlike Stata and SPSS, which make recoding data easy, recoding data in Excel can be clunky, requiring the use of logical IF commands or filtering data. Some features like identifying and removing duplicates or replacing blank or missing data are fairly straightforward, but it is still something students often struggle with. When I began using Excel, I did not realize how difficult cleaning data would be for students. I know to start the semester with clean data sets and save recoding exercises for the end of the semester when they have gotten more comfortable with the program. A related drawback is that besides variable names, there really are no data labels attached to numeric data in Excel. All labels in tables and charts or graphs must be added manually. I do make it a practice to always include a codebook tab in data exercises so that students do not have to guess what the various values mean. I did not do this my first few times teaching with Excel, instead I had tried to include the values in the header row with the variable name, but it has been one of the modifications I have made over time, and students quickly get used to looking at the codebook for information. It also introduces them to the idea of a codebook and identifying how variables are coded by other researchers, which is useful for anyone who uses data created by other researchers, especially MPA students who often work with data collected by various state, local, and federal agencies. Mac and PC Versions Are Slightly Different It is worth noting that the Excel for Mac and Excel for Windows versions have some small, but meaningful differences, most notably when making graphs and charts. I draw attention to these and provide examples using both during class. I find this reduces the number of post-class emails (and errors) I get due to small variations.

296

L. A. BRYANT

Google Sheets vs. Excel and Working Together in Real Time Google Sheets provides a good, free online spreadsheet that many students may want to use in place of Excel. While Google Sheets provides many of the same basic functions as Excel and could easily and effectively be used to teach descriptive statistics, it does not include anywhere near the number of advanced statistical functions that Excel does as part of the standalone program. There are third-party statistics add-ons for Google Sheets that can perform linear regression, ANOVA, and other features, but I generally refrain from suggesting third-party software in my courses unless I have thoroughly vetted it first. Google Sheets might also be desirable if an instructor or TA is looking for a way to work with students in real time on a shared screen. Microsoft Excel also now offers a co-authoring feature as part of the Microsoft Excel 365 environment. While 365 is not exactly the same as the desktop version of Excel, if you are using MS Excel in the classroom, it is preferable to Google Sheets.

Lessons I’ve Learned Along the Way When I first started teaching with Excel, I began the semester by asking students how familiar they were with Excel and if they had ever used it before. Most of my undergraduate students said they were familiar with the program and had used it sparingly. Graduate students, on the other hand, reported they were very comfortable with Excel and used it regularly. I had anticipated this response and designed my course assuming they knew the basics. I soon found out I was wrong. People Don’t Know What They Don’t Know If you simply ask people if they know Excel, they will tell you yes. That is what I discovered. Only a few class meetings into my first semester teaching graduate statistics for MPA students, I discovered that many of them did not know how to create simple mathematical formulas, sort data, or use any shortcuts. This meant I had to rethink my approach to teaching and how detailed I would need to be in my examples. I wrote myself reminders on how to incorporate basic commands as I showed examples in class and reorganized my syllabus to allow more time for repeated demonstrations.

24

STATISTICAL SKILLS FOR THE WORKPLACE …

297

The next semester, rather than ask students if they had used Excel or have them self-identify their skill level, I gave them a survey with very specific questions. I asked how often they used Excel (daily, weekly, occasionally, never), how often they had created tables, cleaned data, or performed basic data management, created formulas, and created charts or graphs. I asked if they had ever used pivot tables or power pivot, and I asked if they had ever written a research paper where they performed their own statistical analysis. This gave me a much better idea of exactly how familiar they were with Excel and I would highly recommend using a similar set of questions if one were to make the switch.6 You could even use the results in a dataset for students. Don’t Overlook the Small (but Useful) Things In my experience, when teaching statistics, we are so focused on making sure students learn the difference between mean, median, and mode that we often overlook teaching them other useful, practical skills. When I started teaching Excel, this became painfully obvious to me. I remember the day I made an offhand comment about how we could use Excel to randomize our data or draw a random sample and one of my students became audibly excited. She worked for the county school board and would have to draw a random sample of schools to visit each month. She typically did this by drawing papers out of a bowl in her office, but had no way to document that it was a truly random draw. Using Excel, she could record her screen and post the random draw of schools. This simple skill allowed her to provide more transparency at work. I now include this as a regular item in my course. One time I inadvertently gave my students a data file in wide format instead of long. I told them this was no big deal, and showed them how to use the copy and paste feature to transpose data. A student who worked for a state agency told me he had spent hours each month performing a similar task by hand. I have since added transposing data as the first step in one of their assignments. While these skills are not directly related to teaching statistics per se, they are data management skills that help people work more efficiently and effectively. That is part of teaching with Excel that I have embraced and learned to value. To that end, I have tried to incorporate at least one basic Excel skill in every lesson and homework assignment. It might be as simple as separating a column of string data into two columns using

298

L. A. BRYANT

Table 24.1 Topics and Functions Covered in Excel Exercises Statistics (using both formulas and data analysis tools)

Basic Excel Functions

Advanced Excel Functions

Descriptive Statistics

Elements of a Spreadsheet

Correlations Histograms

Column Types/Formats Charts/Chart Elements

Z-Scores/Z-Statistic

Tables/Table Formatting

Frequency Tables T-test

Sorting Data Filtering Data

ANOVA Regression

Transposing/Reshaping Data Text to Columns Flash Fill Sparklines/Trendlines

Recoding Data/IF command Pivot Table Slicers/Interactive Table Geographic Data/Maps Randomizing Data Random Selection/Sampling VLookup

text to columns or something as complex as recoding a variable using IF statements, but I think these are some of the skills that really help students become familiar with the software in a way they would not if it were not built into the lessons. Table 24.1 shows a list of statistics I cover as well as basic and advanced Excel features and functions that I work into the exercises throughout the course. I generally start with the most basic functions and progress to more complicated functions over the course of the semester. Pivot Tables: Students’ Biggest Challenge with Excel The first couple of times I taught using Excel, I stuck with basic functions available in the program or available through the data analysis tools addon. One key feature that is not available with these options is crosstabs, which is possibly one of the most useful items in the descriptive statistics toolbox. Determined to include this in my teaching, I discovered that crosstabs are easily created by using pivot tables. I know many longtime users of Excel who are unfamiliar with or intimidated by pivot tables, so I was somewhat hesitant to include them. After watching a few tutorials and playing around with them myself, I realized they are quite simple, yet powerful. I added pivot tables to both of my courses for crosstabs and in

24

STATISTICAL SKILLS FOR THE WORKPLACE …

299

my graduate class, we use them for more complex analysis and to build interactive data dashboards. I will admit that pivot tables are one of the activities that students struggle with at first however, after a little practice most of them pick it up and understand how useful they can be.

Teaching with Excel Can Be Excellent I felt a lot of internal conflicts when deciding to teach using Excel. It is very different than what I use for my own research and it is not what I learned in graduate school. I was not an expert in Excel, though I was quite familiar with it due to my previous jobs outside of academia, so I questioned if I was even capable of teaching statistics effectively with Excel. I have found that teaching with Excel can be quite rewarding. It is much more likely to be used by students beyond the classroom, especially when we consider the number who will never enter graduate school or have a career as a professional researcher, than SPSS or Stata. If you teach in graduate program that is designed for working adults, you get to witness their excitement as they realize how these new skills can be applied to their jobs. Excel offers everything needed to teach introductory-level statistics at the graduate and undergraduate level–from calculating a mean to running a regression. It is also highly accessible and versatile and continues to add new features, improve graphics, and even security features. And maybe one of the most attractive reasons to make the switch–when your students list Excel as a skill on their resumes, they will actually mean it.

Notes 1. There are limitations to Office 365 online, so it is not the best option and I advise students to seek another option. The primary disadvantage is that some of the add-on data analysis tools are not available. I will discuss this more in the next section. 2. Most reports of these types include basic descriptive statistics, often presented in a table and accompanied by a chart or graph. Occasionally, they will have a difference of means test or a regression analysis included, though more often they do not. These are all possible with the basic version of Excel. 3. I use Naked Statistics by Charles Wheelan as my supplemental text, but there are likely other books that would be appropriate.

300

L. A. BRYANT

4. I have also adopted the practice of reminding my students to make a habit of always creating a duplicate copy of the data they are using and keep the original sheet or sheets untouched. 5. It should be noted that there are ways to write scripts for Excel using Visual Basic, but this is outside the scope of either my graduate or undergraduate courses. 6. As it turned out, most of the students who reported using Excel daily or weekly were using forms or spreadsheets that simply required them to enter numbers, but not create their own forms, formulas, tables, or charts. Only a couple of students in my now five years of teaching graduate statistics had ever used pivot tables.

References Forrester Consulting. n.d. Think Spreadsheet Risk Isn’t A Threat? Think Again. Accessed January 19, 2021. https://incisive.com/wp-content/uploads/For rester-Opportunity-Snapshot.pdf. Ingraham, Christopher. 2016. An Alarming Number of Scientific Papers Contain Excel Errors. The Washington Post. WP Company, April 29. https://www. washingtonpost.com/news/wonk/wp/2016/08/26/an-alarming-numberof-scientific-papers-contain-excel-errors/. Vincent, James. 2020. Scientists Rename Human Genes to Stop Microsoft Excel from Misreading Them as Dates. The Verge, August 6. https://www.the verge.com/2020/8/6/21355674/human-genes-rename-microsoft-excel-mis reading-dates.

CHAPTER 25

Sneaking in Statistics Andre P. Audette

“Well, now I will never need to use math again!” Days after receiving word that my scores on the Advanced Placement calculus exam fulfilled my math requirement for college, I distinctly recall speaking these words to my fellow high school students. You see, as a political science major, I reasoned, there would be no use for math anymore. I would instead be focusing on running campaigns, choosing governmental systems, and developing public policies. I could not foresee all of the ways that statistics–and even calculus!–could play a fundamental role in my career. Yet this phrase is one that informs my research methods pedagogy to this day. Of course, I was wrong. Although my undergraduate training did not include a heavy dose of quantitative methods, I eventually came to see the utility of data for campaign polling, determining which systems of government best serve the people, and evaluating the effectiveness of public policies. Graduate school further opened my eyes to the world of “math for political scientists” (as the title of the first course in the

A. P. Audette (B) Department of Political Science, Monmouth College, Monmouth, IL, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_25

301

302

A. P. AUDETTE

methods sequence indicated), and most of my research and teaching now involves quantitative methods in some way. In fact, the first class I taught independently was a quantitative research methods course. It was the spring of my third year in graduate school, and the university across town had a last-minute opening for someone to teach a research methods course for advanced undergraduates and Master of Public Affairs students. I had taken the required quantitative methods sequence for my graduate work–and then some–and had successfully served as a teaching assistant for introductory American politics courses several times. I also worked at our campus’s teaching and learning center and felt prepared to take the next step in independent teaching. Soon after the call went out, I sent an email to my trusted advisors to see what they thought. While my advisors were receptive to me teaching a course, to my surprise, some were not so keen on that first course being a research methods class. At the time, I was not familiar with the language of “service courses,” which many consider research methods to be due to its high workload. Additionally, they cautioned, instructors of methods courses tend to receive lower marks on their teaching evaluations due to the dry and tricky material; I needed strong evaluations to go on the academic job market. However, wanting to provide opportunities for me to get my desired teaching experience, they agreed to recommend me for the job. On top of the difficult subject matter, the course was scheduled to meet during a difficult time slot: every Wednesday from 5:30–8 pm. Although teaching research design and statistics for two and a half hours a week was not the first teaching experience I expected, I made the most of it. The aphorism “the best way to learn is to teach” certainly applied, as I spent most weeks confirming what I thought I knew about t-tests, regressions, and p-values. I maintain to this day that it was the semester I learned the most about quantitative methodology. The hours spent learning all of these details made me confident in teaching the material, although, on reflection, it also led to a tendency to focus on too much of the micro-level statistical points at the expense of the broader takeaways I hoped my students would leave the class with. As the semester continued, I found myself trimming more and more of the technical statistical details to hit home the broader applications of the statistics that we were learning. Rather than starting with math and moving into application, I started with a problem and then showed students the math behind it. As it turns out, statistics was not the most compelling way to start a two-and-a-half-hour class period.

25

SNEAKING IN STATISTICS

303

By the end of the semester, I was pleased with how most classes turned out, and as we headed for the home stretch, the students began to engage in fastidious work on their projects. This led to even more work for me, as I kept up with each student’s research project and worked to troubleshoot data and software errors, on top of teaching research writing skills. This was rewarding work, but in the midst of my own research and coursework, it made me question whether there was a different, less taxing way to teach research methods. As the students dug deeper into their projects, I also saw the struggle many students had with the “scary” math that we discussed. For some it was liberating to grapple with the language of statistics and do their own research; others had the same mindset I did as an undergraduate, and just needed the credit to get through. By the end of the course, I felt confident that some students left with a new appreciation of statistics and quantitative methods. However, others were likely glad to be finished and would hesitate to dive back into the world of statistics without more prompting. Therein lies the wisdom in the phrase “now I will never need to use math again.” It has always been part of my teaching philosophy to meet students where they are at, especially when it comes to research methods, and I found that I was able to do so by leveraging my own thoughts and misunderstandings as an undergraduate. In my experiences working with undergraduate students–both inside and outside of research methods courses–many have the same attitude about math that I did. In fact, many have come to fear math or even choose to be a political science major to avoid any sort of quantitative reasoning. Although I fully agree that political science literacy and careers require at least minimal quantitative skills, one of the largest lessons I learned in my first research methods course was to recognize and try to respond to the emotional hurdles that come with research methods. Students were much quicker to learn once I opened up about my own prior hesitations with statistics and the sheer difficulty that comes with essentially learning a new language of quantitative methods. With these newfound lessons, I was ready to teach more statistics, including to students like me who never planned to take another statistics-driven course. Unlike the initial fears, my teaching evaluations were also solid. As I started on the next stage of my career, finishing graduate school and searching for academic jobs, I felt that the successful experience teaching research methods would be a useful asset on the job market. I applied to numerous jobs that asked for a background in teaching quantitative or

304

A. P. AUDETTE

mixed methods. A large portion of my teaching portfolio described the work I did to write guidebooks for common practical problems students face in the research process and meeting with students in office hours as they each wrote a full-length research paper using linear or nonlinear regression analysis. I had practiced interview answers about teaching methods and all the lessons I learned. I was fully prepared to teach sample courses on research methods and to accept a position with at least some methodology requirement. Instead, the job I ended up securing at Monmouth College was not advertised as having a research methods component, and our institution to this point still does not require a methods course for our students, nor does it offer a standalone political science methods course. Much like my high school perception of political science, it would seem that my students indeed found the right major if they were trying to avoid math! But the lessons learned from my methods teaching were not for naught… My first semester on the tenure track brought three new classes, including a Political Psychology course. At the time, Monmouth College was in the midst of establishing a “Quantitative Reasoning in Practice” (QRP) requirement, where each major would establish a course where students are exposed to basic quantitative skills or recommend such a course in another department. Although I did not seek a QRP designation in my first semester, I designed the course with the intention that it would eventually fulfill the QRP requirement for our majors. Additionally, it seemed difficult to teach a methods-heavy subject without some instruction on the methods themselves. This would be my quantitative methods outlet. As I was constructing my Political Psychology course, I was also finishing up several research projects that utilized survey experimental methods. I had recently conducted a few inexpensive experiments with co-authors on Amazon Mechanical Turk (MTurk) and found it to be an easily accessible and fun method. Survey experiments are also widely used in the field of Political Psychology, and I wanted to share some of my own work with my students. Also wanting to demonstrate some active and engaged learning in my teaching from my first semester, I took a chance: my students and I were going to design, conduct, analyze, and write papers on an original survey experiment. (And, having designed this before ever testing it in a class, we were going to do so over the course of just the last month of class.)

25

SNEAKING IN STATISTICS

305

Knowing enough about the practice of scaffolding–that is, structuring a class and supporting students to build up to a difficult concept–I had to find a way to prepare students for this task and garner enough buyin so as to not make it an impossible final project, including for some of the first-year students who would be taking the class. I talked about the general idea of the project on the first day of class and ensured that students were reading survey experimental work throughout the semester. Every few class periods, I sprinkled in miniature lessons about the logic of survey experiments. When we had a spare 15 minutes left of class, I asked students to complete worksheets designing survey experiments for a variety of political-psychological questions. For example, one worksheet asked if one wanted to know whether a Muslim presidential candidate would face discrimination among primary voters, what are two ways that one could design a survey experiment to figure this out? Students could then propose a simple question wording experiment, a list experiment, or another experiment to answer this question before moving on to a question about whether celebrity endorsements influence voters. Using the lessons from my research methods course, these worksheets were typically themed based on the substantive topic of the week: elections, moral intuitions, personality politics, racism, political scandals, etc. This gave students an opportunity to realize that there were different ways to answer the same question, gave them exposure to different types of experiments, and allowed them to be creative in how they would answer questions with a quantitative, empirical approach. Finally, I met with every student to provide resources for them as they designed and wrote up experiment proposals on the topics of their choice. Although survey experiments were not in the course description, nor part of the main substantive takeaways of the class, the experiment project essentially made its way into each of our class sessions. As students learned political psychology, along the way they also learned the logic behind survey experiments. We also had working sessions to develop our project and think about how it would fit into the world of political psychology, discussing question wording, demographic variables, and literature reviews. But the true test waited at the end of the semester when we would gather responses to our experiment from MTurkers around the country. The students designed a survey on responses to the Deferred Action for Childhood Arrivals (DACA) program. Our main goal was to determine whether respondents felt DACA recipients were more “deserving”

306

A. P. AUDETTE

if they were employed in a construction job or if they were students at a university. My students each contributed $20 for the survey (in lieu of purchasing an additional textbook for the course), and, sure enough, we found statistically significant results on our research question. We spent a full class period in the computer lab learning how to work with the data and run means tests in Excel. Every student left that day with at least some basic results that they could use to write their final paper for class. The guidebooks I developed in my previous research methods course came in handy as students grew excited about the project and wanted to run some of their own tests when they got back to their dorm rooms. Suddenly, a class that was not designed to teach students research methods had students talking about t-tests, regressions, and p values just as much as my research methods students a few years before. Still, some of my best-laid plans had to be adjusted. As students worked on the project, I cut back on readings and offered more time in class to address questions related to our research. One downside of sneaking in statistics was that, despite my statements to the contrary, students were not always aware that what they were learning would come back in the final project. The time for sneaking was over, and I did more review to solidify what we had started early on. What started as a “trial by fire” the first semester of teaching was giving me little insights here and there of how it could be adjusted in the future: more review of how survey experiments work, more notetaking on important methods concepts and group work, and even more planned support (particularly in class)–much like one might do more extensively in a research methods course. By the end of the semester of Political Psychology, each student had written a full-length research paper using original data on a project that they collectively designed. Although the students were as stressed as always about final papers, and although I was still early on in my teaching career, I had never seen students so engaged when working with statistics. Our final class period had not even started before students were talking with each other about what findings they used and what work went into their literature reviews. In comparison to the final presentations in my research methods class, I seemed to have gained more buy-in from students in this class. The students were even more excited when a news story about the project appeared on our college’s website after papers were turned in; I returned the next semester with students coming back to my office to further discuss our research.

25

SNEAKING IN STATISTICS

307

This project could have very easily been a fluke–the result of the students in the class, the project miraculously working out, or my own relative inexperience in the classroom. Still, I was encouraged by the results in my students’ final papers and in their evaluations of the course. I decided I would need to try “sneaking in statistics” again. My next opportunity to introduce a full-length survey project was in my Public Opinion course the following year. We focused less on experimental methods and more on traditional means of analyzing polling data. I followed a similar format as the Political Psychology course and snuck very basic statistics and research logic into the course throughout the semester by having students analyze early drafts of surveys I had conducted. I scaffolded the survey project by having students analyze polls from different sources, write poll questions, and propose topics for our class survey, and also allocated more time to focus on the project over the course of the semester. Learning from the Political Psychology course, I increased the amount of review and scaffolding to better support students in the course; students needed to remember the principles of which poll questions and polling methods were effective and which were not. Ultimately, we selected two topics the students came up with: social media’s effect on political participation and changing norms for political candidates. Once more, the students and I designed the survey, fielded it, and analyzed the results collectively and individually. And once more, I left the semester impressed by the engagement students had with the topic, project, and even with the quantitative aspect of the course. Importantly, this did not come at the expense of the substantive public opinion material that the course was intended to teach; rather, it helped solidify and give students first-hand experience applying the material to the world around us. By the time I returned to the Political Psychology course the next year (now a fully “QRP” designated course, along with the Public Opinion class), I had grown more confident in this method of teaching research to our students. Students still signed up for the courses for the material but left with a greater understanding of how political scientists use quantitative methods in their research. I began having students approach me about using polls and survey experiments for their senior research projects, seemingly heightening the methods training that our students were receiving (particularly without a formal methods course). The students who were most interested in learning more about quantitative methods also began asking about collaboration on research outside

308

A. P. AUDETTE

of the classroom and learning more about the methods that we used in the course, even signing up for research methods courses in other departments. And the students who did not go that far in their adoption of quantitative methodology still had the tools they needed to read a quantitative paper and assess which poll questions were more or less biased. In many ways, our department and our students were seeing the benefits of sneaking in statistics. Apart from the full-length survey project, I also found new ways in other courses to teach statistics, whether it was analyzing Census data to discuss the growing power of the Latino vote, comparing numbers across religious groups to discuss their political mobilization strategies, or having students gerrymander districts to change the makeup of the U.S. Congress. In most cases, quantitative reasoning was not a primary goal of the course; in fact, it may not appear in my learning goals at all. But in a world where increasingly students must grapple with quantitative political data, it makes sense to “sneak” quantitative literacy assignments into a variety of courses and not isolate it in the research methods sequence. Moreover, when those same students took my Political Psychology or Public Opinion courses–ones with a more intensive research project–they were more prepared, having seen the ways that quantitative thinking can contribute to a more comprehensive understanding of political science. The largest gains in quantitative literacy have come from students who, like me, never anticipated using math again in college. By incorporating statistics into substantive political science courses, anecdotally, I see quantitative political science becoming more accessible to students. Additionally, students who are most interested in data and statistics or who plan to go on to graduate work have had opportunities to go beyond the basic statistics used in the classroom to explore different avenues in their papers or even collaborate with me on independent research projects–all while not overwhelming those less keen on learning statistics in the first place. As our students work up to their individual research project capstone, the applied skills students take away can benefit them in their later coursework. The end result is a scaffolded statistical experience minus much of the negative emotional energy that is frequently attached to formal research methods courses. Plus, as someone who has come to appreciate the world of quantitative methods much more than I ever thought I would, I enjoy finding new ways to share this mindset shift with my students.

25

SNEAKING IN STATISTICS

309

While there are many benefits to sneaking statistics into substantive political science courses, this method is not a replacement for the depth of a research methods course. In programs where students are expected to regularly apply quantitative skills or where most students are projected to attend graduate school or use advanced statistics in their careers, a formal methods sequence would obviously serve students better. However, an extensive quantitative methods path does not meet most of our (Monmouth College) students where they are. Our department is constrained in being able to fill and cover a research methods course, and our students who go into more intensive quantitative careers are able to receive training in other departments. As such, I have come to embrace “sneaking in statistics” as a worthwhile model, especially for those programs such as ours that are unable to devote a course to research methods, and even for those with a research methods sequence. Students benefit from multiple exposures to quantitative methods and sneaking in statistics helps ensure that exposure to quantitative methods does not come at the expense of motivation and self-efficacy. Moreover, the practice of sneaking in statistics has made me a more effective instructor of research methods. By exposing students to quantitative methods throughout their political science education, the stakes are lower each time I teach the topic. Each course is likely not the only time that students will encounter quantitative reasoning. It also forces me to regularly recall my attitude toward statistics when I entered college and to place myself in the shoes of my students, who may be learning applied statistics for the very first time in my class. I cannot suppose any real or assumed experience with the topic merely because we discussed it a few weeks back in a research methods course; instead, I must work to explain the statistics and the connection to our substantive course material. As such, my explanations and answers to questions, at least from my perspective, tend to be more practical and approachable. Over the course of my career thus far, I have gone mostly by chance from branding myself as a research methods teacher to one who “sneaks in statistics.” My first independent teaching experience happened to be research methods because it was an open class and I wanted to teach. My current position does not involve teaching research methods because it was the job that happened to work out best in my life. I appreciate the advantages and disadvantages of both approaches, but I am satisfied personally and pedagogically with the opportunity to sneak statistics into my substantive political science courses. It is a strategy that has worked

310

A. P. AUDETTE

well in our context of a small liberal arts college, where many students come to college with the same attitude I had: “well, now I will never need to use math again!” Even though I (now) disagree, the phrase has become more and more instructive in my goal of teaching quantitative literacy and reminds me of the need to consider practical and emotional hurdles students face when learning research methods.

CHAPTER 26

Pedagogical Recommendations for Applied Statistics Courses Jennifer Bachner

Introduction Political science is consistently among the most popular undergraduate majors. In 2017, over 40,000 students were awarded a bachelor’s degree in this area (Data USA 2020). Moreover, tens of thousands of master’s degrees in public administration, public policy and public management are awarded each year. Students graduating from these undergraduate and graduate programs most frequently pursue careers in the law, government and the broader policy community, yet the quantitative methods courses required in these programs are almost always taught by academics with minimal practitioner experience (Lewis 2017). Based on my own instructional journey, this chapter will discuss how methods instructors can bridge the academic-practitioner divide to ensure their courses are valuable to students pursuing careers in government and industry.

J. Bachner (B) Data Analytics and Policy Program, Johns Hopkins University, Washington, DC, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_26

311

312

J. BACHNER

Importantly, the careers political science graduates pursue in both the public and private sectors have become increasingly quantitative. A decade ago, policy advisers, political consultants, program managers and public administrators did not need significant analytical expertise. Today that is not the case. To influence policymaking and decision making in today’s government agencies and private companies, the ability to analyze and interpret data is essential. In short, the quantitative methods components of undergraduate and graduate government-related degrees have never been more relevant to students’ career success than right now. This chapter will begin by reflecting on my journey as a quantitative methods instructor, first as a teaching assistant in a math boot camp, to the present, as Director of the Master of Science in Data Analytics and Policy (DAP) Program at Johns Hopkins University. The chapter will then explore the ways in which this experience teaching undergraduate, master’s and PhD students in government-related fields has taught me valuable lessons about how research methods faculty can ensure that our students develop a skill set and knowledge base that will propel them forward outside of the academy.

My Instructional Journey: From Teaching Assistant to Program Director My first experience teaching statistical methods was as a teaching assistant for Harvard’s Math Pre-fresher for incoming PhD students in the Department of Government. This “math bootcamp” covered probability theory, calculus and linear algebra and introduced students to the R programming language. The students were committed to careers in academia with a focus on conducting social science research. I then served as a teaching assistant for several terms for an econometrics course for undergraduate students. This course focused on using regression models for descriptive and causal inference. The students were eager to learn the content, though they were focused on developing skills for use outside of academia. Many of these students were pursuing careers in management consulting, policy analysis and the law. This experience led me to serve as a teaching assistant for a similar econometrics course at the graduate level. This “empirical methods” course was required for students pursuing a master’s in public policy at the Harvard Kennedy School. Many of these students, who were looking to pursue careers as policymakers in government agencies, struggled in the course and felt, at the time, that

26

PEDAGOGICAL RECOMMENDATIONS …

313

the methods being taught were not central to the skill set they needed to be successful. This teaching experience, more so than the previous two, caused me to think critically about how I can draw links between quantitative research methods and their relevance to the real world of policymaking and government leadership. After graduating with my PhD in political science, I joined the faculty at the Johns Hopkins University Center for Advanced Governmental Studies where I taught a research methods course for students working on their thesis for the Master of Arts in Government program. This course focused primarily on question formulation and research design. Most students in the program were building careers in federal agencies, state governments, government contract firms, polling firms and non-profit organizations. Over the next few years, my colleagues and I observed a dramatic increase in student demand for coursework in quantitative methods, as the fields of policymaking, governance and politics were moving quickly toward evidence-based decision making. The open data, civic technology and data-driven policymaking movements were gaining momentum and my students wanted to be part of these efforts. With this in mind, we developed a new master’s program focused on the application of quantitative methods to solving meaningful societal challenges. We launched the Master of Science in Government Analytics program in 2014 and have since expanded the program to include emphases on data visualization, machine learning, text analysis and programming. In 2020, we renamed the program to be the Master of Science in Data Analytics and Policy to better reflect the expanded scope of the degree and its relevance to both the public and private sector. As the program’s director, I teach several of the core courses, oversee the faculty and manage the curriculum. The program’s students, who generally work full time while pursuing the degree, are dedicated professionals who are extremely passionate about learning new ways to leverage data to advance their organization’s mission. Taken together, these experiences of teaching undergraduate, master’s and PhD students in various areas of social science research methods have taught me important lessons about how to shape and present the course content to maximize its value to students, most of whom are striving for meaningful careers in government, industry and advocacy.

314

J. BACHNER

Lessons Learned About Developing Impactful Teaching Content There are numerous statistics textbooks on the market, most of which feature similar content related to probability theory, descriptive statistics, regression analysis and visualization. This content is important and irreplaceable. It is inconceivable that any student would graduate from a top-quality undergraduate or graduate political science program without some exposure to quantitative methods; these skills have become essential to the jobs that many political science graduates are seeking. A narrow focus on statistical skills, however, leaves students at a disadvantage. My teaching experience has taught me that it is equally important to provide students in my methods courses with (1) examples of real-world applications, (2) technical and soft skills that complement and strengthen traditional statistical skills and (3) the ability to communicate the results of an analysis effectively in writing. Lesson 1: Incorporate Real-World Examples When I first began teaching statistics, I relied heavily on examples I’d encountered in my own graduate school experience in political science. These examples were drawn from the academic literature in my areas of study and therefore focused heavily on voting behavior, political socialization and public opinion. While these examples are interesting to students and certainly have real-world relevance, they neglect the many applications of quantitative methods to policy and governance challenges that students will encounter upon graduation. As a result, I have worked hard to incorporate a much greater variety of applications into my course content, particularly drawing from examples of how analytics are used in government agencies and policy-oriented research organizations. This can seem like a daunting task for someone with an academic background. However, for about six years now we have partnered with REI Systems (a government contract firm with an analytics focus) to host a Government Analytics Breakfast (GAB) Forum. The forum meets every other month and typically features a Chief Data Officer, Chief Risk Officer or Chief Evaluation Officer from a government agency. The speakers discuss ways in which their organizations have used analytics to optimize performance and efficiency. The Chief Risk Officer for the Internal Revenue Service, for example, discussed how the agency uses analytics

26

PEDAGOGICAL RECOMMENDATIONS …

315

to monitor its infrastructure, such as its hardware technology, to prioritize areas that need resources and updating. Data scientists from both the Securities and Exchange Commission and General Services Administration have presented case studies that showcase how the agencies are using text analytics to (1) analyze public comments and (2) ensure that requests for proposals for technology contracts contain required provisions related to disability accommodations. The GAB Forum has served as a tremendous opportunity for students to learn from leaders working in and with government. Further, it is livestreamed and recorded, so the events are accessible to anyone and can even be incorporated into a course as a supplementary resource. As an instructor, one of the biggest benefits has been the trove of concrete examples I’ve been able to gather about how government agencies and the private sector are using descriptive statistics, regression analysis, text analysis, machine learning and other quantitative tools to solve meaningful policy and governance problems. It is often challenging for academics to find opportunities to interact with policy practitioners; the GAB Forum has bridged this gap and helped me to become a much more effective instructor of analytics for those planning to work outside of academia. There are certainly many other valuable resources beyond the GAB Forum that are underutilized by instructors of quantitative methods. Research organizations such as the Sunlight Foundation, New America and Results for America conduct thoughtful, original and methodologically rigorous scholarship that focuses on addressing timely policy and governance issues. As an example, I have frequently showcased a social network analysis conducted by researchers at the Sunlight Foundation that identified the six issues in the area of immigration receiving the most lobbying attention, which included high-skill visas and family issues. This research has provided a great way for students to see, for example, how a policymaker can use a social network analysis to better understand the advocacy landscape surrounding a particular policy area. In short, we should be sure to look beyond the traditional academic literature when identifying applications of quantitative methods. If we rely solely on the journals we read during our graduate school years, or on those journals related to our current academic interests, we are likely to overlook a lot of useful research that is worthy of inclusion in our applied statistics courses. To be sure, most research that appears outside of academic journals has not been peer-reviewed. Nonetheless, it adds

316

J. BACHNER

important value where academic research is lacking, particularly when the students are working hard to pursue non-academic careers. A reasonable balance of academic and non-academic examples of quantitative methods applications may offer the best approach. Lesson 2: Emphasize Secondary Technical and Soft Skills Traditional statistics courses introduce students to probability theory, descriptive statistics, basic visualizations, regression analysis and measurements of uncertainty. Further, these courses teach students how to perform these analyses using a statistical software program. While these analytical skills are essential, they are not sufficient. It is common practice to assign problem sets and exams with clean datasets. These datasets are often free of missing data and omit any mention of measurement concerns, such as misreporting. Moreover, students are generally asked to calculate and interpret results on their own for these assignments. They are rarely asked to work in teams, develop a research design or present their findings in a way that is accessible to leaders who may not have a deep background in analytical methods. These peripheral technical and soft skills, however, are critically important to conducting empirical research, particularly in a non-academic setting, such as a government agency, polling firm or think tank. We can best prepare our students to be competitive job applicants by emphasizing these skills alongside traditional statistics. Data Cleaning When I teach my capstone seminar, in which students conduct an original, empirical analysis, students are often surprised at how much time and effort it takes to prepare their dataset for analysis. In the course, students are expected to use an existing dataset from a government agency or other reputable organization to answer an interesting and original research question. This necessarily requires students to deal with a variety of data cleaning issues, such as variable renaming, variable recoding, data merging, data sub-setting and identifying data inaccuracies (e.g., typographical errors, such as a number that exceeds the possible range of a variable). After teaching both statistics courses and the capstone seminar for several years, I came to realize that the above skills, as well as other skills related to coding and manipulating data, were vital to graduating from the

26

PEDAGOGICAL RECOMMENDATIONS …

317

program with a complete quantitative toolkit. The challenge, however, was that there was not enough space in the syllabi to teach these skills in an organized and comprehensive manner. As a result, my colleagues and I developed a new course on programming and data management. This course introduces students to the R language and covers the fundamentals of programming, including functions, data structures, data cleaning and manipulation, commenting and dealing with errors. Students are encouraged to take the course alongside the introductory statistics course. This new programming course is now required for the DAP program, and students have reported that the course has given them the foundational programming knowledge necessary to learn new packages and tools as they move forward with learning different types of analytical methods. Data Visualization Although it is fairly straightforward to create simple graphs such as scatter plots and bar plots in popular software packages, data visualization has expanded well beyond these basic means of displaying data. Today there is an appreciation for the value of displaying data in a way that is both visually appealing and scientifically revealing. Through the use of visualization tools, social scientists can glean insights from data in a way that is impossible with traditional statistical methods. Further, they can often advance their arguments more effectively using well-designed charts and graphs than through crowded tables. Additionally, in our new world where information is often shared via blogs and social media, rather than through printed journal articles, the ability to create beautiful graphs, and especially interactive and animated ones, is a highly marketable skill for students entering the job market in all fields related to the social sciences, business and journalism. In the DAP program, we now require a course on data visualization as part of the core requirements, as we recognize that it is a necessary skill set for anyone seeking to work in a field informed by data science. The course covers visualization in both Tableau and R, with a particular focus on using the ggplot2 package. And while an entire course on visualization might be unrealistic to require in many undergraduate and graduate programs, it is worthwhile to dedicate at least two weeks to the topic in any applied statistics course. Even two weeks of foundational content can provide students with a survey of the visualization tools that are now available and can be learned on one’s own with help from a variety of online tutorials and short courses.

318

J. BACHNER

Teamwork Statistical analysis and research at both the undergraduate and graduate levels are typically framed as individual undertakings. Students usually complete final projects, theses and capstones on their own to demonstrate they have met the learning objectives for a course or program. But while individual work is important, particularly from an assessment standpoint, it fails to mirror the collaborative nature of research in both academia and industry (Endersby 1996; Hunter and Leahey 2008). The quantitative research process is often long and complex, requiring team members who each bring a different skill set to the project. In government agencies, think tanks and consulting firms, for example, a research team might consist of a statistician, data scientist, subject matter expert and several research assistants to assist with data collecting, cleaning, coding and analytical tasks. With this mind, political science students should have opportunities to work on partner or team-based research projects. Working with others to conduct social science research teaches important soft skills related to communication, compromise, division of responsibility and accountability. Learning how to collaborate on research can be, at times, painful and frustrating, but these experiences can empower students to navigate the complexities of professional relationships when they enter the workforce. For example, a student may feel that a collaborator is not contributing their fair share of work to the project. Considering this issue from a student’s perspective, King (2006) writes, “A normative standard that is much more in your career interest is to ask yourself instead only: Is your coauthor making a positive contribution to your paper? If it’s a positive contribution, then you’re getting something out of your collaboration. Be thankful.” This is not the kind of lesson you can learn in R or Stata, but it is essential to becoming a good analyst. One way we’ve incorporated group work into the DAP program is through our course titled Applied Performance Analytics. This course focuses on how city governments use analytics to support data-driven decision making. At the start of the course, students are divided into groups of six in which each member assumes the role of a city official: mayor, public health director, budget director, police commissioner, human capital director or public works director. Each student is given information about the responsibilities and goals for their position. As the course proceeds, the groups are presented with policy challenges and accompanying data that they are expected to analyze when developing

26

PEDAGOGICAL RECOMMENDATIONS …

319

recommendations. Each student is expected to consider the constraints and stakeholder interests associated with their role when contributing to the development of these recommendations. In their course evaluations, students who have completed the course consistently express their appreciation for the experience, remarking on how much they learned about how analytics are used in a real-world context. Lesson 3: Teach Scientific Writing and Communication Over the years I have slowly, but consistently, modified the problem sets and final exams I use in my statistics courses to require an increased amount of writing. While a good analysis is essential, the results are of little value if they are not presented effectively in prose. Because the communication of results necessarily comes after an analysis is complete, it is often a topic that is completely absent from a traditional statistics syllabus. My interest in weaving an emphasis on scientific writing into all courses in the DAP program grew out of my frustration with reading students’ final capstone projects in the first couple of years after the program launched. I realized that we were successfully teaching students how to perform quantitative analyses, and many students were using sophisticated techniques like geospatial modeling, text analysis and machine learning to tackle important policy issues. However, I also observed that students struggled to frame their research questions and discuss their results in a way that was accessible to an average reader, and in particular, to a decisionmaker in their field. Throughout the program, courses now include instruction on how students can do the following when writing about their statistical findings: Explain why the research question is important. Students (like all scientific writers) need to remember to answer the “so what?” question for the reader. Students are encouraged to provide context and motivation when introducing their research question so the reader knows why they should bother with the rest of the paper. While the answer to the question is usually obvious to the writer, who has been immersed in the analysis, it is often insufficiently stated for the reader. Underscore the key takeaway point. Students are frequently under the misconception that a study’s takeaway point should be saved for

320

J. BACHNER

the end of a paper. I encourage students to, instead, state their findings right at the beginning, and continue to emphasize their main 1–3 points throughout their papers. I use the analogy of putting the most important information in an email’s subject line, and then sending several reminder emails to ensure the recipients really get the information they need. Write clear, concise sentences. I try to remind my students that the goal of a scientific paper is to make a data-driven argument. This is best accomplished by writing in a way that is accessible to both a technical and broad audience. Lengthy sentences that are cluttered with jargon and caveats are difficult to read and have the effect of burying the main points. I encourage my students to use footnotes and appendices as necessary to include all of the “extra” information that distracts from their main points. Julie Ann Miller, the former editor of Science News, summarizes this guidance well: “When writing about science, don’t simplify the science; simplify the writing.” Don’t overstate or understate conclusions. Many students (and seasoned academics) find it challenging to make a strong and clear argument without overstating the findings. As students move through their statistics courses, they are often overwhelmed with all of the ways in which an analysis can be flawed. In their own writeups, I then see students nobly addressing all of the limitations of their work. And while it is imperative that researchers be transparent and cautious when drawing conclusions, they should also stand by their findings (when justified) and emphasize the contribution of their work. The above suggestions certainly amount to a lot of additional content, not all of which can be reasonably incorporated into a single statistics course. Nonetheless, statistics instructors can embed more real-world examples, peripheral technical and soft skills, and a focus on scientific writing into their courses, and encourage their colleagues to likewise devote more time to these areas of focus in their methods and substantive courses. A commitment to teaching political science students how to apply their methodological and domain knowledge to critical challenges in both the public and private sector will ensure our field of study continues to serve as a pathway for career success.

26

PEDAGOGICAL RECOMMENDATIONS …

321

References Data USA. Political Science and Government. Accessed August 6, 2020. https:// datausa.io/profile/cip/political-science-government#employment. Endersby, J.W. 1996. Collaborative Research in the Social Sciences: Multiple Authorship and Publication Credit. Social Science Quarterly 77 (2): 375–392. Hunter, L., and E. Leahey. 2008. Collaborative Research in Sociology: Trends and Contributing Factors. The American Sociologist 39 (4): 290–306. King, G. 2006. Publication, Publication. PS: Political Science and Politics 39 (1): 119–125. Lewis, G.B. 2017. Do Political Science Majors Succeed in the Labor Market? Health Sciences 2: 3–5.

CHAPTER 27

The Accidental Methodologist Christopher Zorn

Prologue To be honest, I was a little bit shocked. I had, against many people’s expectations (including my own), secured a tenure-track teaching position during my first year on the academic job market.1 The position was in American politics and public law, my specialties, and I’d be teaching courses (e.g., Constitutional Law and Civil Liberties ) that I already knew and loved. Moreover, I would be teaching at a large, private research university in a major city, and in a department with ample resources, prominent colleagues in my field and others, and historically strong undergraduate and Ph.D. programs. The graduate program, in particular, was (to me) impressive: it was large, substantively diverse, and counted among its alumni several of the best young scholars and teachers in my field. On arrival, then, I was surprised to learn about the program’s graduate course sequence and offerings in methodology, and in particular those in

C. Zorn (B) Department of Political Science, Pennsylvania State University, University Park, PA, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_27

323

324

C. ZORN

statistics and quantitative methods. While the department had a lot going for it—including a dedicated course on qualitative methods—its graduate statistics courses were… not great. The course sequence lacked coherence, and the methods being taught were dated, as was the software being used. There was little emphasis either on technical detail or on intuition; instead, “cookbook” applications were the norm, with course content mostly focused on learning pre-packaged routines for commonly used statistical techniques. Few, if any, students elected coursework beyond the department’s minimum requirements, or enrolled in methods courses in any of the university’s many other highly regarded departments. This all came to be reflected in their projects, including their theses; after some time, I began to feel that stronger methods of training could improve our already excellent students’ publication chances and job prospects. After a year or so, I spoke with my department chair, a senior scholar in my subfield and a renowned advisor and mentor. The conversation went something like this: Me: “You know, it seems like our methods classes are kind of dated.” My department chair: “Oh, really?” Me: “Yeah. Like, [gives an example of something that was pretty badly outdated].” My department chair: “Do you think you could do better?” Me: “…um…maybe?” My department chair: “OK. Go ahead.” And that is how I became a political methodologist.

(Mostly) Happy Accidents The story above is a bit of an oversimplification, to be sure. In graduate school, I’d completed a “minor field” in methods. I’d taken courses in statistics and econometrics, and used some fairly sophisticated (for the time) statistical methods in my Ph.D. thesis. At the time I started my first job, I’d (once!) attended the annual summer meeting of the Society for Political Methodology. At the same time, if someone had asked me “Are you a methodologist”? My unequivocal response would have been “No.” In many respects, this isn’t surprising. As Roberts (2018, 597) notes, “very few students in my own subfield of comparative politics come to graduate school intending to be methodologists.” While this is changing, I’d argue that, historically, Roberts’ statement is even more broadly

27

THE ACCIDENTAL METHODOLOGIST

325

true: Few students in any subfield of political science come to graduate school intending to be methodologists. Once they arrive, most students learn that methods are something peripheral to the discipline: There are four major subfields, we’re told, or perhaps five if you’re in a program with a particular strength in public policy or public administration, and none of them are called “methodology.” Everything about most graduate programs—from the structure of course offerings to qualifying examinations to the academic job market—is geared around those fields, and students commonly shoehorn themselves into one or more of them, often at the insistent advice of their advisor(s). There are exceptions, of course, with both individual faculty and entire programs that are or have become known for “producing methodologists.” But, as a rule, methodology is something one does, not something one is. There are a number of ironies about this fact. In most graduate programs, methodology courses—including courses in basic statistics and data analysis—are among the very few that are required for every student, irrespective of subfield. This means that there is a more-or-less constant demand for people who can teach methodology at the graduate level. A similar dynamic exists at the undergraduate level, where nearly every political science department requires their majors to take a course in “Scope and Methods” (often a blend of rudimentary philosophy of science, research design, and “baby” statistics) or something similar, and many also require additional courses in methods, quantitative or otherwise. In other words, the demand for individuals with such skills is both constant and relatively high. Moreover, there are non-teaching-related career incentives to focus on methodology: Articles on methodological topics appear in both the discipline’s “top” general journals and in a growing number of methods-focused outlets; books with a methodological focus are regularly published by highly regarded academic presses; citation counts for methodological work are often higher on average than those for work in other subfields; and proficiency with such skills can open up doors to non-academic private-, non-profit-, and government-track careers. But despite these facts, Schrodt’s (1991) tongue-in-cheek anthropological characterization of political methodologists from three decades ago—as “witch doctors,” about whom “everybody agrees you only want one in the village”—still rings true in many of not most departments, including those with graduate degree programs. One might view these circumstances with concern, or even alarm: How is it that we’ve come

326

C. ZORN

to the point where one of the central components of graduate education in our discipline is seemingly so marginalized? How is it that so many core functions of an education in political science—particularly graduate education—have come to be taught by people who have come to that position unintentionally? What I hope to convince you here is that, rather than being a source of concern, the existence of “accidental” methodologists can in fact be a distinct advantage, particularly with respect to teaching methodology at the graduate level. As Roberts (2018, 598) notes, “(T)he vast majority of political methodologists run into methodological problems in the course of their substantive work and then develop approaches to solve these methodological roadblocks.” In other words, methodologists often evolve into that role, rather than emerging fully formed from Ph.D. programs. At least three characteristics of such a career path can be seen as improving—and, perhaps, can be consciously used to improve—the quality of methods instruction at the graduate level.

Relatability In another notable line from his brief essay, Schrodt (1991, 19)—writing about methodologists—states flatly that “(E)verybody fears them; most people hate them.” While the latter may be hyperbole, few can quibble with the former characterization. Who hasn’t begun a conference presentation, job talk, or other research presentation fearful of a potential methodological “gotcha” lurking in the Q&A? Historically, this fear has been compounded by the culture around political methodology, which for decades has been among the most competitive, confrontational, and status-oriented in the discipline (see, e.g., Shannon, 2014; Shames and Wise, 2017; Esarey 2018).2 But while the impact of that culture is most acutely felt in the relative lack of diversity in political methodology’s formal organizations and activities (Hidalgo et al., 2018), it also often spills over into how methodology is taught at the graduate level. For example, an unfortunate tendency in such courses is to emphasize technical details at the expense of applications, and general proofs over specific examples. Too often, this is combined with a dispiriting (and unwarranted) blend of condescension and self-importance that renders the entire experience unpleasant for everyone but the instructor themself. As Shames and Wise (2017, 819–820) trenchantly note, “(W)hen substance becomes subordinate to methods, and when methodological

27

THE ACCIDENTAL METHODOLOGIST

327

discussions seem competitive and nitpicky rather than collaborative and constructive, we want to run in the opposite direction.” Among the currents that—at least potentially—run counter to this culture is the existence of the accidental methodologist. As Roberts (2018) noted above, very few graduate students in political science come to graduate school with the goal of becoming methodologists; most are driven by a combination of a keen interest in politics and a strong desire to gain and contribute to knowledge about political phenomena. For such a first-year M.A. or Ph.D. student interested in (say) the Dutch party system, the professor in charge of a comparative politics proseminar (or a course on political parties, or corporatism, or some other relevant substantive topic) cuts a congenerous figure. In contrast, their introductory statistics professor—through a combination of unfamiliarity, math phobia, and the aforementioned cultural factors—likely engenders some combination of fear and dread, as does the prospect of having to complete one or more such courses on the path to their terminal degree. The irony, of course, is this: many if not most of those teaching graduate quantitative methodology can remember a time when they weren’t a “methodologist” at all, and shared that same dread. Internalizing that simple fact can make an important difference in how methodology is received by our graduate students. That means acknowledging and even embracing students’ math phobia and other anxieties, rather than minimizing (or, worse, deriding) them; facilitating a classroom and department culture around methods that are supportive and collaborative rather than competitive; and making clear that—rather than being the exclusive province of a bright, nerdy few—methodology is something all social scientists do, and can contribute to. In practice, such an approach requires patience, empathy, and above all humility: exactly the things that many “accidental” methodologists sought and benefited from in their own graduate education. Here, I’ll offer a confession that my own graduate students have heard for years: Despite nearly three decades of teaching statistics to M.A. and Ph.D. students, I am really bad at linear algebra. Every spring, I find myself relearning the finer points of characteristic polynomials and QR decompositions, so that I can, to the best of my ability, teach linear regression to my first-year graduate students. Yet I’ve come to view this mental block as an advantage, for at least two reasons. First, with occasional exceptions, most people do not find linear algebra especially intuitive. If I did, it might serve to make me a worse instructor, in that I might not

328

C. ZORN

be as able to understand (and empathize with) my students’ struggles to understand the topic. Second, it helps clarify what is and is not an essential part of their instruction. More broadly, my “accidental” status points up the importance of conveying intuition and practical application along with technical detail, while at the same time offering reassurance that one need neither be fixated on mathematics nor, especially good at it to be successful in that aspect of what we do.

Solving Puzzles In a 2007 essay, Gregory Treverton summarized the difference between a puzzle and a mystery: There’s a reason millions of people try to solve crossword puzzles each day... Even when you can’t find the right answer, you know it exists. Puzzles can be solved; they have answers. But a mystery offers no such comfort. It poses a question that has no definitive answer because the answer is contingent; it depends on a future interaction of many factors, known and unknown. (Treverton, 2007)

Following this distinction, the most interesting questions in political science undoubtedly qualify as mysteries. Yet the day-to-day work of formulating, conducting, and presenting reliable, valid empirical research on those questions often presents itself as a series of puzzles: discrete challenges for which we must find the (or, at least, a) solution. Solving such puzzles requires deep domain-specific knowledge, whether that be an understanding of political and social conditions and processes, familiarity with disciplinary and policy norms, or technical acumen. In most scientific fields outside our own, this need for broad, up-todate expertise across a broad range of areas has led to specialization. Quantitative empirical work in fields from acoustics to virology often relies on teams of researchers, each with specific backgrounds and proficiencies.3 While such fields lean on domain experts for the design and execution of those studies, they also tend to rely on statisticians—many of whom have little or no substantive understanding of the phenomena under study—to conduct and interpret statistical analyses on the resulting data. For reasons both mysterious and mundane, political science has, for the most part, failed to adopt such norms. While co-authorship and other

27

THE ACCIDENTAL METHODOLOGIST

329

forms of collaboration have become more prevalent in recent decades, political scientists have remained reticent to outsource a significant part of their research to those outside the discipline. The result is that we tend to rely on ourselves to solve the puzzles we face in our research. This, in turn, places the accidental methodologist in a particularly valuable position. Because they are themselves substantively motivated, their work tends to be problem-focused, and informed as much or more by theory as by formal or technical considerations. Such a shared perspective arguably both encourages collaboration and makes such shared work easier and more efficient. The tendency to focus on problems also shapes teaching, particularly (I would argue) at the graduate level. The goal of most M.A.- and Ph.D.level courses in quantitative methodology is to convey a set of skills necessary to understand and conduct empirical research. That is, while some courses and programs emphasize the more theoretical aspects of statistical and algorithmic methods, the vast majority are applied in nature. In my own teaching, one manifestation of this emphasis is in the structure of students’ required weekly data analysis exercises. Those exercises almost always contain a simulation-based component, one that requires students to demonstrate both programming skills and their grasp of the statistical or other properties of the techniques on offer. But unlike what might be assigned in (say) a statistics program, those exercises also always contain a “real data” component. The latter most often builds upon an existing piece of published empirical research, and typically requires students to demonstrate both technical and substantive understandings of the questions being asked and answered. More broadly, a focus on applications—on solving the puzzles that confront researchers daily in their quest for knowledge—advantages individuals who themselves have abiding substantive interests, and who have encountered and solved the puzzles that arise in the pursuit of those interests. Prioritizing those interests also fosters a valuable humility, both in terms of what the latest whiz-bang statistical technique can accomplish, but also about the place of methodology in one’s broader understanding of politics.4

Craft As I noted earlier, methodologists occupy a liminal position in political science, a field which itself straddles disciplinary boundaries. On the one

330

C. ZORN

hand, over the past century, political science has become increasingly scientific in its norms and conventions. The initial rise in quantification and increased attention to fundamental empirical concepts like reliability and validity in measurement has in recent decades been followed by a greater emphasis on theory-testing, causality, and cumulation of knowledge. These broad shifts have been accompanied by changes in undergraduate and graduate training, disciplinary practices (including replication and preregistration requirements at peer-reviewed journals), and career paths. At the same time, political science remains a social science, with all the attendant challenges that imply. Scholars in other STEM fields often note—not incorrectly—the uncertain, contingent, and sometimes seemingly arbitrary nature of our scientific enterprise, commenting that much of what we do seems to be “more art than science.” Against this backdrop, teachers of graduate methods in political science must navigate between the Scylla of scientism and the Charybdis of nihilism. We are charged with ensuring that our students grasp the power of the tools they are taught, and the details of their proper use, while at the same time avoiding instilling overconfidence their ability to reveal truths about the slippery reality we study. We are to emphasize the importance of contested theories and assumptions in everything we do—including quantitative analyses—without leading our charges to believe that social science is nothing more than a disingenuous exercise in rhetoric. In short: We must convince them that what we do (and, more relevantly here, how we do it) is neither fully an art nor fully a science. Between art and science lies craft, once described by Roseanne Cash as “the dovetailing of discipline and imagination, dedication and inspiration.” We might think of craft as embodying three distinct characteristics. First, craft is a synthesis of the practical and the creative; both the practice of a craft and its product are neither wholly utilitarian nor wholly aesthetic, but a blend of both. Second, craft is inextricably bound up with the idea of tools: their creation, adoption, modification, use (and misuse), and potential. Finally, mastery of a craft is acquired and refined both through formal instruction and through experience. In teaching this craft, it’s easy to emphasize the importance of practice and repetition; as one of my own former professors was fond of saying, “The only way to get better at methods is to do more of it.” But equally significant—and perhaps even more so, given the practical limitations of time and attention in a graduate program—are narratives that provide

27

THE ACCIDENTAL METHODOLOGIST

331

nascent researchers with a vision for how theory, data, and methods can come together to advance knowledge. One approach I have found valuable is to offer graduate students “research bundles” of my own work: a final, published paper or book chapter, together with selections from the months (or sometimes years) of previous drafts, computer code, reviewer and editorial suggestions (including rejections), email chains, and other materials that were integral to the development of the project over time. My experience suggests that students gain diverse benefits from these materials. By “seeing” the research process, they begin to understand the ways in which it can be messy, convoluted, and contentious. They also begin to internalize the dynamics of collaboration, the norms by which our profession operates, and the fundamental unpredictability of many of its institutions. Perhaps most important, such “bundles” make clear that the practice of research depends equally on the methodical grind of science and on flashes of innovation and artistry that often seem unpredictable if not wholly random. More generally, if empirical research in political science is a craft—practical yet inspired, creative yet disciplined, both careful and novel, and honed through experience—then teaching graduate methods is perhaps best likened to overseeing an apprenticeship: a combination of rigorous training, disciplined practice, and the passing down of mores and folkways that leads to deep knowledge and enables decades of future work. Viewing what we do as a craft, and our teaching as an apprenticeship, underscores further the value of accidental methodologists. The pragmatism that accompanies years (or decades) of focused research in the pursuit of substantive knowledge forces us to strike a balance between overconfidence in what the methods we teach can achieve and pessimism around their inevitable limitations. That pragmatism also underscores the importance of discipline and focus, and the value of balancing formal training with exploration, play, and other less-structured paths to developing critical skills. Finally, because we understand that the world moves fast and careers are long, we teach students both the tools they need to know and also how to acquire new tools going forward, a skill that is arguably more important than any single method.

Epilogue The year following my conversation with my department chair, I taught my first graduate statistics course. Like many in that situation, I frequently

332

C. ZORN

found myself challenged: by the course material itself, but also by how best to convey it; by striking the right balance between theory and practice; and by the questions asked by my students, for which my answers all too often felt insufficient. Grappling with these challenges meant relearning things I thought I knew, through the eyes of others with very different backgrounds, strengths, and goals. It also meant figuring out, often in real time, what was and was not effective. In this respect, learning to teach methods was and is very much like learning to use them: a form of an apprenticeship, but this one was led by my students, whose puzzled looks and nodding heads helped make me better at what I do. In the end, I learned at least as much—and probably more—than the students themselves, including about the limitations of my own abilities as a teacher and (especially) as a methodologist. The latter point highlights a final salutary attribute of the accidental methodologist: the fact that their existence reflects the dynamic and often creative dialectic—some might say tension—between teaching and research. While teaching methodology arose somewhat spontaneously for me, once I began doing so, that teaching influenced the choices I made about research and writing, and vice-versa. To take but one example, one of my first published articles on methodology evolved from a short seminar I was asked to give on panel data analysis, a topic about which, at the time, I knew relatively little. Publication of that paper led me in turn to devote more of my teaching to that topic, and also to do further research and writing in that area. In the social sciences, this is a common tale: teaching methodology, particularly at the graduate level, often shapes and is shaped by one’s research, including one’s substantive interests, skills, and personality. Moreover, because of its uniquely shared character—no matter our expertise or subfield, we’ve all taken methods courses—graduate-level teaching in methods is, especially, driven by the lived experience both of doing research and of learning to do so. In light of this, I suggest that those of us who find ourselves accidental methodologists embrace that status, and use its inherent strengths to our advantage, and to that of our students as well.

Notes 1. I was—and remain—keenly aware of the combination of privilege, historical accident, and dumb luck that led to this occurrence, and of how unlike the

27

THE ACCIDENTAL METHODOLOGIST

333

vast majority of academic careers both then and now this was. My hope is that this unusualness doesn’t undermine any value this chapter might have to anyone reading it. 2. At least some of that culture arises from far broader trends in STEM fields more generally. See, e.g., Malcom and Feder (2016), and cites therein. 3. The fact of that specialization is in turn incorporated into the institutions and norms of scientific publishing. For example, many scientific journals have recently adopted the CRediT taxonomy (https://casrai.org/credit/), which requires that each author specifies in which of fourteen categories her or his individual contribution(s) to the article were made. 4. I often tell my own graduate students that, despite their required status, my methodology classes are the least important courses they will take in graduate school, because the methods they learn there are effectively useless without the substantive knowledge they gain elsewhere.

References Esarey, J. 2018. What Makes Someone a Political Methodologist? PS: Political Science & Politics 51 (3): 588–596. Hidalgo, F. D., S. Linn, M. Roberts, B. Sinclair, and R.Titiunik. 2018. Report on Diversity and Inclusion in the Society for Political Methodology. Report to the Society for Political Methodology, February 21, 2018. Malcom, S., and M. Feder, eds. 2016. Barriers and Opportunities for 2-Year and 4-Year STEM Degrees: Systemic Change to Support Students’ Diverse Pathways. Washington, DC: National Academies Press. Roberts, M.E. 2018. “What Is Political Methodology?” PS: Political Science & Politics 51: 597–601. Schrodt, P.A. 1991. Political Methodology Explained. The Political Methodologist 4: 19. Shames, S.L., and T. Wise. 2017. Gender, Diversity, and Methods in Political Science: A Theory of Selection and Survival Biases. PS: Political Science & Politics 50: 811–823. Shannon, M. 2014. Barriers to Women’s Participation in Political Methodology: Graduate School and Beyond. The Political Methodologist 21: 2–6. Treverton, G.F. 2007. Risks and Riddles. Smithsonian 38: 98.

CHAPTER 28

From Step-Child to Innovative Leader: Political Science Research Methods Over the Decades Andreas Sobisch

I have taught political science research methods since the early 1990s, interrupted for a few years by a stint in university administration. At the time I was a newly-minted Ph.D. on a tenure-track appointment at John Carroll University, a mid-size liberal arts university in Cleveland. I volunteered for the assignment because I felt well-prepared and at ease with the material, even though I had struggled at times with math and statistics. But I had been very fortunate to have had several excellent research methods instructors at both the undergraduate and graduate levels, and this gave me the confidence necessary to teach this class. My first instinct was to emulate the courses and instructors that I was already familiar with. This approach was certainly not unreasonable, though it obviously provided less opportunity for innovation. But I wasn’t

A. Sobisch (B) Department of Political Science, John Carroll University, Cleveland, OH, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_28

335

336

A. SOBISCH

looking to be innovative in those days, I was looking to survive. Since I had never taught an upper-division course before starting my new position, I had to prepare a number of brand-new classes within a relatively short period of time. Not only was this terrifying, it was also a lot of work! The research methods teaching assignment was therefore very opportune: all I had to do was teach what I already knew. There was minimal need to develop specialized expertise or conduct labor-intensive background research the way I had to do for my other classes in the early years. For example, I developed a course on Christian Democratic parties in Europe, which required me to read up on nineteenth-century Catholic social and political thought, about which I, a Lutheran from northern Germany, knew practically nothing. Teaching research methods definitely did not present this kind of a challenge. Or so I thought. I made the conscious decision to place my new course on as broad a topical footing as possible in order to reduce the chance of overstretching my comfort zone. Faculty at smaller institutions, who usually teach a wide variety of courses, will certainly be familiar with this mindset. I ended up including a little bit of everything in this course: epistemology, philosophy of science, research design, probability theory, and basic statistics. I still have that first syllabus! There existed at least two major difficulties with teaching methods in those days (naturally, these insights did not occur to me until much later). For one, the course was not well integrated into our political science curriculum; I suspect that this may have been typical of such courses at the time. It was a free-standing course, focusing on what, for the students, must have seemed fairly esoteric and technical topics rather than politics, much less political “science.” The course evaluations after the first semester of teaching this class reflected this: to the typical undergraduate political science major the course did not seem particularly relevant nor, more discouragingly, in any way connected to what they were studying in their other political science classes. They considered it boring and, frankly, useless. In fact, since this was a required course for the major, students resented having to take it (this resentment was so intense that our department seriously considered dropping the requirement. We believed that it was causing us to lose majors). I doubt that it occurred to these students to ask themselves what was “scientific” about political science. To wit: political science classes taught knowledge, not how that knowledge was created in the first place.

28

FROM STEP-CHILD TO INNOVATIVE LEADER: POLITICAL …

337

At the time, undergraduate courses were generally lecture-based, and not as centered on student participation as they typically are today. Thick, expensive textbooks were the norm in most classes, including in research methods. In these early years it never occurred to me to use original research articles in my classes. Today, I do this with regularity, even at the introductory level. Since none of us professors had received any training in pedagogy, most of us probably felt more comfortable with such a traditional, textbook-centered approach, unless we were especially gifted or creative. Fortunately, as my career progressed, I managed to leave this “old-fashioned” approach to teaching behind me. I was aided in that by the significant technological innovations that have taken place in our profession over the past 25 years. As we will see further below, this has benefitted the teaching of research methods in particular. A second reason why teaching research methods was difficult was that the opportunities for students to practice their newly acquired research and analytical skills were severely underdeveloped, at least at my university (and, I suspect, at most similar-size institutions as well). Today we take for granted that everyone has access to reasonably user-friendly software on their own computers, that data is ubiquitous, and that every paper ever published is only a few keystrokes away, but that was surely not the case in the early 1990s. These modern conveniences probably did not become the norm at most institutions until 8–10 years later. Instead, campus computing was based on cumbersome and inaccessible mainframe systems; PCs and laptops were not in widespread use yet (certainly not among students); and, let us not forget, there was no Internet, not even a campus-wide computer network. In fact, few students were computer literate and even fewer were at all interested in learning how to use them. “Book learning,” as noted above, was the order of the day. Even library catalogues were generally still completely analog; at best, the available digital search systems were poorly publicized and not easily accessible to undergraduates (our library had one computer that had a low baudrate link-up to the libraries of Cleveland State and Case Western Reserve universities, but it was the best kept secret on campus. I discovered it only by accident). So even the simple assignment of conducting a literature review on a given topic meant, at the very least, countless hours of searching through card catalogues and bound back issues of journals (provided the university had a subscription to them), or traveling to the libraries of nearby research universities.

338

A. SOBISCH

The major turn-around for me occurred in the summer of 1997. I was invited to Seattle, Washington, to attend a workshop sponsored by an innovative start-up by the name of MicroCase. The company had been founded by a former sociology professor at the University of Washington, who had set out to create a comprehensive, yet affordable and easy-touse data analysis software capable of allowing undergraduates to conduct their own statistical analyses of authentic social science data. The company compiled five data sets, excerpting variables from the NES, GSS, WVS, UCR and World Almanac, and packaged them, along with the requisite analytical software, on a pair of floppy discs that students could install on their own computers (or that could be installed in a university’s computerlabs that had become fairly commonplace by then). I immediately realized the potential for this software to revolutionize the teaching, and learning, of research methods and adopted it for my class the very next semester (although, it must be said, the full utilization of its potential still had to await further progress in the dispersion of classroom projection technology). Classes would become significantly more “hands on,” with instructors being able to demonstrate the analysis of data as well as the creation of tables and graphs “live” during class time; and with students having the opportunity to conduct their own “original” research within the confines of a single semester. Research methods thereby came to embrace, and benefit from, the “experiential learning” ethos that was beginning to sweep across higher education. Even complete novices could now simulate the scholarly research process. I would even go as far as to argue that this technological innovation helped transform the subfield of research methods from a laggard into a leader in modern social science teaching: no longer would these classes be dull and abstract; on the contrary, they were fast becoming the most exciting ones in the curriculum. Students used to lament having to take our research methods course, but in my experience that is no longer the case. The process described above went hand-in-hand with our students fast becoming more technologically savvy. This includes basic computer literacy. For them, learning to use a new software is generally no longer difficult. I use SPSS in my class, and it has worked very well for me and for the students. It is user-friendly, with a layout similar to Microsoft Excel, even if the functionality is a little different. Students intuitively understand how data is organized in SPSS, and the pull-down menus function in a way virtually identical to what they already know. Even

28

FROM STEP-CHILD TO INNOVATIVE LEADER: POLITICAL …

339

during the initial COVID-19 lockdown in the spring of 2020, when classes shifted to remote learning on short notice, very few students had difficulties transitioning to using the SPSS software remotely on the campus network (unless they had connectivity issues). They were similarly adept at managing the interface between remote SPSS output files and our learning management system. A few years earlier a calamity like COVID-19 would have had an even more significant impact on student learning. Students’ quantitative literacy has also much improved compared to 30 years ago, as many high school curricula have placed much greater emphasis on numerical competency. While math phobia still exists, basic statistical concepts, such as, say, measures of central tendency and dispersion, are now relatively familiar territory that no longer absorb a significant amount of class time. This frees up time for other subjects. For example, it has allowed me to digress on more specialized topics, such as how to conduct survey research, and I am now able to carry the class all the way through to multivariate analysis and even logistic regression. It has also allowed me to spend more time on subjects that I consider particularly important, for example, contingency table analysis, which is a tricky, but extremely valuable, skill. As will be discussed at the end of this essay, this opportunity to teach more advanced analytical techniques, in greater depth, is the result of the redesign of our research methods curriculum as much as it is our students’ quantitative aptitude. This discussion of the development of teaching research methods would not be complete without mentioning the enormous innovations in general information technology. For example, versatile search engines, such as JSTOR, along with fingertip access to most, if not all, academic journals, have allowed even relatively inexperienced researchers (i.e., undergraduates) to efficiently locate relevant sources, and good models, for their semester-long research projects. The lack of access to source materials used to be a major impediment to such research because of the time-consuming nature of literature searches in the past. Even seemingly small things, such as that most academic journals nowadays publish abstracts along with their research articles, have made an enormous difference in terms of helping students to identify useful sources quickly. Finally, any researcher today has access to a plethora of public and private databases, mostly free of charge, on the Internet. A good example are the FBI crime statistics. These are stored in a user-friendly Excel format and can be downloaded into SPSS with just a few mouse-clicks.

340

A. SOBISCH

As college instructors we all improve our craft over the years; we gain expertise and confidence, and we learn from our mistakes and stick to approaches and techniques that work well, discarding those that do not. And we develop new ones. This seems particularly relevant in the area of research methods, where many concepts and practices are notoriously vexing for inexperienced students; examples include conceptual definitions, levels of measurement, constructing proper hypotheses, or analyzing contingency tables, to name but a few. I have found that a key is to make the experience fun and relevant by connecting lessons and assignments to what they already know (e.g., sports stats); employing interesting videos and other visuals that can easily be found on the Internet; or by using creative, but short writing assignments so that they develop the habit of connecting arguments to evidence that they have collected and tabulated themselves. It is a welcome indicator of success when we are able to observe improved student performance over time, even more so if we can connect that improvement to specific adjustments we have made to our teaching. For example, though I have always been a great believer in “Cooperative Learning,” I have struggled at times with integrating it into my classes for fear that it would be too time-consuming if used regularly. But this teaching tool is ideally suited for my research methods classes. Students have to complete weekly assignments, and I require that these be completed in groups of two (I used to require larger groups, but that ended up being logistically problematic). The objective is for students to “talk through” the assignments, helping each other to complete the homework and thus learn from each other. This has the added benefit of cutting down the number of assignments I have to grade, significantly reducing turn-around time. The quality of these take-home assignments has improved dramatically since I have adopted this method. Another way of gauging improvement that is ideally suited to research methods courses is that on tests and exams we can ask the same questions from semester to semester without students being able to “cheat” because it is relatively easy to modify the specific content of an assignment without modifying its substance. For example, we can ask students to define concepts, phrase hypotheses, create and/or analyze tables and graphs, but by changing the variables involved we can make it an entirely new assignment. Another thing that I have recommended to anyone who would listen is to switch to take-home midterm and final exams. I started doing this almost a quarter century ago, and I have never looked back. It

28

FROM STEP-CHILD TO INNOVATIVE LEADER: POLITICAL …

341

makes for higher quality answers, and we can always adjust our grading to account for students being able to use their “books and notes.” But I concede that this is highly idiosyncratic: I have never enjoyed cramming for stressful exams, and so I see no reason for imposing this on my students today. Nor have I found it particularly illuminating to grade students’ “poor” answers. My philosophy is that the best learning happens by “doing it correctly,” not through memorizing information. Most recently, the COVID-related switch to online learning has reinforced my belief in the value of customizing my supporting teaching materials. For example, instead of using a commercially published workbook containing SPSS lessons and assignments, I have created an entire series of instructional videos (using our educational learning system’s video recording application) on how to use SPSS, conduct basic data analysis, and create attractive visuals. These 10–30 min. videos turned out to be quite easy to produce, and I managed to complete each one on the first or second “take.” They cover the same ground as the workbook does, minus the parts that I do not like, such as confusing or overly detailed lessons and assignments. Such customization also saves the students a bundle. I plan to expand on this idea by “converting” an entire textbook (in my first-year statistics class; see below) onto instructional videos and narrated power-point presentations. In that way I “save” about half of the textbook’s content. As explained earlier, even as a much younger instructor I had never been completely happy with the “textbook approach” to college teaching. These books tended to be expensive for students, sometimes ridiculously so, and very often I was unable, because of time constraints, to use all of the chapters in class (research methods textbooks are notoriously over-ambitious). That seemed wasteful and insulting to students. But more importantly, I discovered that textbooks have a way of creating a pedagogical straightjacket such that they tend to tie instructors to a particular theoretical approach, which leaves a lot less room for their own creativity. Textbooks also constitute secondary literature, that is, they force students to study a subject by reading “about” it, rather than “engaging” the material more directly. I realized this most clearly years ago in my introductory comparative politics class, where the classic “country-by-country” approach could never do justice to the rich variety of cases and theories that make up this vast field (the problem tends to be somewhat less acute in upper division classes, since large textbooks are not as common). It turned out that, in the early 2000s, MicroCase

342

A. SOBISCH

also published a comparative politics version of their data sets, complete with a workbook organized along various theoretical themes. This allowed me to “test” various theories empirically in class, while it empowered the students to conduct actual data analysis in the context of a semester-long research paper assignment. Students thus engaged primary sources even in their introductory classes and led me to realize how valuable this was. A little over a decade ago, my department embarked on an effort to upgrade our methods curriculum, such that it was, by moving significant parts of the content of our free-standing “Political Science Research Methods” course to a second required course, aptly christened “Introduction to Methods.” The new course was designed to provide a deeper and more complete introduction to the theoretical underpinnings of academic political science research. It included topics such as philosophy of science, research design, ethics, qualitative research, various theoretical approaches to the study of politics, history of political science, and it even provided a primer on quantitative and statistical concepts such that students would be able to “hit the ground running” at the next level. At the same time, the existing research methods course was upgraded from 3 to 4 credits by including a new “lab” component. This lab component represented an official recognition that research methods involved a significant hands-on element that required that additional class time be allocated to it. Thus, we now had a mandated two-course methods sequence with a total of seven credit hours. A few years later, as part of a college-wide core-curriculum revision, a third course was added to our methods sequence: “political analysis.” This 100-level course was designed to introduce first-year students (most of them prospective political science majors) to “foundational quantitative analysis in a political context”: that is, basic statistics, but with a political science bent. This course is part of the college-wide “Quantitative Analysis” requirement, and similar courses exist in other departments, such as Mathematics, Biology, and Psychology. The department further created an Applied Politics concentration that is designed to prepare students for the world of professional politics, such as campaign management, consulting, and research. We also work closely with the Mathematics department in administering and staffing their Data Science major and minor as several of our students choose to minor in data science. “Research Methods” have come a long way since being the “stepchild” of political science education, at least at my institution. The value of methodological competence has been recognized by even the most

28

FROM STEP-CHILD TO INNOVATIVE LEADER: POLITICAL …

343

obstinate traditionalists in my department. After all, quantitative skills are highly attractive in today’s job market. At the same time, it is not of particular relevance to us what our graduates do with the knowledge and skills they have acquired in our new 3-course methods sequence. They don’t seem to follow any particular pattern in terms of their postgraduate careers: some attend graduate programs, many go to law school, and others enter the public or private sectors directly. My department’s overriding goal is to teach our students how to think like a social scientist: to observe the world with an analytical mindset and get into the habit of looking for linkages between phenomena, that is, for potential causes and effects. We believe that these habits and skills will suit them well in their careers as well as in life, as citizens.

CHAPTER 29

Teach Me If You Can: Teaching Political Science Majors Statistics at a Hispanic-Serving Institution Dongkyu Kim

When I taught an introductory statistics class for the first time in 2016, I immediately realized that something was seriously wrong. My pedagogy was not working. I was probably too excited. Right after finishing my Ph.D. program, I might have been too ambitious and eager to teach. The goal was simple; I wanted my students to understand the theories of statistics and apply them to real-world situations. Unfortunately, the outcome was a complete failure. After the first semester of teaching statistics, I had a chance to have a conversation with one of the top students from that class. He said that although he had tried hard to follow my class, he was not able to understand most of my lectures. I was shocked because I believed that at least the top 5% of that class was understanding

D. Kim (B) Department of Political Science, University of Texas Rio Grande Valley, Edinburg, TX, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_29

345

346

D. KIM

what I was teaching. It turned out that I was not able to teach anybody in my first statistics class. For almost every semester since then, I have taught an Introduction to Political Science Research course that incorporates research design components into the introductory statistics class, which has allowed me to tirelessly revisit my method of teaching statistics. I recently received a Facebook message from another top student who took my class in fall 2019 and started an online master’s program in summer 2020. She said, “OMG Dr. Kim, I literally could not thank you enough for your research class! I feel like I’m one step ahead of some classmates. I’m so grateful for your class and your teaching methods”! I felt rewarded and relieved. Although I still need to revisit my pedagogy in many aspects, I feel like I am on the right track to accomplish my original goal. As such, I believe that many instructors in Political Science would be interested in my method of overcoming some fundamental challenges in teaching an introductory statistics class.

Teach Me If You Can In retrospect, I think that the key was understanding the student body of my institution. Located in a border area of South Texas, our university serves mostly first-generation, low-income college students who are predominantly Hispanic. Although I knew this when I took the job, I did not initially understand what it would mean for my teaching. For this student population, family comes first; most students live with family and work almost full time. Accordingly, my students need to balance their college lives with both work duties and familial obligations. Like many other students in public universities, what most of our students want is simply to pass classes. This tendency—to be happy with a C—is very prevalent among our students. For me, the student body of my institution seems to have the following mentality: “I don’t have time for this. Teach me if you can.” At first, I could not understand why most students did not complete their assignments before attending class. Some said that they needed to take care of their great grandmothers. Others said that their kids were sick and work shifts did not allow them to attend the class. As a variety of excuses accumulated in my mailbox, one thing became clear: I could not assume that my students had time to sit down for hours to digest 20–30 pages for my class. Once I acknowledged this reality, I felt that I needed to change

29

TEACH ME IF YOU CAN: TEACHING POLITICAL SCIENCE MAJORS …

347

almost everything about my class. For the remaining part of this chapter, I want to share the changes I made to meet this challenge.

How to Make Them Read Even though I understood that students did not have enough time to prepare for my class, I could not entirely drop the course readings because previewing course materials expedites the learning process. Having students who are prepared is always better than having students who have no idea what they will learn for that day. This is especially the case for statistics, as it is almost like learning a new language. Before learning about statistical theories, formulating hypotheses, collecting and analyzing data, and interpreting results, students need to familiarize themselves with some basic vocabulary, such as “parameter,” “standard error,” “sampling distribution,” and “p-value.” The more they are exposed to these concepts, the better their chances of understanding what they are learning. In that regard, the course reading was critical. The first task was to adopt the right textbook. Above all, I needed a textbook with chapters that introduced and briefly explained basic statistical concepts in about 4–5 pages. Serving as a good primer, this might be an effective tool to cope with a student body with the teach-me-if-you-can attitude. In many cases, textbooks—even great ones—have 30–page chapters. After assigning at least five different textbooks over several semesters, I concluded that a typical statistics textbook would not work. Furthermore, it was difficult to find a textbook that aligned well with my style of teaching statistics. It was so hard to find the right one for me that I seriously considered writing my own textbook. Second, I learned to avoid textbooks that explain statistical concepts through mathematics. Certainly, equations are unavoidable in teaching statistics. However, I saw that books full of mathematical equations and proofs scared my students. One of the major reasons why my students and I were so frustrated when I was teaching statistics for the first time in 2016 was related to this issue. Most students in my class hate numbers and equations. Often, they wait to take the course until the last semester before graduation. When they finally take my class, they encounter a textbook full of equations. This simply confirms their fear and makes it look hard for them to successfully complete my course. Fortunately, I was able to find a textbook that meets these criteria: Pyrczak and Oh’s (2018) Making Sense of Statistics.

348

D. KIM

Of course, having the right textbook was not enough to make the students read. For students to overcome their teach-me-if-you-can attitude, the instructor needs to provide some kind of incentive. Because most students are happy just to pass the course, there is no big difference between having them read 4 pages and 40 pages. Simply, my students were not motivated to read. My strategy was to make the reading part of the grading scheme. I usually set aside 10% of the total grade for reading quizzes. Once students realized that it constituted a significant portion of their grade and that they needed to do it to pass the course and eventually graduate, they would do their job. I administered reading quizzes online and made them available throughout the week so that students could have enough time to take them at their convenience. Also, as the reading quizzes introduced new concepts and vocabularies, all quizzes were open book without any time limit. It worked out well, and the reading quiz became an integral part of my class. As the students had already been primed with the new concepts for the day, it became a lot easier for me to teach. It was also more pleasant, as the students were able to absorb the new knowledge more easily.

Basic Tasks to Become a Good Instructor While preparing students to be ready for the class, I also needed to prepare myself to be an effective instructor. The first task I encountered was to set up clear boundaries for my lectures. As mentioned before, I wanted to teach my students the theories of statistics and their application to real-world situations. And I believed that the students needed to learn regression analysis. Of course, the time constraints determined what I could cover in an introductory class. With a typical semester system, about 20 lectures are appropriate, considering exams, review sessions, and holidays. During this time, I could cover topics such as research designs, significance test, descriptive statistics, t-test, chi-square test, analysis of variance, correlation, and regression analysis to accomplish the set goal. In other words, it was somewhat clear what I needed to teach. However, we always want to teach more. This challenge becomes apparent whenever I try to squeeze in some additional topics. In a sense, I believe that this is an important practice to strengthen my teaching because we never know what will work or not without trying it. But it sometimes becomes problematic because this additional knowledge could simply go beyond the level and expectations of an introductory statistics

29

TEACH ME IF YOU CAN: TEACHING POLITICAL SCIENCE MAJORS …

349

class. Regression analysis with a limited dependent variable is a typical example of this. Right after I tried to explain the concept of maximum likelihood and predicted probabilities several years ago, I immediately realized that the topic was not only difficult to explain to beginners of statistics, but also scary enough to discourage them. Thus, I now set clear boundaries for my lectures, and I stay within them. The second task was to slow down the pace of my teaching. Every time I teach an introductory statistics class, I announce that I do not assume that the students have any prior knowledge. However, I have found that I keep forgetting about my own pledge without realizing it. There was one incident that made me realize the importance of repeatedly reminding myself not to assume that my students would know the language I use when teaching statistics. When teaching the concept of average, one student asked me what the summation notation (sigma) means. He did not seem embarrassed to ask that question at all, and no one laughed at his question. I was shocked. What about the square root in explaining standard error? What about the integral when I taught the probability density function? In retrospect, I believe that many students were left behind simply because of the language I used and the way I explained things by falsely assuming what they would already know. Over time, I have learned to keep checking back on my assumption regarding students’ prior knowledge and slow down the pace if necessary. In this way, I have become more attentive to where my students are and more adaptive to what they need. Over time, I realized that the slow-paced lecture goes well with my pedagogy for students with the teach-me-if-you-can attitude. First, because most of my students juggle school, work, and family obligations, it is important to provide no more information than they can digest in a day. At a slower pace, I can deliver the knowledge effectively to the students by repeating the concept and practicing the same thing several times to make sure everyone understands. As I will elaborate later, I typically spread out a topic across multiple modules to downsize the volume of knowledge to enhance my students’ learning process for the day. Ever since I realistically adjusted my pace, I have seen fewer students disappear or drop out in the middle of the semester. Second, the slower pace leaves room for students to learn a statistical package. Nowadays, most introductory statistics classes in Political Science teach students a computer program for data analysis. Learning a computer program adds another layer of anxiety and agony for students

350

D. KIM

who are already challenged with the conceptual and theoretical aspects of statistics. A slow-paced lecture smooths the transition from the lecture to the lab sessions. Additionally, it allows me enough time to teach students some basic data management skills. Because I do not teach any estimation techniques until mid-semester, I have about six weeks reserved for teaching preparatory data work, such as finding, downloading, managing, merging, and cleaning data. Many students do not know that statistical analysis requires tedious and effortful management of data. Slower-paced lectures allow me to teach the related skills.

Secret Ingredients Then, how can I teach statistics effectively? To most students, learning statistics is boring and painful. By teaching an introductory statistics class over and over, I have realized that the way I explain this already boring and challenging subject could be one of the major culprits for the anxieties or miseries my students have experienced in my class. I must confess that I have lived in my statistical bubble. As an instructor, I forgot the fact that I had a long history of mathematical education before being equipped with a good command of statistics and that I was trained to use statistical tools for about five years in an intensive doctorate program. Accordingly, explanations based on mathematical statistics make sense to me, but this was not the case for my students. The first lesson I learned was that everything should be taught in basic terms. For instance, when I taught the central limit theorem (CLT) in 2016, I started my lecture by showing my students different types of probability distribution functions, including Bernoulli, Binomial, Poisson, normal, and standard normal distribution. Then, I explained the normal approximation process. The immediate question I received was not about how things work regarding the CLT, but about if they needed to memorize all of these different types of distributions for the exam. When I said no, their expressions indicated, “then, why are you teaching those to us”? One thing became clear to me. I needed to learn how to explain statistical concepts simply and intuitively. The first strategy was visualization. Let me take the CLT as an example. For the slow-paced learning process, I spread out some essential components across several modules. First, I spend time explaining different types of variables rather than the probability density functions, such as nominal, ordinal, interval, and ratio variables. Students learn that different types of measurements result in varying types of frequency distributions. I show

29

TEACH ME IF YOU CAN: TEACHING POLITICAL SCIENCE MAJORS …

351

them how these look with lots of graphs. In the following module, students learn about the concept of sampling and sampling techniques. Here, the students learn that we typically study one sample out of the infinite numbers of potential samples we can hypothetically draw from the population. Next, I introduce the concept of the sampling distribution. By utilizing a computer program (STATA), I randomly draw a hypothetical population and then take 10,000 samples of 30 observations. With data on 10,000 means, I demonstrate what the frequency distribution of these means looks like. In the last module, I finally reveal that the CLT describes the shapes of the sampling distribution of the means, which the students already saw through computer simulation. I summarize the CLT with three arguments: normal approximation and convergence, the law of large numbers, and the estimation of the parameter. For every step, instead of just verbally explaining it, I help them visualize how it works through computer simulations. Second, I have found that analogies are effective tools to explain statistical concepts in basic terms. The concept of the degree of freedom is a typical example. I remember one of my teachers saying that the degree of freedom is just N minus 1. Well, this might be a way to circumvent the tricky context of the concept, but I do not believe it is an effective way to enhance students’ understanding. Thus, when I teach, I prefer to provide a technically correct definition. Instead of just saying “N - 1” to my students, I offer them a full definition: “The number of values that are free to vary to estimate the population parameter” or “the number of chances to estimate the parameter with a given sample.” Then, I came up with an analogy of a hamburger tasting. The story goes like this. A student heard about a new burger that just came out in the nearest restaurant. The student has just enough money to taste five items on the menu. When he/she arrives at the restaurant, however, he/she forgets the name of that burger and finds out that there are only five burgers on the menu. So, with the given money, how many chances does he/she have to taste it? From this analogy, students could learn that when we are estimating the population average with a sample, the degree of freedom is the number of observations. In this burger case, the person in the story has five chances. When he/she has tried four burgers and is left with only enough money for the last item, he/she is no longer free to choose. I use this analogy to explain the “N - 1” degree of freedom for calculating a standard error. I have seen that students find this analogy interesting and that it helps them understand this critical concept easily. For every statistical concept

352

D. KIM

I teach, I try to come up with an analogy because this method makes it easier to convey the tangible meanings of mathematical concepts. The third strategy was to install a repetitive learning process. I believe that effective learning is a repetitive and thus possibly tedious and painful process. I came up with two tactics for this. First, I spare at least 15 minutes to review the course material previously covered before I start a new lecture, summarizing my previous lecture and briefly going over the main points. Also, throughout the semester, I assign several review sessions through which I can touch base with my students regarding how much they know and what they need help understanding. Usually, I call on students by their names and ask specific questions. If the student cannot answer the question, I explain that concept one more time. Students do not like to be called upon, but this practice pressures them to review the material outside of the classroom, as they do not want to be embarrassed. One caveat, though, is that this could make some of them drop the course. In this regard, setting a welcoming and encouraging tone in class is important. Once everyone feels comfortable talking about what they know and sharing their struggles in the conversation-based review session, this makes a huge difference in students’ attitudes. This could significantly attenuate the teach-me-if-you-can attitude. Nowadays, I see students flipping through their notes and teaching themselves before class starts. It feels excellent. Along with the regular review sessions, I also redesigned my entire class around some ideas for the purpose of repetitive learning. I label them “anchors” because these tools can hold my students anchored tightly amidst the scary sea of statistics. The first anchor is a symbolic figure that connects the sample to the population with an arrow, which can be found in any textbook for an introductory statistics class. For about ten modules, students get to see this figure over and over. After introducing it in the lecture about sampling bias, I repeatedly use this picture to explain other concepts, such as sampling distribution of the means, sampling techniques, re-sampling simulation, CLT, central tendency, zscore, and finally, hypothesis testing. There are two reasons why I use this figurative anchor. First, I believe that the image is more effective than words to help learners grasp statistical concepts. Second, despite the centrality of CLT or sampling distribution to most parametric statistics, many students fail to recognize its importance once they are introduced to specific estimation techniques simply because the class does not cover

29

TEACH ME IF YOU CAN: TEACHING POLITICAL SCIENCE MAJORS …

353

it sufficiently. While I do teach some estimation skills, I want my students to remember the big picture. The second anchor comes out of the first one. When I teach significance testing, I make it clear that the calculation of 95% confidence interval (CI) around the estimate comes from the first anchor. Then, the equation for calculating CI becomes the second anchor. I call it “the golden equation” and let my students memorize it: the estimate plus/minus 1.96 times the standard error. I ask my students about it in every review session. Furthermore, I clearly go over how this golden equation can be applied to other estimation techniques, such as simple t-test, difference-in-means test, paired t-test, correlation analysis, and regression coefficients. I believe that this second anchor is a particularly effective tool in teaching statistical concepts. In many cases, students assume that they have to conquer all statistical techniques in order to understand the rationale behind them without realizing that theories of statistics and the golden equation are the essential part of most statistical estimation techniques. This anchor helps them connect the dots. In other words, this anchor helps my students to relate the first half of my course to the second half. For most undergraduate students, regression tables look scary. After anchoring them securely into the fundamental knowledge of theories of statistics, I see students approach and understand those scary tables with confidence. Most of my students can read regression tables correctly by the end of the semester. Lastly, I want to emphasize the importance of consistency across class activities. For instance, when I first designed the reading quiz, I thought that it was just a primer for the students before coming to class, having them read the textbook so they could follow my lectures. Therefore, I did not pay close attention to the language used in the textbook and the specific ways that concepts were explained. As a result, I failed to address some inconsistencies between the readings and my lectures. One day, an inquisitive student came to me quite confused because the textbook did not consider the degree of freedom in calculating a variance, unlike my lecture. I felt horrible telling my student that the information in the textbook was wrong. At the same time, I have found that all books are different in terms of the depth in which they cover statistical concepts. For example, some textbooks treat the CLT as just a different name for the normal approximation process, and others go deeper and explain that it means that the linear combination of random variables is also a random variable. Of course, we cannot and do not need to teach

354

D. KIM

everything. However, it is important to feed our students with consistent information in order to avoid confusing them.

Conclusion As an instructor, teaching an introductory statistics class almost every semester for about five years at a university that serves mostly firstgeneration and underprivileged college students has been a valuable opportunity. It has allowed me to experiment with various pedagogical approaches and continuously update them based on the feedback I have received. Having to teach this course every single semester expedited the trial-and-error process and helped me finally find a better way to teach my students, which would not have been possible in the ordinary circumstances where an instructor teaches a course only once every couple of years. It also helped me hone my statistical knowledge and skills. I could have adopted a draconian class policy, hammering my students with mathematical statistics and problem-solving practice and imposing a strict grading scheme. Although I believe that every pedagogy has merit, this was not the right option for me. Situating myself in their shoes, I could not disregard the substantial level of fear the students had about mathematics and the obstacles preventing them from investing themselves in studying statistics. Most instructors who have faced a similar situation would likely agree that teaching an introductory statistics class in a “standard” and mini-graduate seminar style would scare off most undergraduate students and simply not work. I believe that many instructors of introductory statistics classes in Political Science share my concerns. I hope that my experiences and approach help to address their challenges and frustrations.

Reference Pyrczak, F., and D.M. Oh. 2018. Making Sense of Statistics: A Conceptual Overview. New York: Routledge.

CHAPTER 30

Excel, in More Ways Than One Whitney Ross Manzo

Introduction Growing up, I identified pretty strongly with Hermione Granger. If you never read or saw the Harry Potter books or movies, the main thing you should know about her is that she’s a nerd. She loves learning, she loves the library, and most of all, she LOVES knowing the right answer. That was basically me from the ages of 5–18. I hated math and science. I made fine grades in these classes, but “fine” isn’t enough for the student who wants to be The Best. When my high school guidance counselor tried to make me take the advanced calculus course my senior year, to “challenge” myself, I laughed and took the regular version instead. I scoured Rate My Professor in college to find the easiest possible science class to fulfill my Gen Ed. It was so easy to play this role of the STEM hater, because no one expected differently; after all, I was a girl! And girls just aren’t good at these subjects.

W. R. Manzo (B) Department of History, Political Science, and International Studies, Meredith College, Raleigh, NC, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_30

355

356

W. R. MANZO

It turns out that girls aren’t actually worse at math than boys; studies have shown that girls achieve similar scores in both elementary and middle school math (Hargreaves et al. 2008) and that math anxiety is more significantly related to poor math performance than gender (Engelhard 1990). What matters more is the stereotype that girls aren’t good at math. This is passed down to girls as young as kindergarten from their parents and teachers (Gunderson et al. 2011), and girls whose mothers passed on the stereotype performed poorly in math, while girls whose mothers strongly objected to the stereotype performed at a similar level to boys (Tomasetto et al. 2011). I suppose the girls and math stereotype worked quite well on me, because I grew up believing it as the gospel truth. So, when the time came in my undergraduate career to settle on a major, I ran far from STEM types and chose what I considered at the time to be about as far from math as you could get: political science. Of course, little did I know at the time that there’s actually quite a bit of “science” in the study of political science. I chose it as my major because I probably wanted to go to law school and was obsessed with CNN, not because I wanted to learn the philosophy of science and how to correctly perform a social science experiment. I especially didn’t want to learn any more math. Imagine my horror when, late in my junior year, one of my professors blithely mentioned that he required all of his senior thesis students to include at least some statistical analysis. It was like a record scratch in my brain—say what ? I ended up being able to skip any math in my senior thesis by choosing a different professor to be my mentor, but I now knew that math was actually pretty important to political science. Years later, when I was choosing whether I wanted to complete a history PhD or a political science PhD, the main point of contention was whether I wanted to learn a new language (required by most history graduate programs) or engage with math. I chose math, but it was a close one. If you had told me then that not only would I learn a lot of math in my graduate studies, but I would regularly TA for an advanced statistics class and earn a certificate in methodology, I would have laughed myself silly. The idea of actually teaching math as the instructor of record would have sounded as realistic to me as the idea that I could attend a school of magic in Scotland! And yet, I really did learn all kinds of math and methods and I really do teach it as a professor of political science every spring.

30

EXCEL, IN MORE WAYS THAN ONE

357

Unfortunately, because I teach at a women’s college, I often hear stories about how terrible math and science are. I am sympathetic—after all, I believed the same for a good chunk of my life—but it does mean that I have an uphill battle on my hands. Students are already nervous when it comes to research methods (Milligan et al. 2014; Bridges et al. 1998), so when I tell my students that I’m going to teach them social science statistics, the sounds of dismay can probably be heard in space. However, as I have developed my methods course, I have learned that teaching them statistics is one of the most useful things I can do. Specifically, teaching them how to properly use the software program Excel for descriptive statistics, testing of simple hypotheses, and data visualization has paid dividends beyond my wildest dreams.

How I Started the Course When I was interviewing for my current position—a general American politics position at a teaching-intensive undergraduate-only institution— I was explicitly asked if I could teach methods because “the current professor is tired of doing it.” As a nervous interviewee, of course I answered yes, visions of having to hastily review Bayesian statistics and logit models dancing in my head. Imagine my relief when “methods” at this school meant only teaching how to develop a good research question, review relevant literature, and design a research project! Students didn’t even need to carry the project out (that was reserved for the senior thesis class, which was supposed to be taken in the semester following methods). I will be the first to admit that I disapproved of this methods course at first. After all, I was fresh out of graduate school, having just learned the absolute cutting edge in social science methodology, so I wasn’t even sure that “methods” was an appropriate name for the class. “Thinking About Research” felt more accurate. Also, as is the nature of most graduate students, I now knew just enough about methods to be super critical of other people’s use of them. Because I was still in this mindset, even though now I was a real professor, my first instinct was to critique whatever had gone before me in my institution’s methods class. Looking back, I can only laugh at how clueless I was—thinking carefully about a research question and what has been found on the topic previously is very obviously just as important as whatever the latest statistical modeling technique is. In my defense, I had just graduated from a well-regarded

358

W. R. MANZO

R1 graduate program and the culture definitely exhibited a strong bias against smaller institutions (Jafar 2012). Another reason I was determined to shake things up regarding methods instruction was that I desperately wanted to impress my new colleagues. I was acutely aware that, now that I was on the tenure track, the clock was ticking to prove that I deserved to earn tenure and promotion. At a teaching institution like mine, teaching excellence and creative pedagogy are regarded similarly to fancy publications, so I was highly motivated to disrupt the status quo and make a big splash. Because I had already pinpointed the methods course as a place of concern, I decided that course was the obvious opportunity. I even politely refused the very kind offer of the professor who had taught the class before me (the one who was eager to offload it) to help me get started. Think of what an awesome tale I could tell come review time, I thought, having taken this broken methods course and fixed it up all by myself so that our institution could begin churning out social science methods geniuses! (The hubris of newly minted PhDs sometimes knows no bounds.) Lastly, I wanted to add statistics to the course specifically because I wanted to help my female students lose their fear of math. After all, I had been them once upon a time, and felt that I had missed many years of skills-building by failing to see that statistics was so useful. I wanted to save them from my fate, and along the way wouldn’t it be nice for my tenure portfolio to be able to show that I helped increase the quantitative literacy of our famously math-avoiding women? Of course, when it came time to actually teach the methods course, I was extremely nervous. As I have mentioned, I had taken extra courses like Categorical and Limited Dependent Variables and Time Series in my graduate program to earn a Methodology certificate, and I had also served as the Teaching Assistant for the fancy graduate methods professor for several semesters in grad school, so I intellectually knew that I knew what I was doing. However, as therapists will tell you, knowing something intellectually is very different from knowing something emotionally, and as a first year assistant professor imposter syndrome was my constant friend. When I looked at the “Research Design and Methods” assignment on the department course list and saw my name next to it, I sometimes felt pride—I was being trusted with an upper level required course!—but more often I felt terror. I don’t know anything about teaching methods, I thought. I barely know how to use them!

30

EXCEL, IN MORE WAYS THAN ONE

359

So, while writing the syllabus for POL 334 Research Design and Methods, I felt an odd and discomfiting combination of arrogance, fear, and determination to prove myself. This resulted in me falling back onto what I knew: I used the same textbook and similar assignments as my introductory methods course in grad school, I set the course calendar to move at the same pace as my graduate cohort, and I decided to be the first Meredith College political science professor to teach regression. This way, I could be both familiar with what we were covering and proud that I was changing the game. As you might imagine, putting juniors in a course that moved quickly and skipped basic statistics to jump into regression without preparing them beforehand didn’t go so well. My students were shell-shocked, intimidated, and each class period was a silent and awkward horror show. As I began to recognize that they simply weren’t ready for what I was teaching, I tried ever more desperately to at least make them learn something difficult, and my distress made me harsher with them than I should have been. By the end of the semester, we were all totally ready to forget the whole thing ever happened. If you’re picturing a dumpster fire, you’re very close. Looking back, I see now what the problem was: I was entirely too focused on me and not focused enough on my students. The only things I thought of while writing the course were self-centered: my snobbish belief that the previous methods course hadn’t been rigorous enough, my need to prove myself to my colleagues, and my desire to alter the methods curriculum to make my mark. It was not my best teaching moment, and this showed up in my evaluations. While methods courses are known for having some of the worst evaluations for professors (Ryan et al. 2014), and I was well aware that the course had gone poorly, it was still very hard to read the students’ perceptions. I knew then that I would be rewriting the class from the ground up again.

Learning What Works While this was a very tough experience, it did grant me one gift: I learned pretty quickly that my approach to my job needed to change. First, I needed to let go of the haughty R1-methods mindset that I had been trained in and shift to a teaching institution mindset. To be 100% clear, professors at teaching institutions, community colleges, and basically anywhere that is not considered R1 are still very much scholars

360

W. R. MANZO

who produce highly technical, valuable work. However, they also usually have explicit requirements for tenure and promotion regarding teaching, and building and delivering strong college courses takes a significant amount of time. This is made doubly hard by the fact that most graduate programs, which are located at R1 schools, do not actually teach their students how to teach (Manzo and Mitchell 2018). In my case, I clearly needed to spend time researching the appropriate construction of a methods course at my kind of institution, rather than simply copying and pasting from my grad school curriculum, and practicing different kinds of teaching methods. It felt wrong for me to be giving up “research time,” but in reality, prepping and building good classes is also research—especially at a teaching institution like mine. Second, I needed to switch from a self-centered focus to a studentcentered focus. In my dumpster fire semester, I had been trying to do what my graduate school professors had done: reproduce methodologically advanced social science PhDs. However, that was far from who my students were—I was teaching at a small liberal arts school, where few go on to a PhD program. My students were similar to political science majors at most schools throughout the country—they either intended to go to law school, where research methodology and statistics are often viewed as “sociological gobbledygook” (Rocco 2017), or they wanted to work in government doing policy. So, I needed to begin listening to my students and what their goals were, for both this particular class and in general, so I could better tailor what I was teaching to what they wanted and needed to learn. As a side note, listening to students is both wonderful and very difficult. This is because there are two ways to hear them, and both have flaws. First, you could directly ask students what they want, or what they think about your class. This method is good because you can ask clarifying follow up questions, but this method is bad because you have to doubt the sincerity of everything they say, since there is a huge power differential in play. The second method, anonymous student evaluations of teaching (SETs), fixes the problems involved with students fearing being honest, but we also know that SETs are rife with well-documented bias problems (Holman et al. 2019). So, you are more likely to get real impressions of a class, but also much more likely to be exposed to abuse. I usually blend these methods: I ask in class how things are going several times a semester, and I carefully read the comments section of my SETs. Sometimes I’ll even pull aside a student who has taken another of

30

EXCEL, IN MORE WAYS THAN ONE

361

my classes and ask them to honestly tell me how the current class is going. I explicitly say in each of these methods that I want serious, constructive feedback, and give an example of something I’ve changed as a result of good feedback to show that I really will listen and take suggestions to heart. Listening to my students has led to pretty big changes in the methods course over time, but each successive year I only made small changes. This is because smaller changes are easier to implement, and also because then I could test the impact of each on its own. In this practice I was heavily inspired by Lang’s book, Small Teaching. For those of you who would like to avoid my growing pains and skip right to where you are a good methods teacher, consider the following recommendations: 1. First, determine your audience. Do you teach at a prestigious R1, a community college, or somewhere in between? The topics you focus on should reflect what your students need and want to learn for their future. Let’s be honest—the only students who need to learn regression are those headed for a social science PhD program, and there is simply not a large enough group at my institution to justify its inclusion in the course, so I cut it. This means that I teach statistics up to hypothesis testing. However, if you know that your class will include future researchers, you may want to go further. 2. Next, think about ways to build community in your class. Methods can be hard for undergraduates—especially women, who have the aforementioned STEM fears in their heads—so creating assignments that encourage working together can help students feel less overwhelmed and alone in their struggles. For example, when I started teaching the course students could choose to research whatever question they wanted, with the only limitation that it be political (and, as we know, everything is political). This led to wildly different projects, and it became hard for one person (me) to oversee them all. I decided to switch to a seminar-style format, where the students vote on a theme and then everyone chooses a question within the theme. Previous themes have included women and politics, inequality, and race/ethnicity politics, so the students still have quite a bit of latitude to choose a topic interesting to them. After the theme is chosen, I provide 3–4 starter readings to help them get started. This way, students can talk to each other about their

362

W. R. MANZO

projects and collaborate in ways that were impossible when everyone was pursuing their own random question. 3. Lastly, think carefully about which statistical software program your students should learn. I was trained on Stata, but no one outside of academia really uses it, and it’s expensive, so I knew this was inappropriate for my students. I thought about using R, but at that time, I hadn’t ever used it myself and I didn’t really want to spend time learning a whole new programming language while I was trying to earn tenure. (I have since learned R, but I definitely think my students would hate it.) My institution has a site license for SPSS, so I tried that, but my students found it complicated. After forcing SPSS for two years, I realized that, since I had cut regression, everything we needed to do in SPSS could be accomplished in Excel.

Conclusion: Excel is the Most Important Thing When I moved from a self-centered mode to a student-centered mode and increased my listening, I naturally opened up more to listening to other groups on campus as well. I heard my older colleagues talk about how hard it was to learn Excel for grading, and how they wished they’d started learning earlier. I heard internship supervisors say that my students were excellent communicators and great time managers, but were unable to complete basic data entry tasks. I had also heard my students say that SPSS was too hard since they didn’t even know how to use Excel, so this got me thinking—was I making things overly complicated? So, in my third iteration of the course, I decided to only use Excel to teach statistics. Excel comes with basic descriptive stats equations, and if you add the Data Analysis add-in it can do even more. I use Excel for all my gradebook calculations, and I even use Excel for basic running of results for the Meredith Poll (an academic survey research outfit housed in my department), so I believed it was easy enough for them to learn but also useful in a variety of contexts, which would increase their likelihood of using it again. Additionally, our students love working on the Poll, and students in this methods class in particular help write questions for the Poll as part of our unit on survey research, so I thought they might like it if I showed them how I created poll results. It ended up more successful than I could have hoped. They especially loved coming up with hypotheses based on questions on the poll and then using Excel to answer them. We couldn’t jump straight there—I had to

30

EXCEL, IN MORE WAYS THAN ONE

363

teach them the basic functions first, like the command “countif”—but once they knew what to do, they became very excited to see who could be the first to determine which candidate was the preferred candidate for older voters, or how women differed from men on believing the country is on the right track. After they got that down, I then taught them how to turn those results into charts and graphs, and talked about the importance of being able to easily and clearly communicate findings. Along the way, I made sure to point out that many real-world workplaces use Excel, and being able to use data to answer questions and present those answers in a visually appealing manner was a useful skill in a variety of jobs. That semester ended up earning me the highest SETs I had ever gotten in the methods course. Students nearly universally reported that they enjoyed learning Excel and had no idea that statistics could be so easy or fun. I was, of course, very pleased; it’s always nice to hear that students got something out of your class, and good SETs are always a welcome addition to the tenure file. However, I was also a little frustrated that it took me so long to land on Excel. I had thought that I’d given up the R1-methods mindset in my second year, but I guess it took me a little longer to lose the stats package snobbery in favor of the most widely used spreadsheet program in the country (Robarts 2014). In the end, I believe both me and my students have won. My students are gaining real transferable skills, applicable to many sectors, and hopefully even the future lawyers have learned the importance of quantitative social science research. For me, I can claim that I did what I set out to do—fundamentally changing the methods course and adding statistics to increase numerical literacy among our students—even if it looks differently than I first imagined. However, it wasn’t these changes that ultimately helped me earn tenure, I think; it was the better teacher and colleague that I became when I made listening the centerpiece of my approach to my work.

References Bridges, G.S., D.M. Gillmore, J.L. Pershing, and K.A. Bates. 1998. Teaching Quantitative Research Methods: A Quasi-Experimental Analysis. Teaching Sociology 26 (1): 14–28. Engelhard, G. 1990. Math Anxiety, Mother’s Education, and the Mathematics Performance of Adolescent Boys and Girls: Evidence from the United States

364

W. R. MANZO

and Thailand. Journal of Psychology 124 (3): 289–298. https://doi.org/10. 1080/00223980.1990.10543224. Gunderson, E.A., G. Ramirez, S.C. Levine, and S.L. Beilock. 2011. The Role of Parents and Teachers in the Development of Gender-Related Matt Attitudes. Sex Roles 66: 153–166. https://doi.org/10.1007/s11199-011-9996-2. Hargreaves, M., M. Homer, and B. Swinnerton. 2008. A Comparison of Performance and Attitudes in Mathematics Amongst the ‘Gifted’. Are Boys Better at Mathematics or Do They Just Think They Are? Assessment in Education: Principles, Policy, and Practice 15 (1): 19–38. https://doi.org/10.1080/096 95940701876037. Holman, M., E. Key, and R. Kreitzer. 2019. Evidence of Bias in Standard Evaluations of Teaching. Accessed August 1, 2020. http://www.rebeccakreitzer. com/bias/. Jafar, A. 2012. The R1 Bias. Inside Higher Ed, March 18. https://www.inside highered.com/blogs/university-venus/r1-bias. Lang, J.M. 2016. Small Teaching: Everyday Lessons from the Science of Learning. San Francisco, CA: Jossey-Bass. Manzo, W.R., and K.M.W. Mitchell. 2018. We Need to Rethink Training for Ph.Ds. Inside Higher Ed, September 11. https://www.insidehighered.com/ advice/2018/09/11/academic-training-phds-needs-focus-more-teaching-opi nion. Milligan, L., J. Rose, and R. Harris. 2014. Convincing Students? Quantitative Junkies, Avoiders, and Converts on a Cross-Disciplinary Course Using Quantitative Narratives. Enhancing Learning in the Social Sciences 6 (2): 59–73. https://doi.org/10.11120/elss.2014.00039. Robarts, S. 2014. Spreadsheet Software: Top 5 on the Market. Tech Radar, October 15. https://www.techradar.com/news/software/business-software/ spreadsheet-software-top-five-on-the-market-1257738. Rocco, P. 2017. Justice Roberts Said Political Science Is ‘Sociological Gobbledygook.’ Here’s Why He Said It, and Why He’s Mistaken. Washington Post, October 4. https://www.washingtonpost.com/news/monkey-cage/wp/ 2017/10/04/justice-roberts-said-political-science-is-sociological-gobbledyg ook-heres-why-he-said-it-and-why-hes-mistaken/. Ryan, M., C. Saunders, E. Rainsford, and E. Thompson. 2014. Improving Research Methods Teaching and Learning in Politics and International Relations: A ‘Reality Show’ Approach. Politics 34 (1): 85–97. https://doi.org/ 10.1111/1467-9256.12020. Tomasetto, C., F. Romana Alparone, and M. Cadinu. 2011. Girls’ Math Performance under Stereotype Threat: The Moderating Role of Mothers’ Gender Stereotypes. Developmental Psychology 47 (4): 943–949.

PART IV

Teaching Research Writing

CHAPTER 31

Research Articles, Not Research Papers: Empowering Students Through Research Writing William O’Brochta

In my first course as a teaching assistant, the instructor asked students to write a paper articulating a novel theoretical argument based on assigned readings on social justice. Reading through several rough drafts, I noticed that most students successfully summarized the assigned readings, but then proceeded to re-state one of the authors’ theoretical arguments. Out of curiosity, I asked a student why he had not developed his own theory. The student explained that he was unsure how to write a theoretical argument and that he lacked anything valuable to add to existing work. I realized that the student did not feel that he was part of the discipline, and that he had not been shown how his own ideas and experiences could lead to new, valuable, and relevant political science insights. Since then,

W. O’Brochta (B) Department of Political Science, Louisiana Tech University, Ruston, LA, USA e-mail: [email protected]

© The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_31

367

368

W. O’BROCHTA

I have begun to guide students in all my courses through the research article writing process. I have found that empowering students to work through the research article writing process as professional political scientists (1) is an effective way to make political science relevant, (2) teaches critically undersupplied writing skills applicable to many future careers, and (3) creates an inclusive and collaborative classroom climate. I begin by reflecting on several initial attempts at using research-based assignments to engage students. These piecemeal approaches led me to design my introductory and upper-level comparative politics courses around the research article writing process. This process involves separating a research article into its constituent components and devoting substantial time to helping students produce drafts of each component that they combine together into a completed research article by the end of the course. I especially highlight my use of research-based simulations—a collaborative policymaking activity based on student research—as a key part of making student research relevant and meaningful. Researchbased simulations are also an accessible way for instructors to test out teaching the research article writing process in their courses. Completing the research article writing process is more than turning in a research paper. By working through the article writing process, students operate in the same space as the authors of published journal articles that we read in class. Once students fully understand their new role as political scientists, they frequently exceed both my expectations and their own.

Initial Research-Based Assignments I initially assumed that junior and senior political science majors knew how to write literature reviews and theoretical arguments—what are typically termed “research papers.” This assumption was driven partly by the fact that these activities are so frequently assigned in upper-level undergraduate courses, and partly because I utilize these skills often. During my first time as a teaching assistant, I realized that students’ exposure to and practice with these concepts was limited, regardless of their grade level or experience in the major. My first attempt at a solution was to pick a research paper component and to build interesting substantive content to provide students with many opportunities to develop that component. That is, I felt that the traditional research paper model could be improved. I had the opportunity to play an active role in helping students develop a theoretically grounded research paper in the subsequent semester. The

31

RESEARCH ARTICLES, NOT RESEARCH PAPERS …

369

course—Political Protest and Violence—naturally lent itself to many interesting case studies wherein students could innovate on existing theoretical models using the case facts as a guide. Along with trying to spark students’ interest in the assignment early, the instructor and I set several intermediate deadlines (e.g., a thesis statement and a partial draft) in an attempt to encourage students to start working on the paper and to seek guidance if they were unsure how to develop a theoretical argument. We had some success: student interest in the final papers improved, but most theoretical arguments still failed to articulate a unique contribution. Around this time, I started thinking about what I could do to further scaffold this end-of-semester research paper assignment. By chance, I was working on revising a syllabus in order to co-teach a course titled Immigration, Identity, and the Internet. One of the chief goals of the course was to develop student writing ability, and the main pedagogical tool used was repetition and instructor review. Students drafted a literature review and had a one-on-one tutorial meeting with the instructor before turning in a final draft, repeating the process several times throughout the semester. I thought that the draft and repetition idea could be effective, but I wondered if scaffolding a literature review paper before a theoretical argument paper could help make the literature review more relevant and improve the structure of and innovation in the theoretical paper. In the revised syllabus, we encouraged, but did not require, students to link their literature review to their theory, and we added peer review to the existing instructor tutorials. We faced a challenge before students even submitted their final literature reviews. Based on my experience from previous semesters, I was prepared to work with students’ literature reviews to develop the subsequent theory papers. What I did not expect was that the students, mostly senior political science majors, did not know how to write a literature review. Students felt that they knew how to complete the assignment and said so ahead of time, but their literature reviews were almost entirely disconnected summaries of various readings. It was during this experience that I concluded that students would be best served by integrating the entire research article writing process into as many courses as possible including, but not limited to, introductory courses. Repeatedly assigning a literature review or theory research paper in many different courses does little to provide students with the space to build and reinforce their writing skills if instructors do not teach students how political scientists complete this task.

370

W. O’BROCHTA

Research Article Framework My initial rationale for adopting research article writing assignments in all of my introductory and substantive courses was the simple motivation that students needed practice and instruction in order to successfully complete literature reviews and theory papers. I took all of the weight previously assigned to exams and other papers and put it into a series of scaffolded research article assignments (about 50% of the course points). In any given course, students are free to choose any research question of interest within the bounds of the course topic. The course is split into units, where students work through the research question; annotated bibliography and literature review; theoretical argument; research design; and introduction, abstract, and conclusion sequentially (see Fig. 31.1). In each unit, students read journal articles that relate to a relevant substantive topic and to the article component we are working on. Students write a reading journal before class that reflects on both substantive and article writing content, and class time is evenly split between these two topics. For example, when reading about electoral systems and working on the theory

Research Question

…

Literature Review and Annotated Bibliography

Assigned Article Reading Journal Substantive and Article Writing Classes Draft Article Workshop Peer Review Evaluate Article Component

Theory Research-Based Simulation Research Design Full Research Article (with Introduction, Abstract, and Conclusion) and In-Class Presentation

Fig. 31.1 Research article writing course structure

31

RESEARCH ARTICLES, NOT RESEARCH PAPERS …

371

part of their article, students critically analyze the hypotheses authors presented in substantive journal articles on electoral systems. Toward the end of every unit, students submit drafts of their research article, adding the new components and revising what they have already written. This is followed by peer review and instructor evaluation. Though I did not fully realize it at the time, guiding students through the research article writing process is akin to inviting them to become political scientists. Political scientists work through the research article writing process; they do not write standalone literature reviews or theoretical arguments. Hence, the writing practice inherent in research article writing is a side benefit to empowering students to become part of the discipline. From this perspective, it became clear that the best course of action was to integrate other “things political scientists do” into my courses. Being a political scientist means sharing ideas, collaborating, making public policy connections, working with the community, and developing writing and critical thinking skills. Though most students will not have careers as political scientists, critical thinking, writing, and collaboration are all widely sought-after skills. Working through the research article writing process is the most natural vehicle for integrating these aspects into a course because students are participating in generating knowledge in the same format as the published journal articles that they read. Political scientists use their research articles as building blocks for other important components of their work. I seek to expose students to this process in class. Peer reviews and short presentations provide opportunities for students to learn from and teach one another and to reinforce course concepts. Simulations mimic real-world applied work that naturally extends from student research and can help students see the ways in which their work relates to policy. Since students are completing the entire research article process, their conclusions often speak to local community issues, which makes it easy to integrate course components on community engagement. Finally, student ownership of their research article and the responsibility to teach others about it promotes diversity, equity, and inclusion in the classroom. Students develop a common identity as political scientists wherein they learn that they need to rely on classmates for help throughout the research article writing process. Because most students have not written a research article before, students begin the process

372

W. O’BROCHTA

from the same starting point, regardless of their major or life experiences. As such, research article writing has had a positive impact on my classroom climate, with students leveraging their colleagues’ diversity and perspectives in a serious and meaningful way. Asking students to work through the research article writing process prompted a shift in the way I think about educating students beyond the actual assignment. To me, there is no reason why the actual work political scientists do should be relegated to senior capstone experiences or to individual research with instructors. These opportunities are inequitably provided to selected majors late in their careers, while other students take political science courses lacking these learning outcomes. The components of research article writing are not particularly complex, and they can be taught. In order to do so, research article writing requires investment on the part of the instructor and a change in the power dynamic between the instructor and students. The exciting part about working with students through the article writing process is that they raise new questions and problems all of the time. I have found that embracing these new questions and problems is the most productive way forward. Students are often best equipped to lead the discussion of their research articles, with the instructor serving in an advising role. More practically, focusing on research article writing presents numerous challenges. Course content must be pared down to make room for writing instruction. Summative assessments take creativity to implement effectively. Course enrollment matters for how much individual attention instructors can provide to each student. The overall model is flexible to adapt to these challenges. For example, instructors could scaffold the article writing assignment across courses to have students complete an article draft minus empirical analysis in introductory courses, to hone in on research design and quantitative analysis in upper-level courses, and to create publication ready work in senior capstones.

The Ebb and Flow of the Article Writing Process When incorporating the research article writing process into a course, I have found it very important to keep up with and manage student expectations. Students react to different parts of the research article writing process in similar ways to career political scientists. However, students are typically writing a research article for the first time, so they do not know if

31

RESEARCH ARTICLES, NOT RESEARCH PAPERS …

373

the difficulties they are facing or their lack of confidence at certain points is normal. Part of inviting students to become political scientists showing them that the published journal articles they read started out as rough drafts just like the article drafts they are writing for the course. Each section of the research article requires different forms of expectations management. Initial student excitement and interest in the research writing process is generally enough to last through the research question stage. As long as I encourage students to come up with questions that they are interested in and questions based on some important public policy problem, students usually have many ideas. Refining initial ideas and settling on a final research question takes some extra time for students with limited experience reading political science research. However, most initial research ideas can be turned into workable research questions without too much difficulty. Literature reviews are generally where students get discouraged, regardless of prior experience. Conducting a literature review almost always reveals one of two situations: either much research has already been done on the students’ research question or there is essentially no existing literature. In the former case, it is critical to help the student return to her initial research question and policy motivation to try to identify at least one small difference between existing work and the research question. Since I find that students usually become interested in research questions because of actual public policy problems, there is almost always some nuance to a real-world case that is missing from existing literature. When students find this nuance, they are set to write an interesting theory. Students are creative thinkers, and they are bound to generate research ideas that speak to little existing literature in political science. Many of these ideas pertain to research areas that the discipline has long marginalized like identity-based issues or political history and sociology. Other research questions are too new to appear in political science journals. In these cases, I advocate helping students find alternative sources—including those from other disciplines—instead of telling them to generate a new research idea. If the research question and the contribution that students develop is interesting to them and is relevant to public policy, I do not let the lack of perceived importance in the discipline stand in students’ way. Most theoretical arguments can be successfully constructed if these research question and literature review issues are worked out.

374

W. O’BROCHTA

The next hurdle is the research design, and student motivations and feelings at this point tend to differ depending on whether they must conduct the empirical analysis or not. If the empirical analysis is required, students may propose conservative research designs that they feel that they can implement, but that only partially test their theory. On the other hand, if students do not conduct the empirical analysis, then they may propose an ambitious design and be unsatisfied that they never get to analyze results to bring closure to their article. In both cases, I advocate for having students develop primary and secondary research designs, with one design using conveniently available data and the other relying on some causal identification strategy where new data collection is required. Students analyzing data for a results section can focus on the available data for their primary research design so that the empirical analysis is achievable, while those not analyzing empirical data can highlight the best possible causally identified design. The advantage of asking students to include both a primary and a secondary design is that describing available data is more familiar, whereas thinking about the best possible research design is more exciting and informative. I find the last part of the article writing process—putting together an introduction, abstract, and conclusion—essential, as it allows students to walk away from the course with a fully finished research article. Conclusions also provide a great place to integrate policy implications from the research-based simulation’s policy briefing paper discussed below. Adding these relatively straightforward components that we work on a lot during class time also spaces out assignments in the course so that they land at different times than traditional midterm and final exams in other courses. Research article writing takes significant time and effort, and I find it worthwhile to devote the last several classes of the term to students polishing their articles, presenting them, and celebrating their achievements.

Research-Based Simulations: An Accessible Pedagogy My main piece of advice for instructors interested in structuring their courses around the research article writing process is to treat students as political scientists and to let this idea guide course design decisions. I fully realize that a shift to research article writing is a big step and that interested instructors may want to try out part of this pedagogy before fully

31

RESEARCH ARTICLES, NOT RESEARCH PAPERS …

375

adopting it. At the same time, I have already noted that just assigning students to write one part of a research article is unlikely to produce desirable results. Research-based simulations act as an accessible way to quickly bring in some of the core ideas of the research article writing process without needing to re-design an entire course or curriculum. The assignments that I call “research-based simulations” have three core components: an individual research-based task that students complete before the simulation, a small group public policy component, and a class long legislative simulation exercise. Preparing for and completing the simulation encapsulates most of the learning goals inherent in the research article writing process, albeit in a much shorter time frame. In my courses, the simulation takes place right after students have developed their research questions, literature reviews, and theoretical arguments Around this time, students tend to begin questioning how the theory that they developed applies to practical situations. Students take their theory and hypotheses and use them as evidence to write a policy briefing paper. The idea is to help students to think of their research article as providing a potential solution to an important public policy problem and, likewise, to describe how a public policy problem can be solved based on the theoretical argument that they developed. In order to adapt this assignment to a context where the simulation is not part of a broader research article writing effort, instructors can rely more heavily on the research that goes into the policy briefing paper. In my experience, students working through the research article writing process have already developed basically all of the briefing paper: their research question is based on some real public policy problem, their literature review identifies potential solutions, and their theoretical argument provides guidance on the solution most likely to work. For instructors employing a research-based simulation in a course without the emphasis on the research article writing process, I suggest investing more time in guiding students through each part of the briefing paper, providing essentially a mini lesson on asking research questions, referring to literature, and developing hypotheses. With the briefing paper done, the other two components can proceed in the same manner regardless of whether a course is structured around research article writing or not. Students form small groups based on the topics of their briefing papers and work together to synthesize each other’s proposed policy solutions into one coherent policy proposal. In upper-level courses, I ask students to work together in their group to

376

W. O’BROCHTA

collect quantitative and qualitative data from their community that reinforces the group’s policy proposal. The group work step is critical for instructors using this simulation as a standalone assignment because it provides some of the collaborative learning environment that students benefit from during the research article writing process. It also underlines the importance of community engagement in political science and diversifies student perspectives. Students bring both their individual and group work to class for a legislative policymaking simulation. The exact parameters of the decisionmaking body can be influenced by the content of the course. As a comparative politics instructor, I tend to ask students to select country (by vote) to use as an example at the beginning of the simulation and to have them adapt their policy proposals to the political and legislative environment in that country during the first few minutes of the simulation. My rules for the simulation are simple: groups have to try to stick to their original policy proposals and some policy must be passed by the legislature before the end of the simulation. Though the mechanics of the simulation can be made more complex if desired, the basic point is that students must work to convince others about the merit of their policy proposal while simultaneously bargaining with other groups who have competing interests. The pre-simulation briefing paper and group assignment are critical for the simulation’s success. These components help to make the activity more equitable for those with less public speaking experience or background knowledge and to ensure that the simulation is grounded in theories and facts, not rhetoric. Students usually pass some policy proposal during the simulation, and the debrief following the simulation tends to be filled with a combination of excitement about the simulation and recognition of the difficulty of policymaking and how often well thought out policy proposals are changed during policymaking for the sake of compromise. The energy resulting from this simulation helps me propel my students through the last half of the research article writing process. This pedagogy also encapsulates many of the ideals that political science tries to espouse: careful research, policy relevance, and collaboration with peers. By emphasizing and expanding on each step of the research-based simulation, students can quickly develop many of the skills that are part of the research article writing process.

31

RESEARCH ARTICLES, NOT RESEARCH PAPERS …

377

Conclusion Exposure to and experience with the components of the political science research article writing process are critical for students’ success in senior capstone and research methods courses and are some of the parts of political science that most easily transfer to post-college careers. As an instructor, I initially underestimated my students’ ability to successfully work through the entire research article writing process. With the right environment and motivation, students can successfully transition to becoming political scientists. Students’ careers as political scientists may be relatively short—often relegated to one course in one semester. Yet even small doses of political science training can prove valuable. I recently taught an Introduction to Comparative Politics course with a mix of junior and senior non-majors and younger students who were potential majors. One of the younger students, who had decided on a career as a lawyer, said that he was actually interested in American politics, but took my course just to get some major requirements completed. Although his research project was about voter fraud in Uganda, he told me some time after the course finished that the process of writing a research article convinced him that he was interested in pursuing a social science major. He felt that he had a big head start on coursework and research more closely aligned with his interests because he learned how to be a political scientist in that course. Another student, a natural science major, took the same course as her final requirement before graduating and pursuing a career in medicine. She was quite adept at writing research procedures in the natural sciences, but the critical thinking skills and different perspectives in social science research were immediately useful for her as she embarked on her career. Though neither of these students intended on further developing their research articles, both students remarked that the research article writing process was the key defining aspect of the course and provided major benefits for their future careers. More than any other assignment, I have found that students are empowered when working through the research article writing process. For students, reading published journal articles and having the confidence that they, too, have gone through this process shifts their perceptions about the discipline, increasing their interest, and improving the classroom climate. Whereas students tend to immediately see the applicability of creating an infographic or slide presentation, teaching the research

378

W. O’BROCHTA

article writing process requires significant investment from the instructor to explain why the pedagogy is relevant and meaningful. However, once students are able to move past their initial trepidation, the impact of this pedagogy is far-reaching.

CHAPTER 32

Integrating Research Writing and Research Methods: Toward a More Seamless Curriculum Martin S. Edwards

When I was a junior in college, I received the following comment on a paper that I submitted: “Your writing style needs considerable work. You write as if you are writing for the New York Times.” We know that direct feedback on writing feels more personal and cuts deeper than more mundane comments about run-on sentences or passive voice. (And we all have reactions when reading comments from reviewers, so it is not as if this process of dealing with criticism ever really ends.) In my case, I never asked the professor for follow up, as I really did not know where to start. It was not clear to me how to address the first part of the comment: What does “considerable work” mean in practice? What would that look like? Would I need to take high school AP English over again? More

M. S. Edwards (B) School of Diplomacy and International Relations, Seton Hall University, South Orange, NJ, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_32

379

380

M. S. EDWARDS

pragmatically, there was an immediacy that I could not address: How do I get that help? The second sentence was even more mysterious: How could this comment be critical? Why is writing for one of the nation’s leading newspapers a bad thing? It is precisely because comments such as these generate emotional reactions in students (and indeed, are not that helpful) that we try to avoid making them in comments on written work. The fact that I can still recall this comment suggests that it was not at all helpful. And yet, in defense of the professor who wrote it, there is a deeper rationale for starting with this vignette. This comment has certainly shaped how I give comments on written work, but it has also shaped how I infuse the teaching of writing into the teaching of research methods. As someone who routinely teaches our capstone courses, in which students use the tools that they have learned in our Research Methods classes to develop and write a piece of social science research, I have learned that I am still teaching writing even as I am teaching issues regarding statistical techniques or how to use Mill’s Method of Difference. In this chapter, I will discuss how I have tried to fuse research writing and research methods to create a more seamless curriculum. The goal is that students get the attention that they need on both the writing side and the methodology side, with a mind to keeping the feedback both substantive and substantial. It is worth stressing that this is not easy. Reading student capstone papers with an eye to improving both the content of the research and the writing can be challenging. Imparting feedback in these cases, as noted in the vignette above, can prove daunting. Professors want to be able to cover both pieces, but the challenge is to do in a way that addresses these different problems effectively. As a reader, I need to keep the two halves separate to avoid getting frustrated. As a professor, I also need to avoid overloading students with comments so that they are paralyzed. Below, I set the stage by outlining the curriculum in my program, turn to my pedagogical efforts to knit together what can seem to students to be two very disparate courses, and close with teaching writing for students after the course is completed. The goal of this last step is to give students a deliverable as their writing goes forward just in time for them to go on the job market. I should state at the outset that my program is a professional school of international affairs, and so this accounts for the pragmatic bent of the advice in the pages below.

32

INTEGRATING RESEARCH WRITING AND RESEARCH METHODS …

381

The Challenge of Creating a Seamless Curriculum I teach in an international affairs program at a comprehensive university, and my degree is in Political Science.1 I was hired as a “Swiss army knife” candidate capable of covering classes in International Organization and International Political Economy, and I have always assigned empirical papers in my undergraduate and graduate classes. It had never been my intention to teach in our research methods sequence, but I was assigned our graduate capstone following a petition signed by over 20 students. I built that class around the idea of applied work submitted sequentially, as students submitted a brief problem statement, a presentation, a detailed research design, and then a rough draft and a finished manuscript. This class relies heavily on frequent office hours and less on formal meetings. The use of a professor-faculty model fits my own personal style. While a different approach might be more group work-based, I wanted to give students additional opportunities for mentorship and avoid the collective action problems that are often inherent in group work. The once per week class might only meet a few times in the course of the semester, and I’ve added extra meetings as needed to address cross-cutting issues in student work. To make scheduling easy, I use the conventional class time as additional office hours to meet with students. A word about our curriculum is in order, as it helps to underscore the challenge of making the curriculum seamless. At both the graduate and undergraduate levels, we offer a one-term research methods class and a one-term research project class. The methods class is taught either as a survey of different types of methodologies with a midterm and a final exam or as a more focused social science research class with homework assignments and a research design as the final assignment. This has important consequences. Students might enter the capstone with a detailed roadmap to a high-quality paper, or others might still be in need of creating that roadmap. With a two-course sequence, student skillsets are an exogenous “given” that you have to work with as a starting point. The follow-up capstone class, in contrast, is more homogenous. Different faculty place different emphasis on how the work gets done. Some rely more on group work and others rely more on one-on-one work with the faculty member. Regardless, the goal of the capstone is to produce a high-quality paper based on empirical research, either qualitative or quantitative. As the final paper from this class forms the basis for our assessment of program learning objectives at the graduate and

382

M. S. EDWARDS

undergraduate levels, it has importance well beyond the conventional assignment for students. The compressed schedule of a two-course sequence has important consequences. It means that the focus on research design basics results in more attention to quantitative than qualitative work. And it also means that the first semester class is basically a technique class. As a result, professors lack the class time to reinforce writing skills. This compression is mirrored in the textbooks that we assign for research methods courses, which might have an example of a research article as a go-by but not much on writing per se.2 This means that professors that want to cover this will have to supplement with their own resources.3 This means that the students that take the second course are frequently a heterogenous group both in terms of their research design skills and their writing skills. Separate from the issue of time constraints, with a one-term research methods class, we are asking students to master a lot of material in a way that might be very alien to them. This means that students who take the follow-up class can have different challenges. Students might be unclear to varying degrees about their writing or about their research question. Creating a seamless curriculum means addressing the concerns of different student groups. Some might be clear about their research question, but have questions or issues with the methodology. Others might have a clear question and a clear grasp of the methodology needed to answer the question. The sequential approach that underpins the assignments, moving from lower to higher stakes, helps address some of these challenges. But the challenge of teaching writing to students in this class, albeit to differing degrees, still exists. In the next section, I discuss my strategies for teaching writing to these diverse student groups.

Different Challenges, Different Strategies Over time, I have increasingly realized that the gains from a more seamless curriculum are substantial. Not realizing the connections between these two challenges has had the consequence of not addressing student writing terribly well, which does not ideally prepare students for the challenge of the job market. Developing ways to address the challenges of both groups has been a very recent focus, and I hope to better assess the impact of my approach as outlined below.

32

INTEGRATING RESEARCH WRITING AND RESEARCH METHODS …

383

As I have noted above, there are students with different challenges in a capstone course such as this. Below is a simple two-by-two table with a typology and a recommended teaching strategy: Writing problems? Research design problems?

Yes

Yes

Group A: Triage and Focus Group C: Focus and Supplemental coaching Group B: Triage and Group D: Support Supplemental coaching

No

No

I am thinking of triage here in the medical sense, referring students to different resources on the basis of severity. I will focus a lot on students with challenges, but even if a student has unproblematic writing, there is still a lot about the form of a social science research paper that we want to teach them. Below, I outline how I handle each student group with a mind to giving everyone the resources that they need to be successful. Group A: Triage and Focus Unfortunately, not every student shows up ready to prepare a publishablequality work. Here the challenge is to be precise on exactly what the problem is, offer concrete action steps, and deliver the medicine in a manner in which the patient will want to take it. I rely on past practice here and give prompt referrals to our campus writing center for structural problems that require one-on-one attention: grammar, run-on sentences, passive voice, paragraphing, etc. I ask the student to bring a past assignment as well as the current assignment so that the tutor has a better sense of the scope of the problem. After all, the more that I equip the tutor to do their job, the better positioned they will be to help the student. Thinking about how best to deliver this referral is also important, given that I want to avoid the experience at the opening of this essay. The way that I do this is to let the student know in advance that this is coming. This is done in an email sent as soon as I can before the papers are returned. Decoupling this from the grade helps to make sure that the message gets through. Using an email rather than a returned grade to do this allows me to be a little more colloquial, and I reference that I have done this with

384

M. S. EDWARDS

other students in the past, and that they really made improvements as a result. I also pitch this as providing them extra help (as more of a positive) than telling them that they need a tutor (which can certainly be seen as a negative). When there is real resistance, I have referenced the story at the outset of this chapter to personalize this advice. When I return the assignment, I keep the focus on the research design challenges. Here the danger is overloading the student so that she does not know how to get started. I include to-do lists in the summary, which allows the student to form a clear action plan going forward. This feedback is often typed or scanned, which gives me a permanent record. This approach has two important benefits: It means that I can keep the focus of my comments on the content as it relates to the class. This improves the quality of my feedback for the student, and it ensures a uniformity of treatment that is important. It also serves to discipline me: By tuning out the noise of writing mistakes as I grade, it allows me to best offer substantive feedback on the research component of the work to strengthen that and help the student moving forward. The critical thing for students in this group is focus. If there is any group of students that is in danger of falling behind in the class, it is the students in this group. I will put these students on an accelerated schedule of follow-ups with me so that we address both the writing and the research design challenges. Having the detailed comments to fall back as a written record for us both reinforces accountability and makes planning for these one-on-one meetings easy. Group B: Triage and Support For those students who have writing challenges but have the mechanics of the research design in a good place, I follow an approach based on the above, but with less intensity. I will refer students to our writing center for follow-up work, delivering the news is the same fashion as above. For these students, the one-on-one follow-ups focus more on how they are seeing challenges in their writing and making improvements, and then I use that opportunity to make sure that the message of the referral to the Writing Center was delivered in a helpful fashion. Meetings with this group of students also have value as an evaluation of the writing center’s services. Pointing out the strengths of the work and how issues in writing can detract from the impact serves to go a long way to mollify student concerns.

32

INTEGRATING RESEARCH WRITING AND RESEARCH METHODS …

385

Group C: Focus and Supplemental Coaching Even if a student’s writing is unproblematic, research design problems can make success impossible. For these students, the approach really is to insist on more follow-up meetings. The use of detailed comments in the form of a to-do list helps students to stay on track. The sequential nature of a research methods class means that small problems can become large ones quickly. Students with issues with case selection or variable operationalization might also have difficulties interpreting findings. Follow-up meetings with students in this group help to keep them on track. It is worth stressing that while much of the focus has been on addressing student areas of difficulty, none of this is to suggest that I do not have additional writing skills to offer exceptional students. Group D: The Value of Supplemental Coaching Some students might have a strong research question and be adept at writing about how they gathered data and interpret their findings. This is obviously welcome. But even in these cases, these students still have a lot to learn about research writing as well, and my work with these students falls into three broad categories: articulating importance, building a literature review, and discussing the broader implications of their findings. All three of these issues are often omitted in a research methods class because of time constraints.4 I discuss how I address each below. We all agree that it is important to sell the importance of one’s scholarship, and this takes on extra importance in policy school. However, students are often ill-equipped to talk about why their research questions are important ones. I use go-bys as examples here.5 In their own work, I ask students to attack this section using an outline. The specific assignment here is for them to delineate three reasons why the research question is important for scholarly research and three reasons why the research question is important for specific policymakers. Framing importance in terms of scholarly research rather than theoretical debates helps students to push past narrow theoretical or paradigmatic boxes, which can be a challenge in international relations scholarship. At the same time, the policy importance half makes sense given that I teach in an International Affairs program. This encourages students to think about policymaking and policymakers more broadly, since the findings might be more germane to the NGO community than the intelligence community.

386

M. S. EDWARDS

This has broader value to help them to think about outreach for their work when completed. With literature reviews, I stress the importance of organization and finding the right things to put in the literature review first. Often students become paralyzed as they end up with a stack of PDFs and are unsure what to do next. I start with reminding them about the value of the literature review in providing useful information to help resolve all of the modeling choices in the research design (from variable operationalization to case selection to choice of methodology). I then recommend two tools to help them build the literature review: We discuss annotated bibliographies to help students better summarize papers, and we discuss how a table can be useful organizationally to talk about connections across papers.6 Finally, once we have findings and once we have talked about them, we need to make those broader connections to close the paper out. In this case, I ask students to reconnect the findings back to the importance section. After all, the first time they wrote the importance section, they did not have findings, and the paper was written with a logic of inquiry. Now they have findings, and those are worth discussing. To write the paper with a logic of presentation, I ask the students to take what they have learned and reconnect the findings to the importance section. This means that they detail how the findings help advance scholarship more generally and how the findings help policymakers to make better decisions. As this suggests, there is a lot of specific coaching aimed at taking the writing of good students and making it even better. This feedback is best delivered one-on-one and tailored to each student’s individual project. But as noted above, I have a number of examples and resources that I refer to that are housed in our learning management system, including go-bys from successful papers from previous semesters.

Carrying the Writing Forward The danger with these capstone classes is that students consider them one-and-done. While a common reaction for students in their last semester, not thinking about writing after the assignment is submitted is a missed opportunity for additional professional development, especially at the graduate level. I close the class with additional resources for thinking

32

INTEGRATING RESEARCH WRITING AND RESEARCH METHODS …

387

about academic publishing and op-ed writing. This helps students to best leverage their work in ways that will help their careers. So, for the last class I take a bit of time to exhort them to do something with what they have written. In so doing, I share examples of students who have published their research.7 Having worked with the student on the revise and resubmit memo gives me a bit of an insider perspective on how this works for students. I also talk about the experiences of other students who distilled their larger paper into an op-ed.8 In some semesters, I have made the op-ed a course assignment, though this depends on the skillsets of the students that I have to work with coming into the class. Regardless, I cover how op-ed writing is stylistically distinct from conventional academic paper writing. Having examples really helps to make this last week of class come alive.

Conclusions Our challenge as educators is to meet students where they are. Students bring very heterogeneous skillsets to this capstone class, and it is important that we think about teaching as a series of different strategies designed to both diagnose the problem and also address it. For many students, social science research does not come naturally, and it takes reinforcement to make things click. For others, writing remains a challenge. The typology noted above and the preceding discussion of teaching strategies is designed to better empower professors so that they can meet the challenge of these courses head-on and make the curriculum more seamless in the process.

Notes 1. Hendrickson et al. (2011) have a good overview on what this means in practice. 2. Rich et al. (2018) and Johnson et al. (2016) both have good examples of annotated papers. 3. Baglione (2020) and Powner (2014) have good general sections on writing. 4. It is important to note that they are not omitted from textbooks. Rich et al. (2018) and Johnson et al. (2016) are good examples with strong sections on literature review writing. 5. I use a classic example (Walt 1988), and one of my own (Edwards 2019) that was inspired by teaching this class. 6. A good example is Steinwand and Stone (2008).

388

M. S. EDWARDS

7. Terry (2017). 8. Stone (2014), Shahi (2020), and Ramos Miranda (2021).

References Baglione, L.A. 2020. Writing a Research Paper in Political Science: A Practical Guide to Inquiry, Structure, and Methods, 4th ed. Washington, DC: CQ/Sage. Edwards, M.S. 2019. The IMF, the WTO, and the Politics of Economic Surveillance. New York: Routledge. Hendrickson, R., M.A. Mueller, and J.R. Strand. 2011. Political Science Careers at Comprehensive Universities: Building Balanced Careers at “Greedy” Institutions. PS: Political Science & Politics 44: 129–134. Johnson, J.B., H.T. Reynolds, and J. Mycoff. 2016. Political Science Research Methods, 8th ed. Washington, DC: CQ/Sage. Powner, L.C. 2014. Empirical Research and Writing: A Political Science Student’s Practical Guide. Washington, DC: CQ/Sage. Ramos Miranda, C. 2021. Sexual violence against males gains interest as an international security threat. Open Global Rights. https://www.opengloba lrights.org/sexual-violence-againstmales-gains-interest-as-an-international-sec urity-threat/?lang=English. Rich, R.C., C.L. Brians, J.B. Manheim, and L. Wilnat. 2018. Empirical Political Analysis: International Edition. London: Taylor and Francis. Shahi, P. 2020. How India’s Elite Competition Poisoned the Internet. The Diplomat. https://thediplomat.com/2020/05/how-indias-elite-compet ition-poisoned-the-internet/. Steinwand, M.C., and R.W. Stone. 2008. The International Monetary Fund: A Review of the Recent Evidence. Review of International Organizations 3: 123–149. Stone, L. 2014. Can Women Make the World More Peaceful? The Guardian. https://www.theguardian.com/global-development-professionalsnetwork/2014/aug/11/women-conflict-peace-society. Terry, D. 2017. Politicized Ethnicity and Income Inequality. Studies in Ethnicity and Nationalism 17: 68–90. Walt, S.M. 1988. Testing Theories of Alliance Formation: The Case of Southwest Asia. International Organization 42: 275–316.

CHAPTER 33

Empowering Students by Teaching Research-Paper Writing as a Foundational Methods Course Lisa A. Baglione

Over the course of my career, I have become convinced that combining the teaching of research-paper writing and methodology is a great, applied way to impart skills and useful habits to political science and international relations (IR) majors. This combination helps students see linkages between their courses, provides opportunities to pursue independent research, develops stronger relationships with faculty and peers, and establishes positive practices and attitudes. These four characteristics are ones that researchers designate as “high-impact educational practices” because they transform the students as learners and adults, as well as provide them with hard and soft skills that transfer to their future endeavors (HighImpact Educational Practices n.d.). Thus, when done well, teaching

L. A. Baglione (B) Department of Political Science, Saint Joseph’s University, Philadelphia, PA, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_33

389

390

L. A. BAGLIONE

methodology through research-paper writing is an empowering process for all students who take the experience seriously.

Becoming a Convert to Teaching Methods Through Research-Paper Writing The value of research-paper writing became clear as I developed as a teacher. As an advanced graduate student, I joined the team at Cornellin-Washington (CIW), a program for Cornell University’s government and other majors who wanted to intern in Washington while earning a semester’s worth of credit. A central component of the program, which distinguished it from others available at the time, was that students had to write a “real” public-policy thesis related to their internships. What made the thesis “real” was that undergraduates had to ground their study in the relevant literature, make an argument, and evaluate it systematically using evidence they collected, typically from their work experience. To guide them, the Associate Director taught a weekly research methods class and the tutors (the graduate students) worked closely with a subset of about ten undergraduates. Students would write their theses in sections. Each week, regardless of whether something was due, the tutors would meet individually with each student for about half an hour. The combination of the professor’s lecture,1 students’ immersion in public policy, and regular and careful feedback, verbally and in writing, from tutors led to great success. Students would frequently start out with poorly articulated ideas and virtually no background in methodology; some even had weak writing skills. By the end of the semester, as long as my charges gave a good effort, they produced impressive work. These results were not only personally gratifying, but also instructive. At the time, I was discouraged by my own dissertation progress. What I had learned about methodology from my graduate studies seemed removed from the process of researching and writing my substantive chapters. CIW’s program provided undergraduates with a basic structure and showed how different methodological concerns and techniques informed distinct parts of the paper. Guiding my “tutees,” I saw that half-formed and disjoint ideas could become sophisticated as long as the writers, with my help, kept working, thinking, gathering data, analyzing systematically, and pushing forward. I began to realize that if I treated my own work similarly, I would finish my Ph.D. thesis, too.

33

EMPOWERING STUDENTS BY TEACHING …

391

After two years at CIW, I joined the faculty at Saint Joseph’s University, where I still am. I never thought I would be teaching research methods and, honestly, at the time, I didn’t want to. In those days, methodology meant statistics. My own experience2 underlined how unimportant the math skills were if they were divorced from an understanding of the research process, the ability to find appropriate information, knowledge of the basic structure of a research paper, and good writing skills. In time, however, I became convinced that a holistic understanding of methodology and the research process was what our majors needed. It would allow them to read journal articles and books more effectively, as well as to perform their own research and write more sophisticated papers. I also concluded that having experience with their own research projects would help level the playing field between them and students from the most prestigious institutions. Developing the skills for writing a research paper would better allow them to compete for law or graduate school admissions and attractive jobs. Because at the time such a class was a rarity in the USA, SJU graduates would have a meaningful foundation that other students lacked. Back in the early 1990s when I started teaching, few professors taught students how to write a research paper; faculty would give the assignment and collect the finished product at the end of the semester. Instructors would frequently be frustrated with what students submitted, although they hadn’t given many instructions, and most students were similarly unsatisfied. They tended to see these papers as “add-on” assignments. Students rarely used the exercise as a way to pull together the ideas developed in the course or take the time to think, create, and revise. Thus, the research paper was a failure for all parties, and many faculty stopped assigning them. I know I did. Making that decision to take away these assignments pained me. I saw what the thesis process had done in developing my CIW students and how I had benefited from it, too. I was opposed, however, to assigning a research paper unless it was “real,” and I didn’t know how to achieve that goal with a full class of students3 and no TA. Still, I couldn’t stop thinking about the importance of the researchpaper writing experience for opening the world of inquiry and systematic thinking to young people. It also privileges precision in reading, writing, and speaking, which most students don’t possess when they arrive at university. To acquire these talents, students must spend a good deal of time thinking, finding “good” theoretical and empirical information,

392

L. A. BAGLIONE

writing, interacting with other people about sophisticated topics, and rewriting. In fact, writing a “good” research paper is a process. Successful authors have to problem-solve and rethink original thoughts, and they rely on helpful others for feedback. Criticism is desired, a truism that students (and many professionals) don’t realize. The project teaches key lessons that extend beyond the academy: raw talent only takes a person so far, and no one achieves their best product on their own. Skill, stamina, and perseverance in response to constructive criticism are essential (Baglione 2020). As I became more convinced about the importance of this experience, I found an ally in my department. We began team-teaching a required “mini-course” (only 1 credit hour) to give students exposure to some very basic elements of research, including question asking, information literacy, and avoiding plagiarism. Over time, I earned tenure, staffing in the department changed, and I decided I was ready to make the ambitious proposal: Let’s create a writing-centric methods class in which students produce a research paper as a final product. If this plan was to work, however, the course would have to be required, students would need to take it rather early (so it could have an impact on their upper division and especially senior-seminar performance), and I would need buy-in from the rest of our small department. Research-paper writing couldn’t be thought of as the “weird-thing-that-Dr. B-cares-about.” Other faculty had to use elements of and talk supportively about this project. In the end, I was lucky. My ally agreed to co-teach the course if I designed it, and as Department Chair, he corralled the department into going along with the other conditions.4 And, thus, we began. Over about fifteen years, the Department and then IR program made this course a second-year-in-the-major requirement. Later, an additional faculty member joined me with another version of the class. My colleagues and I have tried many different approaches with our students, and our successes increased with time. I have been lucky enough to write about them in Writing a Research Paper in Political Science (2020), now in its fourth edition. As the versions have changed and improved over time, this book is evidence of my own iterative process of teaching, thinking, researching, and writing about research-paper writing. Critics have assailed the course because students don’t emerge from it as excellent researchers, writers, and/or budding political scientists. Others contend that my book, among other weaknesses, doesn’t teach enough or the “right” methods. To be sure, I cannot fully remedy the deficiencies in

33

EMPOWERING STUDENTS BY TEACHING …

393

students’ years of language arts training, nor cover a variety of methods. The purpose, however, is not to create professional political scientists, but to help students become better learners and critical thinkers, understand the research process, improve their written and oral communications skills, and increase enthusiasm for the major. Relatedly, I sought to provide students with foundational skills upon which to grow, as well as to remedy some of the confidence and social capital deficits that are linked to gender, race, and class positions, and are especially challenging at the intersections (Levintova and Staudinger 2018).5 While the approach built on my experiences at CIW, I greatly extended them (Baglione 2020). Without realizing the work that was simultaneously underway in the field of Education, I was emphasizing some of George C. Kuh’s (2008) High-Impact Educational Practices: (a) integrating across courses in the major and among relevant cognates; (b) engaging in “real” research that is as original as constraints will allow; (c) envisioning learning as a social process between the writer, professor, and class members; and (d) spiraling—researching, thinking, writing, and interacting repeatedly—through the process.

Empowering at the Outset: Seeing the Big Picture, Uncovering Their Preexisting Foundations, and Stressing the Deliberative, Honest Process When I teach my research-paper writing class, I address two unspoken truths at the outset. Many students dread (because of the workload) and fear (because of a lack of confidence or ignorance about the content) the course. I tell them I understand their feelings, and I am going to do my best to make this one of the most useful classes in their college career. I’m even going to try to make the course, or at least the memory of it, positive. I close with this reassurance: No student who tries does poorly, and the rest of the class and I am right there with you. The point is: yes, you have a difficult job ahead, but I will give you a map, and I won’t leave you to your own devices. I also stress that the approach is distilled from the wisdom of many talented thinkers who have created a basic structure for them to follow. That form guides the semester’s work. In fact, I reassure students that they have already been exposed to this map in earlier courses, but just didn’t notice it. In this class, we make the form and logic of those earlier

394

L. A. BAGLIONE

materials explicit and stress that the structure will make the project and other school work easier. Students can more easily decode their readings, and even use them as models for their own work. Emphasizing their preparation, we look for ways my colleagues have been developing students’ conceptual and theoretical knowledge of political science and have exposed them to excellent resources. Classes in history, sociology, philosophy, or other fields might also inform their projects. Students also have the library, research, and writing skills from their first-year seminars, first-year English classes, and other courses that they can apply. Thus, not only does a plan exist (which my book and I continually emphasize and elucidate), but students are already starting with useful resources and skills. They can do it! For some, my start with (and continuing emphasis on) positivity is annoying. While I am (only a little) sorry about that, those students with confidence are not my worry, although they do concern me if they tune me out. Generally, all but the most stubborn quickly realize they have something to learn. The reason: very few high schools teach researchpaper writing in the social sciences (NCWASC 2003). Moreover, in many high schools, the emphasis on scoring well on Advanced Placement (AP) or International Baccalaureate (IB) tests moves students away from some of the skills that are essential for asking good questions and evaluating the literature, let alone thinking about methodology. The students who took those exams often have developed facility with documentary analysis and sustaining an argument through a short paper. While these skills are helpful, they are not the same as having the experience of writing many, independently conceived analytical essays throughout a high school career (Tugend 2017). Of course, not all students come from high schools that had such courses. Others also might have had external or internally placed barriers on them that impeded their electing advanced coursework. Even with changes in the accessibility of AP courses in the last fifteen or so years, research shows that economically disadvantaged students, students of color, and often women students are less likely to have such courses because of funding limits, early tracking that restricts those learners, lack of encouragement, and social and internal factors that impede their confidence (Tugend 2017).6 These students, even if they have the capabilities, are often the ones in the most need of the reassurance. They’re the ones I most want to reach. So, I repeat (in various ways): there is a plan, and you are prepared.

33

EMPOWERING STUDENTS BY TEACHING …

395

Saying those words is not sufficient. I have to demonstrate and, in a sense, prove their truth. That I do by uncovering the form of the paper (the map) and helping them identify this form in many of the sources they have already read. Students analyze these “good” examples of political science research with attention to the basic form: Introduction with research question and answer (thesis); Literature Review and more careful (sometimes formal) statement of an argument or hypothesis; Research Design; Analysis and Assessment of that contention; and Conclusion. They also go back over their previous coursework themselves to find interesting materials that might help them launch and pursue their own research. In other words, they learn that they are going to be (1) building on work with which they are already familiar and (2) using these materials to help them find the major voices in the field. In effect, they will be sitting on the shoulders of giants as they proceed through their research projects. Then, reiterating that they will use this form and proceed steadily throughout the semester, we turn to making progress on their projects (Baglione 2020). In the early stages, we work on key basic skills, like information literacy for finding additional “good” sources. We also discuss how to evaluate the quality of the materials they are using, and how the types of sources they will find useful change over the course of the project, from scholarly articles to various forms of raw data. In early phases, they learn to summarize and analyze arguments. They struggle to overcome their tendency to describe what an article is “about,” as opposed to what the author argued and how she used logic and evidence to make her case, even though my colleagues stress these very points when teaching introductory classes. Another important element of this early phase is to encourage students to be critical and creative in their thinking. Most have understood that they need to be analytic, but many are nervous about this “creative” part because they want to “be right” in order to “do well.” Here, the challenge is making them believe in the centrality of honesty and process. I am not suggesting that my students are deceptive, but simply stressing that a widespread fear that frequently impedes them is the worry that their original hunch won’t be “correct” or “smart” enough. I remind them that by designing a good question, performing a thorough literature review, establishing a well-thought out and appropriate research design, and meticulously tracking down and analyzing data as they established in that plan, they set themselves up for success. Then, hard work, determination, and an honest effort accomplish the rest. If they proceed in this

396

L. A. BAGLIONE

manner and earn my “Oks” along the way (basically, if they implement the methodology), then they will produce a “good” paper.

Empowering Habits and Mindsets: The Social and Iterative Nature of the Process Convincing students to trust me and perceive me and their classmates as partners is a central challenge. As I mentioned, on that very first day, I tell students that they are not alone. To reassure and move them forward, they need to trust that I and others are on their side. They also need to learn to engage me regularly and to be ready to have intellectual conversations that advance their progress. These interactions serve as models for later whole-class exercises that emphasize thinking, writing, feedback, rethinking, and rewriting. We start with one-on-one consultations, then add in-class sharing of work and challenges, submission of written work to faculty and peers, and an end-of-semester presentation of their project to a larger audience, including friends, other students in the major, departmental colleagues, and even administrators. The trick is finding ways to emphasize the social that don’t create negative anxiety. Some nerves can prompt better performance, as many musicians, actors, and athletes will attest, but too many worries are counterproductive. That’s why the key to a positive social process starts with students’ trusting that the professor won’t let them appear foolish. To achieve that confidence, I ask them to imagine me as their director, coach, or boss, whichever identity works for them. The point is for them to see me as someone who wants their best performance and who will push them to get there, not as a teacher looking to “dock points” off their score. Our early and intense interactions are related to achieving that relationship, as well as developing a research question, searching for sources, and creating an annotated bibliography in preparation for a literature review. I show them they have some creative control, but I will encourage precision and keep them “on track.” In this phase, they produce relatively short deliverables to which I can quickly react, and then we can come together to discuss. They see that they have to be prepared, that I will ask them questions, and that I want to have a conversation in which they employ specialist language. Early on, I have to learn to read and listen incredibly carefully because students often can’t articulate exactly what is compelling to them. That’s why writing, talking, and listening and doing all that over again is so important.

33

EMPOWERING STUDENTS BY TEACHING …

397

Students also help each other at this early stage, especially if they have trusting relationships and clear guidelines about what makes a research question or an annotated bibliography “good.” Some semesters, class members just begin with that helpful vibe, and everyone benefits. But even when we have to work at the intraclass trust, I can remind students of the map and show them examples of each of the sections, some of which come directly from earlier students’ papers. Evaluating what makes each one “good” helps students see what is expected. Knowing that the excerpts were student-written gives them confidence that they, too, can achieve the goal. Hearing from their peers that they have done well mimicking elements of the form, but might have to make some specific improvements on the execution, makes sharing not so scary. There’s always something positive to say about someone’s work, and seeing others’ challenges and successes is encouraging. Their peers’ difficulties and accomplishments help them each master basic methodology and move forward on their research. This approach uses two kinds of iteration. First, students see multiple variants of a “good” section of a research paper. Analyses of peers’ drafts always come before they need to submit their own work for a grade. Thus, students can use an existing model and/or what they learned workshopping a classmate to improve their own essays. Second, any time students write a new section for a grade, they also go back and revise what came before. That way, they are constantly moving forward, as well as going back to refine, rethink, and improve. This process is admittedly frustrating for some class members, and I validate those feelings so they know that by persevering, they will move beyond the frustration to success. By the end of the semester, students always wish there was more time, but we still spiral ahead, pushing forward while looking back, in order to finish by the due date. As we approach the end of the semester, the most nerve-wracking assignment is often the poster presentation, especially because students present their paper before it is complete. But that is the point: to envision the whole before writing the last line. For some, putting together the poster is the most helpful part of the presentation, and they achieve an “ah ha” moment from seeing the pieces all on one, admittedly large “page.” For others, the light shines from creating their elevator pitch and abstract, explaining what they investigated, why and how they proceeded, what they found, and detailing how they would continue the research in

398

L. A. BAGLIONE

the future. Still others gain from talking to people and answering questions during the actual presentations. After dreading it, the majority of students find they actually enjoy and benefit from making the presentation and listening to others talk about their work. Their final revisions benefit from having the insights gleaned from this experience; they turn in polished papers that easily surpass the minimum page requirement and their previous “best” efforts. Students generally feel extremely proud of their accomplishments.

Empowerment Through Research-Paper Writing Teaching research-paper writing in a step-wise and interactive fashion is a way to teach methodology as well as to empower students. I have seen enormously positive transformations happen many times. While the approach is time-consuming for all involved, the payoffs are huge. The process empowers students by giving them confidence in their abilities and wisdom about the nature of research and research-paper writing, while readying them for other challenges in the academy, the business world, politics, advocacy, and even life. Their semester has shown them that to succeed on a big project is hard but doable, as long as students are willing to devote the necessary time to the task and proceed with a plan, appropriate resources, and supportive others. A positive attitude and a willingness to rethink, solicit, and listen to feedback from knowledgeable interlocutors, and push on are vital, too.

Notes 1. At the time, Steven I. Jackson was the faculty member. He was ably assisted by the head tutor, Jack Moran, who had originally designed the class a decade or so earlier. 2. As an undergraduate, I earned an Sc.B. in Applied Mathematics and Economics. As part of my major curriculum, I took two courses in statistics (with 3-semesters of calculus and linear algebra as prerequisites) followed by two semesters of econometrics. 3. These were the days before writing-intensive caps. 4. Graham Lee was my original partner. Later, Susan P. Liebell taught a new and innovative version of the class. 5. Levintova and Staudinger’s volume Gender in the Political Science Classroom has many useful chapters on this topic.

33

EMPOWERING STUDENTS BY TEACHING …

399

6. On the importance of doing more to reach women students and students at the intersections, see various selections in Levintova and Staudinger, eds. Gender in the Political Science Classroom. Kuh also stresses that HIEPs are especially important for traditionally disadvantaged students.

References Baglione, L. A. 2020. Writing a Research Paper in Political Science: A Practical Guide to Inquiry, Structure, and Methods. Thousand Oaks: CQ/Sage. High-Impact Educational Practices. n.d. American Association of Colleges and Universities. Accessed July 25, 2020. https://www.aacu.org/sites/default/ files/files/LEAP/HIP_tables.pdf. Kuh, G. C. 2008. High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter. Washington, DC: American Association of Colleges and Universities. Levintova, E. M., and A. K. Staudinger, eds. 2018. Gender in the Political Science Classroom. Bloomington, IN: Indiana University Press. National Commission on Writing in America’s Schools and Colleges (NCWASC). 2003. The Neglected R: The Need for a Writing Revolution. New York: College Entrance Examination Board. http://www.collegeboard.com/prod_down loads/writingcom/neglectedr.pdf. Tugend, A. 2017. Who Benefits from the Expansion of A.P. Classes? New York Times, September 7, 2017. https://www.nytimes.com/2017/09/07/ magazine/who-benefits-from-the-expansion-of-ap-classes.html.

CHAPTER 34

From “Good” to “Effective”: Teaching Writing Skills Explicitly in Political Science Colin M. Brown

My personal and research interest in the ways that we teach writing within the discipline came about, like so much social science research, from encountering a real-world puzzle during my time as a teaching assistant: “If my students are generally such clever and articulate writers, why is their writing still so bad?” This is, of course, a somewhat unfair characterization. But working with undergraduates at a highly selective university, the essays and assignments I graded were generally rich with complex language, clever turns of phrase, and elegant prose. And yet, they were not particularly enjoyable to read. Students were over-using jargon, they were polemical far beyond anything appropriate to the topic at hand, and their papers seemed designed to introduce as much evidence as possible without ever analyzing it (e.g., “Fifthly, modernization theory also forcefully proves…”) Students

C. M. Brown (B) College of Social Sciences and Humanities, Northeastern University, Boston, MA, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_34

401

402

C. M. BROWN

also seemed largely unaware of the problems in their writing, which they thought was already quite good. They usually seemed completely surprised at, though mostly open to, the critiques and suggestions they received. So why was this group of largely successful students turning in work that was tedious—at best—to read and to grade? Solving this puzzle, and finding ways to turn “good” writers into “effective” writers, has been the central project in my development as a teacher from graduate school into my current faculty position. The most important takeaway from my work so far has been that there is a need for explicitly naming and focusing on the writing skills that we want to teach. We as political scientists are (usually) not also writing instructors—but the writing skills we want students to have are not just general writing skills; they are intricately tied up with the kinds of thinking we want them to do and the norms of the community we are inviting them into.

Undoing “Good” Writing The answer to my initial question of why my students’ writing was so hard to read was: because they were producing what they had explicitly been taught to write. Most of my students had come up through the American educational system, where they had usually been taught the “5-paragraph essay” style at some point (and often extensively, across multiple classes and grade levels). This format, which emphasizes making 3 clear points on a related topic, is an excellent way to quickly demonstrate knowledge of a subject—to show off what you know. But it encourages a formulaic structure of distinct facts and assertions, and it rarely demands thinking about them in any systematic way beyond their general connection to a topic. My own experience, at moderately under-resourced US public schools, had introduced the 5-paragraph essay at some point but had not really emphasized it. I was therefore also seeing the format mostly from the outside. Once I really became aware of how central this format had been for them, it seemed a lot of the things students did that were causing me such anguish as a grader were not due to any lack of skill, but to the need to unlearn a specific, deeply ingrained and reinforced skill. The 5-paragraph style’s greatest fault when transferred to our highered setting is its failure to do the thing we value most as social scientists— finding evidence for and resolving empirical and/or theoretical questions. In practice, most of what I was spending my time on was getting students to think about what their question and answer actually were, rather than

34

FROM “GOOD” TO “EFFECTIVE”: TEACHING …

403

just what topic their paper was about. For example, students could give great definitions of the collective action problem and could give clear examples as they’d read or heard about them, but had trouble arguing why they were important or what conditions could solve them. Students could demonstrate what they knew in this style, but the format didn’t challenge them to explain why it mattered, or how else it could be applied. One possible solution to this mismatch in writing styles is simply that students need to do more reading, especially of high-level academic writing. Through this extended reading process, it is expected that students will eventually internalize the rhetorical “moves”1 and structures that make for effective and convincing writing. There is some evidence for the idea that simply reading a lot makes one a better writer, and it is true that a disproportionate number of successful academics themselves learned how to write this way (Russell 1991). But there is no reason to think that this is the only successful way to learn how to write, and the trend in research on writing in the disciplines has, if anything, increasingly stressed the importance of writing as a tool for learning how to read, more than the other way around (Klein and Boscolo 2016). The predominance of this approach within our graduate education and faculty training probably shows a selection bias for those of us who adapted successfully to this method of instruction because it was the only one offered. In a world dominated by much more “horizontal” reading than “vertical” reading (cf. Birkerts 1994), today’s students may find this method especially difficult. Students, like academics and the rest of society, are in a reading environment that defaults to shallow reading rather than the kind of deep grappling with texts that brings attention to forms. Lastly, this approach is also likely to compound the disadvantages students bring with them to college. Any grades that account for writing ability—as they do for so many instructors—end up folding existing writing deficits into students’ assessments. If students’ writing problems result from a style that they had deliberately been taught, then it should be possible to teach an academic writing style just as deliberately (rather than just assigning heavy reading loads and hoping for the best). Students probably could eventually read enough to copy our professional writing style, but there is no guarantee they will— or will want to—by the time they leave higher education. The obvious question this left me was: how to teach this style deliberately?

404

C. M. BROWN

How Not to “Cheat” Around the time that I was really starting to notice this mismatch, the department in which I was a graduate student underwent a major academic integrity scandal at the undergraduate level. Over 100 students in a large section of a course were investigated for improper collaboration on a take-home exam, leading to academic sanctions for many of them. The incident happened during a term where I was away doing field research for my dissertation, so I was initially detached from the event. When I got back the next semester and needed a teaching assignment, the opportunity opened up to work as a “writing fellow,” a position that the university funded within each department so that undergraduate students could meet with a graduate student in their specific field to get writing advice. This program was designed as a complement to the university’s writing center, but, as often happens, it came with only the barest training in composition and pedagogy of composition.2 In prior years, the writing fellow had mostly been attached to two introductory courses and had offered the chance to workshop papers with students directly. When I started, I was also asked to spend some of my time on a new project designed to teach writing skills. As the university’s disciplinary process unfolded, the department had spent some time reflecting on the potential causes of the sudden and large cheating event. In talking with faculty and reflecting on the literature about academic dishonesty among students, the department was increasingly convinced that simple lack of knowledge around citation, plagiarism, and collaboration might have played a large role.3 The university had a strict policy around plagiarism, which was soon to be developed into a full honor code. But where exactly did the university teach how not to plagiarize? In so many departments and colleges, students receive little or no formal training in plagiarism unless they seek it out—which they cannot do unless they know to seek it out. There are exceptions, but at the university I studied at, as with the majority of other undergraduate institutions (Gullifer and Tyson 2014; Nelson et al. 2013), academic integrity policies were rarely discussed outside the context of first-year orientation or else from within the university’s judicial and enforcement mechanisms after a breach had already been committed. A student learning the rules while facing expulsion could easily argue that this is too little, too late.

34

FROM “GOOD” TO “EFFECTIVE”: TEACHING …

405

It can be hard for us within a given “discourse community” like academic social science to remember how opaque standards or norms are to those on the outside. Looking at it from a faculty perspective, it is easy to summarize avoiding plagiarism as something like, “cite your sources, and don’t cheat.” But neither of these are purely self-evident concepts outside a particular genre. For example, students with a strong background in the natural sciences may not have encountered how conceptual definitions in the social sciences are often not universal or “common knowledge,” and usually need to be cited as a way of grounding one’s conceptualizations in the existing conversations. Alternatively, students with a humanities background might quote directly to great excess, often with multiple quotes of 5 to 8 sentences each in a 5-page paper. These students, unaware of how important paraphrasing is in the social sciences, were similarly misunderstanding the community for whom they were writing; they likely saw citation as an “empty formality” rather than as something aiding in the ultimate purpose of the paper (Pecorari 2013). What is slightly more troubling, however, is that the consequences for misapplying skills may not fall evenly. I, for example, would have been inclined to see the latter student as heavy-handed and unpleasant to read, maybe as essay “padding” at worst, while the student from the natural sciences who did not cite concepts could easily have appeared to be plagiarizing.

Tools for Teaching Writing When I started in the role of writing fellow, I had two goals for myself. The first was that I wanted to teach students how to develop the analytic styles of writing common to our discipline. How could they make arguments and use evidence in the ways that social scientists do? More importantly, how could they provide evidence in a way that tested their own ideas and hypotheses—that is, engage in critical thinking—rather than simply showing off that they had done the reading and retained a little bit of it? The second was teaching plagiarism in a way that helps it make sense and feel important to undergraduates. How to do this? Figuring out how to teach these took some time and effort, as good teaching generally does. Around that time, I had been introduced to the idea of reflexive practice, and I started to take more systematic notes on the kinds of feedback that I most often gave4 . I noticed that feedback on organization often wasn’t being incorporated when it was made in

406

C. M. BROWN

writing. However, when I had in-person meetings with students, they were open to improving their organization. Simply telling them that a paper was disorganized—or even suggesting specific alternatives—wasn’t helping them unless we were able to have a dialogue about what the purpose of the paper’s organization was in the first place. The use of evidence created similar problems, in that comments or written feedback about a student’s use of evidence usually led them to simply include more examples or data (often just as vaguely related to their point as what they had originally included), rather than showing more clearly and specifically how it supported their argument. But when we had in-person meetings, and we were able to talk about the overall goal of the paper, students were pretty receptive to the comments and tended to incorporate my feedback more successfully in the revised drafts they submitted. After a little while in the role as writing fellow, my conversations with colleagues in our department’s administration led us collectively to decide that we not only needed more writing instruction, but we also needed to set it up in a way that explained our goals explicitly, rather than simply providing unmoored advice and “tips.” Because faculty already felt overwhelmed by how much they had to get through in their courses and didn’t necessarily want to spend a lot of time teaching writing, we also had to figure out a way to create a light “footprint” that didn’t impose on faculty’s time. Given these limits on what we could do, we settled on the simplest solution we could think of: we set up a website that could be required for students in several contexts. It would focus on very specific skills tailored to political science writing, and it would be set up so that teaching assistants could use it in their recitations without being forced to learn writing pedagogy on top of their other extensive commitments. Some of the things we focused on were interpreting paper prompts (in other words, taking the time to understand what the professor was actually asking for), writing a thesis statement that was arguable and falsifiable, and how to paraphrase from sources without plagiarizing. We were lucky, as a number of faculty were willing to let us try these materials out in their courses. While it didn’t create more work for them, it did take time away from recitation sections, and they did have to adjust their assignments slightly to ensure that they were in line with the kind of analytic essays that the department felt was most appropriate for firstand second-year students. We were unlucky in that we didn’t really know how to measure the effectiveness of these tools at the time. Trying to

34

FROM “GOOD” TO “EFFECTIVE”: TEACHING …

407

determine how to measure them led to a new research agenda for several of us on the team,5 but by the time we had the tools to assess the initial project, most of us had taken on new roles that kept us from returning to it. Anecdotally, students seemed to appreciate it and find it intuitive. The language of, “what question is the professor actually asking?” seemed to seep into student understandings of what essays were supposed to be about. When I met with students in our introductory comparative politics class in later semesters who had encountered this tool, our conversations seemed to start from a place of greater understanding than it had in my initial semesters working on the same assignment. As we clarified the goals of analytic writing, we found that conversations around plagiarism got easier. We did include a tool that provided specific practice on paraphrasing without plagiarizing, but we found that the real gains came indirectly through students’ increased general understanding of what social science writing was actually trying to do. The more that we were able to frame analytic writing as answering a question and as joining in an intellectual conversation about that topic, the clearer it was what citations were for. That frame of reference seemed to help students understand that references were not simply a way of demonstrating what (and how much) they’d read—as we realized was often their initial understanding of what the instructor wanted. Other known contributing factors to plagiarism such as poor time management remained a risk, but understanding the broader purpose of academic writing helped remove the uncertainty about what actually constituted “cheating” that itself has been identified as a key trigger for plagiarism (Devlin and Gray 2007; Gullifer and Tyson 2014). Even students who did panic and turned in, let us say, less-than-their-best work still seemed less likely to plagiarize, and more likely to fill space with their own thoughts rather than with summaries or quotes that risked citation failures. What I took away from this was that explicitly naming and teaching writing skills was something that could be done within the context of a political science classroom, and that it had benefits in terms of student learning about our specific topic. Students weren’t just improving as writers in some abstract sense, they were improving as writers who could make arguments where evidence specifically supported or even tested a theory. We had framed these as writing skills, but they really were fundamental reasoning skills of social science.

408

C. M. BROWN

Building Writing into the Course As a new faculty member, when I designed new courses for the first time, I thought about the specific goals that I wanted students to have as writers. I was not expected to explicitly teach writing, and other faculty did teach required composition courses. But by centering writing assignments within my course design, I could think more clearly about the specific kinds of reasoning and skills that I wanted them to practice and develop at each stage in the course. Where my teaching portfolio would largely focus on introductory courses, this could even be a chance to help students set out on the right foot and know the kind of arguments that they would be writing—and reading—in their other courses. My introductory courses now have a number of short, relatively lowstakes papers, each corresponding to a specific writing or analytic skill. For example, in a recent section of my comparative politics course, I assigned a 2-page paper for them to practice selecting comparable cases without focusing on the thesis or evidence, a 2–4-page analytic paper for them to practice making a clear thesis statement and supporting it with evidence, a 4–6-page paper to practice taking feedback and making revisions on their own work, and then a 2–4-page reflective essay to assess their own learning. Students are not required to use the same topic and cases in the first three papers, but I invite them to do so if they wish. As I had experienced when I was a writing fellow, students don’t instantly turn in “perfect” writing, but there is less confusion about what the expectations were than I saw in my early experience as a TA. Students ask more focused questions about the quality of their argument rather than generally asking, “is this enough?” Even the papers that are less successful in using evidence are generally clear that they have a thesis. In order to center students on the specific goals that I have for them, I use both the syllabus and the assignment to state the goals explicitly and show how the assignment relates to them. On the revision paper, for example, I emphasize that their goal is to address feedback, and that they can make a more effective argument regardless of how strong the first paper is. For the comparison paper, the goal isn’t to write beautifully or in any specific format, but simply to make sure the reader can understand their case selection. As part of this, I provide a short list of things on which I will and won’t grade them for each paper (and I make sure they know these are different for each). In the case comparison paper, I emphasize that they don’t need to worry about a thesis statement or

34

FROM “GOOD” TO “EFFECTIVE”: TEACHING …

409

grammar, but they do need to explain the relevant differences or similarities. In the analytic paper and the revisions, I emphasize that they don’t need to show off every thing they’ve learned in the class, but rather they should show that they know how to use evidence to further analyze some thing they learned. For my senior capstone class, instead of scaffolding the skills piece by piece, each assignment focuses on a different audience and is assessed on how well students can present their social science argument in a way that speaks to that specific audience (students otherwise pick their own topic and select their own readings). Students start out by writing book reviews, first for an academic journal, which has clearer goals, and then for a popular journal like The National Review or The Nation, where there is more freedom to draw political or policy-oriented conclusions. They condense their arguments into a 600-word op-ed, and later expand it into a 15-page conference paper. While each specific assignment only teaches one genre, the overall set shows the importance of thinking about audiences and how the same argument may at different times require different standards—and kinds—of evidence. Each paper foregrounds the point that every kind of writing has a specific aim.

Conclusions My own teaching journey has brought me to try a number of different approaches to writing to see if we can get students past writing “well” to writing clearly and effectively. Because of the separation between composition studies and research on teaching the social sciences, it has been sometimes hard to rigorously assess how well this has worked—but it has been rewarding to work with several colleagues to systemize what we know about writing. From my own personal experience, and from the outcomes I have observed in my own students, there is much to gain from clearly expressing our expectations, not just in terms of content, but in terms of skills. By focusing our writing instruction and assignments on the specific genres of writing and the rhetorical tools that are effective within our discipline, we equip students to leave the classroom able to write for audiences beyond just the instructors who have to read them. And, I think, we set ourselves up for a lot more enjoyment in the grading and feedback processes.

410

C. M. BROWN

Acknowledgements The author would like to thank Sarah E. James, Jonah Johnston, George Soroka, and Cheryl Welch for their help in setting him on this professional course, and to Kelly Bauer, Jennifer Bachner, Chelsea Kaufman, and William O’Brochta for feedback on earlier drafts of this chapter.

Notes 1. See Graff and Birkenstein (2006) (who would likely not endorse this pure learning-by-reading approach) for more on the idea of “rhetorical moves.” 2. The university has, laudably, addressed this within the past few years and graduate students in this role now receive considerably more training and support from multiple sources. 3. This also fits with the findings of many studies on plagiarism (see, for example, Gullifer and Tyson 2014). 4. And creating boilerplate comments that I could use to copy and paste for the most common feedback was, and is, a time-saver that I was glad to have on hand. This saves more time for writing more specific examples tailored to the specific student. 5. For some of our initial findings, on the application of rubrics to social science argument, see Brown et al. (2020).

References Birkerts, S. 1994. The Gutenberg Elegies: The Fate of Reading in an Electronic Age. Boston: Faber and Faber. Brown, C., S.E. James, and G. Soroka. 2020. Explicit Content: Two Experiments on Bringing Writing Instruction into the Political Science Classroom. Journal of Political Science Education. https://doi.org/10.1080/15512169. 2020.1716239. Devlin, M., and K. Gray. 2007. In Their Own Words: A Qualitative Study of the Reasons Australian University Students Plagiarize. Higher Education Research & Development 26 (2): 181–198. https://doi.org/10.1080/072943607013 10805. Graff, G., and C. Birkenstein. 2006. “They Say/I Say”: The Moves That Matter in Academic Writing. New York: W.W. Norton & Company. Gullifer, J.M., and G.A. Tyson. 2014. Who Has Read the Policy on Plagiarism? Unpacking Students’ Understanding of Plagiarism. Studies in Higher Education 39 (7): 1202–1218. https://doi.org/10.1080/03075079.2013. 777412. Klein, P.D., and P. Boscolo. 2016. Trends in Research on Writing as a Learning Activity. Journal of Writing Research 7 (3): 311–350. https://doi.org/10. 17239/jowr-2016.07.03.01.

34

FROM “GOOD” TO “EFFECTIVE”: TEACHING …

411

Nelson, L.P., R.K. Nelson, and L. Tichenor. 2013. Understanding Today’s Students: Entry-Level Science Student Involvement in Academic Dishonesty. Journal of College Science Teaching 42 (3): 52–57. https://www.jstor.org/sta ble/43631795. Pecorari, D. 2013. Teaching to Avoid Plagiarism: How to Promote Good Source Use. London: Open University Press. Russell, D.R. 1991. Writing in the Academic Disciplines, 1870–1990: A Curricular History. Carbondale: Southern Illinois University Press.

CHAPTER 35

Revising the Revising Process of Writing in Upper Level Political Science Research Methods Emily M. Farris

On the first day of Survey Research, a course I teach in our upper division research methods sequence, I tell my students that I want them to complete a final paper in my class that could be used as a writing example for either a job application or for graduate school admission. I explain that this goal sets high expectations, both for them to produce and for me to guide in their work over the course of the semester. Undergraduate students in my Survey Research class are expected to write a final paper of at least 20 pages in the model of a traditional political science empirical journal article and use original survey research they conduct themselves. Former students report that they have used these papers and skills from

E. M. Farris (B) Department of Political Science, Texas Christian University, Forth Worth, TX, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_35

413

414

E. M. FARRIS

the class to get into graduate programs, such as a master’s degree in public policy, to get into law schools, and to help design presentations to clients. When discussing the plan for their research projects, I present to students what that process of work will look like over the course of the semester. On my slide, arrows between the steps point both forward and backwards to show the progression, and I emphasize that these steps mean we will revise, revise, and revise to the final paper. And over the course of the years I have taught this class, I too have revised (and revised and revised) how I have approached it, with more of a focus on the different components of the writing process. When I first began teaching this class, I told students their semester goal was to complete a survey, and now it is to complete a paper using survey research. In this narrative, I discuss how my approach and activities in the course have evolved to focus less on technical details of survey research methods and more on the transferable skills of writing. Early iterations of the course focused heavily on the nature of surveys and mechanics of survey research, with students’ final written project designed as only a secondary goal and a mechanism for communicating their survey analysis. Over the years, I have shifted more of the course to develop students’ skills in communicating their ideas and analysis.

Background on the Survey Research Class Political science research methods classes traditionally focus on the process and skills of conducting empirical research. In an upper level methods class, like my Survey Research course, this means focusing on one specific approach to conduct empirical political science research. For some universities, these methods classes serve as or complement disciplinary writing classes. At my university where the capstone class and thesis projects are limited to a select number of students in the Political Science Distinction Club and Honors College, this course allows all students to have the experience of conducting their own research. The class asks students to understand the science of survey research methods and develop their ability to use writing as a means of gaining and expressing an understanding of a subject, and it is a course designated with a writing emphasis for our university curriculum. In addition to its place in the university curriculum, my Survey Research class is an upper level research methods class which fulfills the last course in the methods sequence for our Political Science Bachelor

35

REVISING THE REVISING PROCESS OF WRITING …

415

of Science students at Texas Christian University (TCU). We offer three Political Science degrees in my department: one Bachelor of Science degree and two Bachelor of Arts degrees, with one focused more on political theory and a second on international affairs. To graduate with a Bachelor of Science degree, students complete training in empirical Political Science. They begin the sequential portion of the degree by first completing a math class, typically one on statistics and probability, external to the department. Next they take a class titled “Scope and Methods,” offered by our department that covers an introduction to empirical political science research and general focus on quantitative research methods. And finally, students take an upper level course which allows undergraduate students to specialize in a topic in methodology. Along with Survey Research, the department offers other courses in experimental research, case studies, qualitative research, and other occasional topics to fulfill this upper level methods requirement.

Initial Approach to the Class When I came to TCU as a new assistant professor, I inherited my Survey Research class from two of my colleagues, one of whom also still regularly teaches it. Fortunately, they helpfully shared syllabi and course materials to get me started—a practice I believe all faculty should do for their junior colleagues, particularly for methods courses that can be a challenge to teach. Although I had designed and completed research projects with original surveys myself, I never received formal training in survey research, as that was not offered at my undergraduate or graduate school. While initially worried that my lack of formal training on the topic would limit my ability to design and teach the course, I now understand it actually allows me to better connect to undergraduate students new to the material and share my own experiences in trying to learn on my own. Especially as I began teaching the course, I approached it by imagining what I would have wanted in an entire class dedicated to the subject of survey research. I was excited to dive into the nuances of surveys with students. Initially I developed the course with a primary goal of having students collect and analyze original survey data and a secondary goal of having students better understand the role of surveys in politics. My initial syllabus from Spring 2014 divided the class into two parts: lectures on surveys and their role in politics and workshops focusing on the mechanics of survey research and analysis. In the first part of the

416

E. M. FARRIS

course, lectures primarily centered on how polls are used in politics, and I used Asher’s (2010) Polling and the Public: What Every Citizen Should Know as the primary text for the class. My lectures covered why we should care about polls and what polling in elections and the media looks like. I then moved into understanding the mechanics of survey research with topics on issues such as sampling, interviewer effects, and question wording. Ramping up over the semester, students individually completed their own final research project, which included fielding a campus class survey. I broke down the research process to include four smaller assignments (on their research question, hypotheses, survey questions, and survey administration strategy) and two drafts of parts of their final paper (their literature review and research design). To try to connect the two goals of the class, students completed an article review assignment right after the midpoint of the semester, where they chose a political science research paper that used survey data and analyzed its methods in light of what they had learned in lecture. To assist in their research, students spent about a third of the class in workshop, following their midterm until their final presentations in the last weeks of the course. In the workshop, students spent time together in class with me at the computer lab to write survey questions, analyze the data, and write up their analysis.

Understanding the Challenges of Teaching Survey Research As I regularly taught this course over the years, I came to better know my students and their understanding of the course’s role in their goals. In the six times I have taught the course, only once has a student taken it as an elective, as a minor in Political Science. I soon realized that for many of my students this is not a class they are particularly interested in or excited about. On the end of the semester evaluations for my class over the years, less than half of students agree or strongly agree with the statement “I was interested in taking this course.” I expect this is true for most of the department’s upper level methods courses. If these courses were not required in our degree plan, it is likely not many of our students would take them. This can be a real challenge in teaching methods courses: how do you connect to students who are not initially interested in the topic and might

35

REVISING THE REVISING PROCESS OF WRITING …

417

even be anxious in taking the class? For some of our students, who struggled in their earlier Scope and Methods course, having to take another course building on that material and diving further into the topic of methods can be stressful. Furthermore, for most students, they do not aspire to become survey researchers themselves. I try to reach students in multiple ways, despite these obstacles. I impart my enthusiasm for the topic, by doing things like bringing in my personal copy of the famous 1936 Literary Digest Poll and using amusing material like the online poll to name a UK research vessel “BoatyMcBoatface” to illustrate course concepts. I also connect surveys to their own lives, by having students think about how to design questions to measure who truly makes the “best” taco in town (a difficult task given that we live in Texas). However, I am also more realistic now that they may never share my interest in survey research. So, my goal for my students is not necessarily for them to become future survey researchers at places like Pew Research Center or for them to become political science Ph.D. candidates. Instead, I expect each student leaving my class will have more knowledge in how to consume and produce research, as well as how to communicate their ideas and analysis, as this is something all of them will use in their life. I want students to leave my class with a better understanding of how to effectively use information and express their ideas. In a productive semester, students accomplish this by writing a final research paper and presenting their original research to the class. To combat their dread of methods classes, I am constantly reassuring students that this work can be both challenging and achievable, as well as rewarding by the end of the semester. I reiterate to students what I explained initially about the research process, as seen in Fig. 35.1 from my lecture slides, that we will spend a lot of time building and revising our work as part of the research process. In revising the course, I have made this reiterative process more evident to students, and by the end of the semester, multiple students have commented that they enjoy the class and the opportunity to conduct research despite their initial hesitations. As one student put it on their evaluation, they “actually got to be a political scientist… awesome.”

Revising the Course To accomplish this work and better engage students, I have readjusted my expectations for the class. Over time, I have cut assignments and

418

E. M. FARRIS

Fig. 35.1 Research process

lectures that focused more on the nuances of survey research content and added content and activities to develop students’ writing skills through close reads of model journal articles and assignments such as an annotated bibliography. I now better explain to students the writing process and how these writing skills, like critical thinking skills and the ability to construct an argument, are transferable to other classes and beyond the academy in their future endeavors. I use writing as a way for students to organize and better understand the class’s content and engage with the content. To accomplish the writing goals of the class, which I undervalued initially in developing the course, I have come to view the class less as preparing students to understand all the details of survey research and more as an opportunity for students to conduct original research through surveys and communicate their ideas and analysis. Overall, this is a fairly subtle shift but an important one in my goals for students and approach to the course. This shift is evident looking at the development of my syllabus, lectures, and assignments over the years. Comparing my first syllabus for this course in spring of 2014 to the most recent time I taught in spring 2019, two things are notable to me. First, the length of the syllabus doubled, from six to twelve pages. Most

35

REVISING THE REVISING PROCESS OF WRITING …

419

of this additional length is due to course policies being better developed and campus resources being incorporated into the syllabus. Second, the amount of time spent on the writing process in the course schedule has also at least doubled (see Table 35.1). In my first iteration of the class, I only devoted three lectures explicitly to writing: “Planning Research Projects,” “Researching and Writing a Literature Review,” and “Writing Analysis.” I assigned Lisa Baglione’s (2019) excellent text, Writing a Research Paper in Political Science: A Practical Guide to Inquiry, Structure, and Methods, and I largely expected students to independently review, through the reading, what they needed to know about writing. I assumed through their other classes they would have developed their writing skills, and most writing instruction would come in the form of my feedback on their drafts. Now, I offer five lectures on writing and better understanding the research process more broadly. Table 35.1 demonstrates the differences in my class between 2014 and 2019. The guided lectures are often shorter but cover more topics: “Research Project Introduction,” “Finding the Scholarly Debate,” “Writing the Literature Review,” “Conducting Analysis,” and “Guide to Presenting and Final Paper.” I also incorporate more workshop time focused on their writing, including workshop time beginning earlier in the semester. In addition to providing feedback on students’ smaller assignments and drafts, I now proactively guide students as they initially begin researching and writing through a short lecture at the start of the workshop time. The course is still designed to run in part as a traditional class with lecture and discussion and in part as a lab through workshop days. The first half of the schedule of the semester covers much of the basics of survey research, now using Nardi’s (2018) textbook, Doing Survey Research: A Guide to Quantitative Research Methods. Beginning in week four, and increasing throughout the semester, the class also focuses on the students’ individual research projects through guided lectures and workshop days. Following advice from students in evaluations, I start this process earlier so that they are learning the fundamentals of survey research while also developing their research question and understanding the literature. Students now spend a little under half of the class time in the workshop, up from a third of the time when I first began teaching the course. Instead of devoting only entire class days to workshop time or lectures, I now offer combinations of lecture and workshop days.

420

E. M. FARRIS

Table 35.1 Comparing Syllabi in terms of Lecture Topics and Assignments Spring 2014

Fall 2019

Lectures on Polling

Introduction to Survey Research Media and Polls Election and Polls (2 days)

Lectures on Survey Mechanics

The Problem of Non-Attitudes Wording and Context of Questions (2 days) Sampling Techniques (2 days) Interviewing and Data Collection Procedures Planning Research Projects Researching and Writing a Literature Review Writing Analysis

What Can Surveys Tell Us Media and Polls Election and Polls (2 days) Sampling (2 days) Questionnaire Design (2 days) Quantitative Review

Lectures on Research Process/Writing

Workshops

Research Design Survey Questions (2 days) Survey Format Survey Coding Survey Analysis (2 days) Writing Workshop

Assignments

Exam Article Review Research Proposal Assignments (4): Research Question, Hypotheses, Survey Questions, and Survey Administration Strategy Research Paper Drafts (2) Final Presentation Final Paper

Research Project Introduction Finding the Scholarly Debate Writing the Literature Review Conducting Analysis Guide to Presenting and Final Paper Initial Research Design Annotated Bibliography Survey Questions (2 days) Literature Review (2 days) Research Design Survey Analysis (2 days) Presentation Final Paper Exams (2) Research Proposal Assignments (3): Annotated Bibliography, Hypotheses, and Survey Questions Research Paper Drafts (2) Final Presentation Final Paper

35

REVISING THE REVISING PROCESS OF WRITING …

421

In week four, students select (or are close to selecting) a research question from a broad set of public opinion topics I provide students. While I previously allowed students to select any research topic they wanted in my first few tries of teaching this class, I found it was difficult to manage 20 or so different topics and literatures. Students also found it overwhelming to master the topics alone. So now I limit the topics to three or four contemporary public opinion areas, and I work with students to group them together based on interests. Initially I resisted wanting to guide their topics too heavily, out of concern I might limit students’ interests in the topics, particularly given their low levels of interest in the course. But I found I give better direction and advice in areas I am more familiar with. Students’ progress through the research process in a series of assignments that break down the steps into a number of individual assignments. This level of work may look overwhelming to students at the outset. However, appreciation for this breakdown is the most common comment in my student evaluations. Some representative comments mention their appreciation for the smaller assignments building towards the final paper, making it easier to understand the research process.

Revising Course Assignments and Workshop The assignments are broken down to include three research proposal assignments for their annotated bibliography, hypotheses, and survey questions, two research paper drafts for their literature review and research design, and an academic conference style presentation of their paper. All of these amount to a final paper that is their final assignment due during the exam period. In earlier iterations of the class, I had students complete additional research proposal assignments on their research questions and survey administration design but I have instead incorporated those into class activities, after realizing grading these assignments was not particularly useful and overburdened students with work outside of class. One of the major modifications I have made to the class over time is the early focus on guiding students to better understand how to find a scholarly debate and develop their literature review. Ideally, this is something students would also or would previously develop in their other classes. However, in my own upper level classes beyond Survey Research, I admit that this is often not a priority in my classes. While my students

422

E. M. FARRIS

write papers in those classes that engage with scholars’ research, these papers can often take on different formats than a traditional journal article, such as a policy recommendation proposal. When I informally poll my students in Survey Research to ask if they have written a literature review before, only a few confidently say yes. For many students, this is the first time either they are writing a literature review or realizing they are writing one. To improve students’ literature reviews, they now complete an earlier annotated bibliography as one of their research proposal assignments. To accommodate this shift in the schedule and not overburden students with work, I removed the assignment which asked them to evaluate the survey methods of a published research journal article. The previous assignment was useful in having students think about the rigor of survey research and issues of replication in social science research, and it also asked them to closely examine and familiarize themselves with a piece of research that could then serve as a model for their own papers. However, I believe an annotated bibliography is more useful for students in this class as they develop their writing. The annotated bibliography assignment asks students to find six to eight sources that help them answer their research question. The assignment sheet and review in class helps students understand the purpose of an annotated bibliography and the types of sources and information I am looking for in the assignment. During class, I review how to find peerreviewed academic articles using Google Scholar and academic databases. Using an example research question, we search together for scholarship. I also assign students a research article using survey research as a reading to demonstrate in class how to read and do a “heavy skim” on articles as they are searching at this stage. Finally, I provide students an example of an annotated bibliography entry, using one of my own research articles, which covers the theory, data, and findings and includes an example assessment of how this work would be useful for an example project. In class workshop time, we also discuss how their annotated bibliography scaffolds into the literature review, which comes later in the semester. As we progress through the semester, I include opportunities in the workshop for peer review and discussion, as well as reflection on the process. We discuss what was difficult or challenging in finding scholarship or in writing the entries for the annotated bibliography. As students are going to continue to find research for their eventual literature reviews, we discuss how we might take what they have learned and revise their

35

REVISING THE REVISING PROCESS OF WRITING …

423

current entries and better add more. As I was teaching the course, I found students did not necessarily realize all that they were learning, as it was not material they were studying like they would if they were being assessed through a test. Discussion and reflection built into workshop time allows students to think about where they were and where they are going in the research process, and it allows others to see shared concerns or issues in a group discussion. I find this can build confidence through the revision process so they see their improvement and have peers modeling the process as well.

Conclusion A good course is never static, and I am eager to continue to refine and develop better ways to teach survey research and writing to my undergraduate students. In future iterations of the class I plan to develop additional ways for students to think about research as a tool of communication. I hope to engage students in thinking about their audience, by enhancing their data visualization skills and presenting to new audiences, like to participants in the lifelong learning programs at my university. I aim to remind students that research is only as good as it is communicated.

References Asher, H.B. 2010. Polling and the Public: What Every Citizen Should Know. Thousand Oaks, CA: SAGE CQ Press. Baglione, L. A. 2019. Writing a Research Paper in Political Science: A Practical Guide to Inquiry, Structure, and Methods. Los Angeles: Sage, Fourth Edition. Nardi, P.M. 2018. Doing Survey Research: A Guide to Quantitative Methods. Philadelphia, PA: Routledge.

CHAPTER 36

Systematic ELA Challenges at Post-secondary Institutions: Why Many Two-Year Students Aren’t Prepared for College-Level Writing Lauren Grimes

A Sharp Learning Curve As a new professor, I was always interested in participating in as many training opportunities and educational workshops as possible. I was fully equipped with two degrees in the social sciences, but not a degree specific to teaching. When I was first hired, I assumed I would be assigned a faculty mentor, or given some magical insight on “how to be a professor.” This, of course, was the result of my naivete regarding academia. In my experiences as an undergraduate and graduate student, it seemed to me most post-secondary educators all taught using the same methods and techniques. So, surely, they must have received some sort of similar training that taught them to do so. Thus, my mission was to become

L. Grimes (B) Division of Social and Behavioral Sciences, University of the District of Columbia, Washington, DC, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_36

425

426

L. GRIMES

properly prepared to instruct my students. I was not exactly sure what that looked like, aside from the rigid illustrations I recalled from my time as a university student. But I was confident that academic training would at least help point me in the right direction. As a part of one of my professional development trainings, I was challenged to describe my teaching pedagogy. The term itself—pedagogy—was rather obtuse. It sounded so vast and broad, and foreign to my novice ears. Throughout all of my introductory training it became quite clear that this word, this practice, was critical for professors and instructors to fully understand and implement. During the exercise it dawned on me that although I had a vision of teaching students what I knew, there was so much more involved in creating a pedagogy. It required adopting a teaching philosophy, considering what teaching approaches to use, determining what forms of engagement to implement, creating instructional activities, identifying ways to meet diverse student needs, crafting ways to ensure student productivity, and constructing techniques to deliver assessments and feedback. While I was working on writing the description of my teaching philosophy, I stumbled upon a quote that summed up my personal approach to teaching and instructing in just two (Latin), or four (English), words: “Docendo discimus—By teaching, we learn.” Initially, I believed my profession had all to do with simply teaching and imparting knowledge onto my students. What I quickly realized is that the act of learning is a two-way street, so to speak. In my experiences of transferring information to others, I was learning so much from them as well. One of the most important things I gathered from connecting with my students is that there was an extremely steep learning curve for them and for me as an instructor. Over time, I came to realize this was a result of the disconnect between the needs and abilities of educators, educational institutions, and learners. Colleges and universities need to attract effective instructors and an inquisitive student body in order to operate. Students deserve adequate information necessary to fulfill their degree programs and enter their chosen career fields. Professors deserve to engage in fulfilling academic professions and have access to the proper tools needed to teach others. However, the abilities of each of these groups may not be adequate enough for their intended output. Educational institutions may be full of systematic issues that prevent them from attracting and nurturing their faculty and students. Students may have a passion or at least a general interest in their education, but lack the skills needed to perform to the

36

SYSTEMATIC ELA CHALLENGES AT POST-SECONDARY …

427

degree needed. Professors, like me early on in my career, may not be equipped with the adequate support and resources they need and deserve as educators. One of the first courses I taught when I began my career was “Introduction to Political Science.” Since then, I have taught many more political science courses at varying levels, and one common thread has consistently emerged: research methods skills are imperative for political science students. The way I arrived at this realization was quite an experience. Students often ask what science has to do with politics. They are very much intertwined. “Poli Sci” classes are heavily focused on using scientific methods to explore political systems. Political scientists do not make generalizations and assumptions about political phenomena without any basis for their claims. They must enact the scientific method, which requires a hypothesis that can be empirically verified (i.e. a presumption that is based on phenomena that can be observed), gather data, test the hypothesis, and reject or fail to reject the supposition. As students learn about various subject matter, they observe this methodology process. Then, as a part of providing them opportunities to practice these approaches and assess their knowledge, they apply the scientific method on their own. For example, instead of implementing a final examination, I typically require students to complete research papers on a political topic of their choice. Imagine a student wanting to focus their essay on lack of healthy food options in their neighborhood. They could opt to study the connection between median income and community food insecurity. A potential hypothesis would be, “When the median income of the majority of families within a community is below the federal poverty line, the community is more likely to be located in a ‘food desert.’” The student would need to collect and review data about the dependent variable (the presence of food deserts), the independent variable (family income), and other relevant information related to the current median income, the present poverty line, demographics, food deserts, etc. Finally, an objective determination would need to be made about the hypothesis and whether or not it was supported by the available data. Those steps represent the first half of their research process. The final portion involves gathering this information and formulating it into a coherent piece of collegiate-level writing. Naturally, I assumed all of my students were prepared to complete such exercises. To my dismay, many were not.

428

L. GRIMES

The Expectation Crisis My initial assumption was that every student would know how to conduct research and draft an essay. I knew those skills were necessary for any college student, and I knew it was important for students to be taught these skills, but I did not realize it would be my job to teach them. Over the course of many semesters, I noticed a number of students did not write well, lacked confidence in their writing, were unable to identify scholarly sources, and consistently refrained from citing borrowed works. Conversely, without explicitly requiring it, universities expected me to provide assistance to students that extended beyond enhancing their understanding of the subject matter. The expectation was that instructors who are experienced in a host of areas within political science would also incorporate research methods within their courses, rather than them being taught as separate courses. In my experience, political science research methods courses are not always offered at community colleges. As the research process is a critical element of teaching political science, it is then dependent upon professors to include those skills as a part of their course curriculum. This has proved challenging to me, as my purpose as a professor is to teach students about politics and how it impacts their lives. But beyond teaching about a specific discipline, teaching is a part of the way we prepare students for life. After a series of mishaps, it became very clear to me that: (1) I needed to fully accept my state of prejudice, the extent of my expectations for my students, and for that of the university; and (2) I needed to take action. Before I could take action, I needed to take a closer look at the conundrum I found myself in so that I could really unearth Pandora’s box.

Unearthing the Paradox I decided that I needed to do a deep-dive of the course offerings and educational plans outlined for students within the divisions I taught. Perhaps that information would paint a better picture of what things students were learning before they engaged in my courses. I wanted a better sense of the foundation they had developed before entering my classes. So, I began to look closer at two universities I was affiliated with. My review revealed that University A required no prerequisites for political science classes within the Associate’s degree program, as they are

36

SYSTEMATIC ELA CHALLENGES AT POST-SECONDARY …

429

treated as liberal arts electives within the social sciences. All Associate’s degree (A.A.) students are mandated to take at least two English classes— English Composition I and English Composition II, which cover general writing techniques, composition, and rhetoric. There are three writing courses open to all political science students: “Foundation Writing I,” “Foundation Writing II,” and “Discovery Writing.” The first two courses focus on writing, composition, rhetoric, and the last focuses on literature and writing. All three are offered during years one and two. There are also three mandatory research courses offered within the “research core sequence” for political science majors: “Political Research Skills,” “Methods of Political Science,” and “Senior Seminar.” Senior Seminar is a capstone course that requires a research project and oral presentation. It is identified as a writing intensive course. Interestingly, the roadmap does not recommend students take any of these three courses until at least their year third year of the degree program. From my observations of University B, I discovered that there were three applied skills courses offered that touch on research skills and methods for its associate’s degree students studying political science. It was unclear to me how many students actually take these courses, as none of these courses are mandatory. “Interdisciplinary Research Methods” is designed for students within the social sciences departments. The focus on the course is on devising research questions, formulating hypotheses, practicing qualitative and quantitative research, collecting data, and conducting analyses. Students conduct projects that invite them to practice these skills. “Public Opinion and Polling” integrates political opinion research by having students conduct a project that highlights human behavior, socialization, and opinion construction. “Political Advocacy” prepares students for political advocacy across local, state, and federal governments. Within its course description it specifically notates its emphasis on communication, organization, and research skills. There is only one other category of courses that specifically reports a focus on research within their course descriptions. Political Science students are required to take at least one of the following as a senior seminar: “Political Power: Interest Groups, Lobbying, and Public Policy,” “Comparative Politics and International Relations,” “Seminar on Conflict Theory and Management,” or “Public Policy: Selected Topics.” The latter is the only one that explicitly focuses on research within its course description. The public policy class uses a research seminar structure to discuss national and local policy-making. The major foci of the class include

430

L. GRIMES

legislation, government implementation, regulation, policy issues, and judicial intervention. None of the courses require a research course as a prerequisite. Many A.A. students take the few political science courses at community college with the intention of transferring to the main university and pursuing a Bachelor of Arts degree in the field. This transition is usually quite rough. At the main campus, courses increase in intensity and focus compared to community college courses, and transfer students are not always prepared for the rigor of the classes. That, in addition to the lack of foundational writing courses, can cause a tremendous amount of anxiety for students, and is a recipe for failure. The professors at each campus, while they share a connection to the University system, operate in almost completely separate realms. There is usually no cross-collaboration that occurs between colleagues to ensure curricula aligns in such a way that community college students are prepared for their B.A. courses.

Unintended Impact Overall, what I noticed is that there were many reasons why students were not performing well in class. Whether or not my political science courses were mandatory or offered as electives, the lack of experience with research writing prior to enrollment in college, the absence of adequate pre-requisites related to research and methods, and the imbalance in the sequence of suggested courses led to students not receiving the necessary instruction before taking my classes. In terms of the latter two concerns, the pre-requisites I reviewed did not provide students with the proper foundation in writing and research that would help them excel if they pursued a political science degree. Additionally, the sequencing either did not require research courses, or students were allowed to take such courses when they were further along in their degree program. This means the classes are not always taken in the most effective sequential order. No matter the reason students fell short of mastering research and writing skills, or the culmination of reasons, the conclusion was that students suffered, and it caused a strain on my relationship with them. Students were at risk of failing and continuing on in their studies without acquiring the necessary foundation in research and methods. They were frustrated with the grades and feedback they were receiving.

36

SYSTEMATIC ELA CHALLENGES AT POST-SECONDARY …

431

This created a slight sense of frustration for me as well as I struggled to navigate the situations I was constantly being faced with. My students had the potential to do better, but they were not prepared with the proper tools necessary to do so. My conclusion was that appropriate writing-centered English language arts courses should be mandated as prerequisites for students interested in enrolling in political science courses. For example, community college students should take “English Composition” or “Foundational Writing” courses and Interdisciplinary or “Applied Research Methods” courses before any social science courses are completed. These courses should not only cover the basic concepts of writing and composition. They should explore the application of methods through formulating research questions, collecting deriving hypotheses, understanding qualitative and quantitative research, and conducting analyses. This will create an appropriate foundation for students. Later in their academic career, they would be expected to build upon this knowledge when they take social science-specific methods courses, such as “Political Research Skills” or “Methods of Political Science.” These, in turn, should precede any high-level course (i.e. Senior Seminars).

Reversing the Trend However, until things changed at the universities, I needed a plan to address my dilemma. What was I to do with a student who copied chunks of information from another text without adding quotation marks or including in text citations? That’s plagiarism. How would I respond to a student who simply lists the websites from which they gathered information as their “References”? Honestly, they did not even attempt to incorporate MLA style formatting. What should a professor do when a student’s essay has no structure, no clear thesis, and they include their opinions (after being instructed not to) instead of factual evidence for their claims? Some may consider such a student a lazy, poor writer that just ignored set instructions. But as I reflected on the time I spent with my students in class and how they positively responded to the subject matter and engaged in class, I decided to take a different approach than to write them off as disengaged or uninterested in writing. Although, having to teach research and methods did take away from teaching the core subject matter, I felt a strong obligation to my students to ensure they learned and excelled at such skills. I recognized that the best way for them to embrace those ideals was to have students learn by doing.

432

L. GRIMES

To aid students in successfully preparing for our research and writingintensive academic program, I created a multifaceted and extensive handson learning plan. It required me to first assess each students’ perception of their writing abilities and comprehension of methods, and their research proficiencies. One by one I arranged for informal chats with them. I questioned their feedback for the course as a way to empower them and let them know that I cared about their feedback. Then I would share a bit of feedback of my own. In my experience, when I told them what I noticed about their submissions, to my surprise most students almost immediately admitted their dismay at their own work. They disclosed that they were not pleased with their writings, that they felt unprepared, and they figured at the least they could turn in work and just hope for a decent grade. Following that step, I would have each student take online plagiarism tests to ascertain their knowledge of what various forms of plagiarism looks like in practice, so that they could recognize and avoid them in their own prose. Next, I assigned writing exercises to students throughout the course from the very beginning of our time together. Each one was related to the course and designed to evaluate certain writing, research, and analysis skills. Initial assignments assessed writing skills, and those that followed built on the skills previously taught and practiced. This allowed me to determine which students had more developed writing and inquiry skills, and to identify which students needed more support. During this process, I ensured there were built-in opportunities for constant engagement with my students, and for students to connect with each other. I required students to select their topics in advance for my approval. This was not an attempt to hinder their creative license. By thinking about their research papers ahead of time, instead of drafting them late in the semester, it gave us time to collaborate on ideas and structure solid hypotheses. At times I would have them conduct peer reviews of each other’s work. This was another opportunity for them to practice critical analysis while also connecting with their peers. Students were allowed to submit assignments early for my review, as well. My annotated reviews and feedback lent a helping hand as they completed their projects. Students always appreciate tangibles they can refer to, so I also provided them with research writing toolkits. Each toolkit included resources that assisted them as they took part in the research process. I collaborated with school librarians, gathered resources from other universities, and drafted my own content to produce the toolkit materials. They included information and illustrations on how to cite information; where

36

SYSTEMATIC ELA CHALLENGES AT POST-SECONDARY …

433

and how to research Political Science topics; links to scholarly journals and Open Educational Resources (OERs); instructions on how to draft an essay; guides on comprehending the scientific method; and other related materials. As I learned more about the needs of my students, I tweaked and added to the kits. Even though some of the details were specific to the Sciences, in general the enclosed tools were easily transferable to other subjects. Due to these careful measures, by the end of each course, not only are my students familiar with topics taught throughout the course, but they are also able to efficiently research topics and draft papers showing what they have learned.

Reflecting on My Journey My journey in academia has been full of interesting twists and turns, obstacles and progressions. While my teaching philosophy of always being a learner, even as I instruct others, has remained the same, the initial pedagogy I adopted has blossomed over time. The teaching techniques I use, how I engage with students, and the instructional activities I design have matured. Everything I do is focused on embracing diverse student bodies, ensuring genuine interest and productivity, and assessing student knowledge so that I may provide the most influential learning experience. This journey has taught me how to be more resilient as an instructor. I have learned to refrain from making assumptions about what students do or do not have the ability today. Instead, through conversations with students and intentional measures to ensure they practice researching and writing skills, I have gleaned an irreplaceable knowledge of their needs. At times, it is still burdensome to weave research and methods instruction into my lesson plans. However, the greater benefit far outweighs the costs. It is an honor and a privilege to pass on knowledge to others. I am forever grateful for the opportunity to serve my students and the University at large. Over time, I am confident that schools will see the importance of adopting updated department standards that will enable professors to continue to focus on their specific expertise, and less on methodology. Until then, I will continue to advocate for those modifications and other measures that will greatly benefit students and professors alike. Academia is constantly developing and adapting, and in due time these much needed changes will certainly come to fruition.

CHAPTER 37

Teaching Research Writing to Undergraduates in Political Science and Public Administration in the Online Environment Darrell Lovell

Introduction Teaching methods and research writing has been an acquired taste at the undergraduate level. As other chapters in this volume have pointed out, progressing undergraduate students in methods can follow several different paths depending on the focus of departments and faculty teaching. There is a discussion regarding how to progress this curriculum. As academia progresses in the twenty-first century, political scientists are seeing a difficult variable added to the equation: How do we do this online? Online teaching has progressed in popularity and focus on college campuses. Institutions seeking to expand their student base have leaned

D. Lovell (B) Department of Political Science and Criminal Justice, West Texas A&M University, Canyon, TX, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_37

435

436

D. LOVELL

into the modality as a way to create revenue and accommodate students. Faculty have broken into factions. Some in the academy resist the transition online in favor of the interactive environment of traditional courses. Others embrace the online environment to advance teaching skills, expand course offerings, and—frankly—accommodate personal schedules. Methods and research writing fall into this category. Interaction and practice are hallmarks of stalwart undergraduate methods courses, especially those that are writing intensive. Active learning models are as important as ever, but faculty are being forced to replicate these approaches online. In this chapter some of the dominant barriers will be deconstructed and some potential benefits and opportunities to be effective online are presented. Concepts surrounding the need for traditional interaction, how to supply feedback that is effective, explaining and trusting students to conceptualize complex topics, and simply connecting are discussed.

My Experience Teaching methods at any level was never at the forefront of my academic plan. As a policy and management scholar, teaching methods and research writing was designed to be a limited part of my curriculum. Afterall, for my students the goal was simply to get them to give me something “passable” and leave the techniques teaching to others. Serendipitously, department needs, a want to branch out, and a thirst to help students in a specific area of writing shifted that view. As a first-year full-time faculty member at a two-year college I was approached with the possibility of teaching honors students. The conversation was simple. “Do you want to teach honors students state and local government?” It was a very quick, “Yes.” Faculty are excited to work with motivated students after all and who could be more motivated than honors students? Debunking that myth is for another edited volume. Part of the requirements of those courses, however, was original research. This created questions. “How am I going to teach political science methodology to non-majors?” “How am I going to teach political science methodology and quality writing to freshmen and sophomores?” “How am I going to teach these things and address the student learning objectives?”

37

TEACHING RESEARCH WRITING TO UNDERGRADUATES …

437

Before long that quick yes turned into a “what did I take on?” The first course involved me piecing things together as I wrote lectures on the fly that were incomplete, crammed years of methods knowledge into short bursts causing more confusion than understanding, and making the cardinal mistake of assuming students “knew” things. At any level of education and with any type of content approaching a course with this broken approach is a bad idea. For this undergraduate course it was heightened by an expectation on my end that I would “get them there” and students who quite frankly were in the deep end. One day I assumed they knew what a mean was when discussing an Analysis of Variance (ANOVA) test. I spoke for 35 min before I stopped, they were not going to stop me, and finally found out I had left them behind. All the general rules went out the window as I pressed to reinvent a wheel that was never going to roll. These missteps influenced my current approach that is designed to balance content and technique, interaction, and methodology training. The result of this switch in focus is a living curriculum born of an advanced hybrid method. While the course was not fully online, much of the responsibility for teaching and learning centered on navigating the cyber-highway. Teaching that first course offered more questions than answers. After leaning on videos and asynchronous online learning, I left that course frustrated and feeling like the great students did…well…great. The rest just came along for the ride. Since that first course, I have taught methods and research writing either partially or entirely online at my previous and current institutions. I have also made it a focus of all courses to help students build their writing skills through analysis-based assessments. I repeated that first course, and more recently as part of a bachelor’s degree program for students in emergency management administration, political science, and public administration, I taught a senior seminar dealing with both methods and writing. After three bites at the apple, online learning is still a quagmire for me where methods and research writing are concerned, and I have continued to dive into the deep end to find better approaches. However, after trial and error the need to connect through technology and for expanding on using learning management systems is more benefit than hindrance.

438

D. LOVELL

Challenges of Teaching Methods Online Students expect methods courses to be challenging. When writing is the central focus that certainly adds to the challenge. As a professor I have seen students who are incredibly talented shrink from writing due to issues with previous stigmas or personal battles with writing. In many cases my students—a considerable amount are adult learners—enter writing assignments that include original analysis with little to no basis and confidence to produce. This self-doubt is heightened by the specter of rigorous methodology and writing expectations being taught through a computer screen. While I teach my methods and writing courses to seniors now, there is a residual impact from a lack of exposure to academic writing early in an academic curriculum. At some institutions the class size or lack of faculty resources prohibits assigning significant writing projects in core courses. Without early work on writing, negative perceptions grow for students in undergraduate methods courses. Some see these courses as gatekeeper courses meant to weed out weaker students to identify how successful they will be or their level of comfort and ability with inquiry associated with their major discipline. In fact, I had an Education doctoral student just this summer question if a course that is “heavy on writing” asks if they were being “weeded out of the program,” so this happens at all levels. It was not, but these negative perceptions are common in political science, especially with undergraduate education. For many students, the way to overcome this is to connect with a professor and work through it. In the online environment that route becomes difficult. Not only are students in online undergraduate courses carrying the general phobias of methods and writing, but they’re forced to confront them using technology—Blackboard, Zoom, and Flipgrid—with steep learning curves. I often see students get into a quicksand scenario as they write themselves into holes they do not have the time or motivation to work out of themselves. Worries for students include concern that all they have is a 15-min video to unpack 80 pages of text on writing a quality literature review or selecting the right variables. Students seeking a practical approach to education—those without immediate graduate school aspirations—are increasingly prone to this concern as it takes them away from the focus of their education. Considering student concerns, I see three major barriers to online methods education:

37

TEACHING RESEARCH WRITING TO UNDERGRADUATES …

439

1. In-Depth Analysis (Quantitative and Qualitative) and Writing Development Does not Traditionally Lend to Asynchronous Models. Complex methods and theories have long been considered nuanced to a level that mandates face-to-face instruction. These concepts are believed to be translated far better in a traditional sense as faculty that have years in the curriculum focus on traditional modeling. There are also faculty that put students through the process of writing syntax for statistical programs because it is the best way to learn (thank you to my methods professors for making me do this). Barriers to interaction and intuitive class design have created a stigma around teaching methods, especially in-depth methodology, online. In fairness, the technological tools are available to improve online methods teaching, but there is a hangover effect from faculty that are unfamiliar with those tools. Now there is technology at the fingertip of faculty at a number of institutions due to student’s needs shifting in this direction, publisher and private sector influence, and pandemic response to issues such as COVID-19 that have forced spending on online learning tools. Still, the barriers and stigma of asynchronous learning’s compatibility with methodology remain. 2. Online Courses Are Perceived as Less Rigorous by Faculty. An unflattering perception of the rigors of online learning is a barrier aligned more with faculty perception than models of learning. While faculty are being exposed to online learning through expectation shifts by administrators, professional development, and simply changing times; transitioning away from a perception that online equals easy is difficult. The perception barrier emerges from faculty that have been trained in the traditional sense asserting that their way is the best way. Shifting administrative expectations and market realities forces compliance but does not remove this barrier. In fact, from faculty water cooler talk on Twitter, forced compliance may foster traditional bias that includes the perception that online curriculum at the undergraduate and graduate level is devalued in admission decisions to graduate schools and the image of the institution (Seaman, 2009; Sen, 2020). 3. The Online Model as a Method of Last Resort/Convenience for Students … and Faculty.

440

D. LOVELL

Convenience is a dirty word in traditional academia. But online learning has leaned into the value of convenience for years as faculty and students shape their view of online courses and their development in what they think the modality holds. For students, the idea lends to an “out of sight, I’ll be out of mind” model. Seeing courses online as quicker routes to degrees or program completion has bled over into individual course ideas (Early and Murphy 2009). Students expect that online courses are born out of making the trip to the finish line easier, with learn-at-your-own-pace models that package material in very “digestible” chunks. Faculty, myself included, have bolstered this perception. While students believe online learning is the path of lesser resistance, we as faculty sometimes see online models as ways to “teach without teaching.” Since they are not in the classroom, the model of PowerPoint-ChapterQuiz became a dominant view. Book publishers facilitate this view by supplying the materials making the courses more management focused than pedagogically focused. The results are students who expect easy curriculum and assessments and faculty who embolden those thoughts by designing plug-and-play curriculum.

Having Success with Online Learning in Methods Overcoming these barriers means planning to be successful and actively engaging. That is a tried and true method regardless of forum. The important job of a professor moving methods and research writing online is to do these tasks at a distance.

Demystifying the Online Classroom There is no getting around it: online learning and writing heavy courses have stigmas and there is work to be done to undo these negative stereotypes. Building in introductory materials and messages is a first step to creating a quality online methods course. Getting started materials usually deal with the do’s and don’ts of a class. “Do read the syllabus” … “Don’t be late with assignments.” These are important messages that cannot be overlooked in an online course, especially for undergraduates that are still learning quality student behavior. Those messages, however, must include taking down the barriers to online learning.

37

TEACHING RESEARCH WRITING TO UNDERGRADUATES …

441

Assure students that they are not in this learning environment alone. Provide video evidence that you exist—yes, they do want to see your face regardless of what you may think. In those videos be honest about expectations but also lay out how you will translate methods to them in a way that is digestible. Starting the semester by having students submit a video response or do multiple small group question and answer sessions of 10–15 min is a great way to clarify what online learning is in your class and how it will translate to helping them be successful especially where writing is concerned. Holding chat sessions or class town halls supply a chance to connect but also models that your methods course will involve two-way communication. Entering methods, research writing, and analysis training, I do my best to get my students writing and assure them that they will be aided in their progress and not left to the “LMS desert.”

Demystifying Methods and Writing As you undo the online myth, taking out another one makes sense. Simply put, students are not trained or familiar with many of the academic skills required for quality methods and research writing. Many of them will not be ready to take the leap into writing based on their own research rather than someone else’s published work—we’ve all read “that” literature review that is disguised as a research paper. Students understand this better than any of us. They are aware of their deficiencies in this area. Part of the early part of my methods course has to be geared to assuring them that the class will help them bridge this gap in their training rather than an exercise in exposing and punishing them for it. Demystifying research writing and methodology is the more difficult job. Students taking undergraduate methods courses have cursory knowledge of methods at best, especially at institutions outside the R1 realm—and students in research-focused programs will struggle as well. Taking steps to assure them that the class will build their knowledge base and skills is essential. Faculty also struggle in this area as many in the academy teaching these courses are doing so out of necessity, me included. While faculty have a proclivity to boast about their chameleonlike abilities to “teach any class” during interviews, translating skills to methods courses without it being a primary area of study is difficult.

442

D. LOVELL

Building buy-in for the online course that focuses on writing can be done in several ways. An early methods assignment I have used that is fun and shows the usefulness of methodologies is important. One example can be having students bring in Nielsen data ratings for their favorite TV show and find who is watching to compare the type of show and write a brief on perceived target audience versus the show’s actual viewers. Another example could be to have students review their previous 10 emails from professors and conduct an analysis on how many times they “attempt” humor to connect with students and run and analyze a descriptive review on faculty approaches to student interaction. Having students see methods work in areas they may not consider methodological will help. Another quality step towards building buy-in is to show how research writing and creating quality research is difficult. In my last online methods course I had students watch a YouTube video of a presentation of the “25 worst research ideas” that poked fun at “unique” topics and questions. It is also useful to show students examples of ways that their professions or targeted professions use research writing and methods. Connecting to a student’s major or career goal can be aided by the online environment because of the ability to assign and embed videos and examples into the course. If your course is primarily students with graduate school aspirations show them a video of a virtual conference presentation. If your course is primarily students from public administration, then show them a video and example of how frequencies and methods presentation can aide in stronger use of e-governance. For those in political science, a video of a presentation on current election data usage can be used to frame campaign messaging. Bringing methods and writing into a realm that students can connect to is vital to getting them over the next hurdle, which is learning this stuff online.

Putting Technology to Good Use---Get Acquainted with Using and Training Students on Screencast-O-Matic, Zoom, and Flipgrid As you move from introducing your course to crafting your pedagogical approaches, put technology to good use. Students will need your help. Two-way communication is a staple of traditional methods courses. It is

37

TEACHING RESEARCH WRITING TO UNDERGRADUATES …

443

even more important when writing. Engaging a professor on writing and methodology one-on-one where students can get real time feedback is vital. Professors in online courses MUST simulate that. Connecting with students is where the toys come in. As COVID-19 showed, institutions were unprepared to deal with online learning at a wide scale. Old hands at online teaching had a bag of tricks to simulate traditional interaction. Professors who were transitioning to online for the first time had to play catch up and the institutions scrambled to get licenses to give them a leg up. While these changes were made on the fly, they have added to the faculty toolbox for expanding curriculum online. Professors teaching methods and research writing need to be able to use interactive and engaging technology to create an effective interaction with students and make it a consistent part of the course. Screencasto-Matic is a gateway to this approach. The application is inexpensive individually and allows for professors to give a lecture while showing their screen at the same time. When done recording they can upload, and students can watch. Instructional videos are the baseline for research writing and methods training. Students want to see how it is done so they can put the approaches into action. Videos in a course library can also address issues that rural students or students with inconsistent access to quality internet can have with synchronous interactions and allows those struggling or needing a refresher to revisit the material. While a good start, methods work requires professors to go beyond this approach. Students will have questions. When they are working through their methods homework and writing drafts, they will have questions on what the video said and how that applies to them. Knowing that, professors in methods courses must be able and willing to engage in a synchronous avenue. Enter Zoom or WebEx. Video meeting software allows for two-way communication in real time. Students can share their screens and show professors where they are missing the connections. Students and professors can engage real time about writing a draft and crafting language to present and frame results or a theory.

444

D. LOVELL

Flip the Online Classroom---Teach Them How to Teach Themselves Faculty are some of the most creative individuals in many fields—simply look at the research being done in political science and public administration today for evidence. There are times that I read an APSR article and get the urge to take a methods course myself. As I discuss courses in online modalities with those who have bought into the method, the litany of ideas is endless. One of the main goals is to mimic something that has been done in the classroom for decades. Fostering online learning communities that allow for students to teach themselves is a key to methodology learning and building the skill of writing. Creating avenues for students to engage is important. More so, my work in online classes has shown that these students, like traditional students, will resist engaging the professor. In my courses, I have tried to move beyond the asynchronous method of discussion boards. While discussion boards hold their function, they are low level in their ability to create an environment of engaged and collective learning. Instead, I have shifted to using methods such as VoiceThread and video-based discussions that mimic traditional engagement. Students present their work and voice their feedback and build contextual engagement. Peer review builds from virtual engagement and a great extension I have found for expanding on research writing. In my most recent capstone class, I offered up peer review of the final product as an option. It shortened the semester a bit as it required the student to finish their project two weeks earlier, but it also engaged students in the critique functionality. Peer comments and reflection benefits both the writer and the reviewer as it opens dialogue regarding methods and writing processes, performance, and expands the voice of suggestion past the professor– student relationship. I used a discussion board meet up that allowed students to pair up on their own based on topic or interest… or simple friendship. The idea was to provoke engagement. Peer commenting is just one example, but it leads to the larger point: having students engage each other on material cannot be hampered by the digital divide. In a writing course, peer review is a classic and true method of engagement. Finding unique ways to use technology to increase that is a trick the online professor must master.

37

TEACHING RESEARCH WRITING TO UNDERGRADUATES …

445

Critical Takeaways and Conclusion The goal of this chapter is simple: to share the message that online learning is not a barrier to teaching methods and the quality research writing that goes with it. I aim to add to the reader’s toolbox for overcoming the stigmas of online teaching and design modern curriculum for a growing field of students. Whether a move to online methods courses is a pet project, part of an online program, or a matter of convenience for students and faculty, the barriers to online learning exist and must be overcome. Faculty and students see online learning traditionally as being easier, or less rigorous, or a means to an end approach to learning, it offers a unique opportunity to expand methods learning. I certainly did. My first experience teaching methods with an online component was a hodgepodge of “this sounds cool” ideas and head slapping when it went awry. Years later… I still slap my head a lot. The keys that I have taken away, though, are to use the online environment to my advantage. By redefining the flipped classroom, taking advantage of technology, and being forced to confront the myths of online learning and methods learning in general I believe I’ve become a more aware professor in methods and research writing. Methods and research writing present a complex curriculum that can lead an undergraduate down a road to disliking political science; it can also lead to roads of inquiry and engagement and a fervor for graduate study and advanced methods use post-graduation. Taking advantage of the latter is up to the faculty member and how they approach methods in the e-learning age.

References Early, J., and L. Murphy. 2009. Self-Actualization and E-Learning: A Qualitative Investigation of University Faculty’s Perceived Needs for Effective Online Instruction. International Journal on E-Learning 8 (2): 223–240. Seaman, J. 2009. Online Learning as a Strategic Asset. Volume II: The paradox of faculty voices: View and experiences with online learning. Washington, DC: Association of Public and Land-grant Universities and Babson Survey Research Group. Retrieved from, http://www.aplu.org/document.doc?id= 1879.

446

D. LOVELL

Sen, S. 2020. Faculty Perceptions of Online Teaching Effectiveness and Indicators of Quality. International Research Journal on Human Resources and Social Sciences 7 (5): 103–110.

CHAPTER 38

Teaching Methods in the Context of a Writing Intensive Course Jessica A. J. Rich

Jessica Rich: A Hypocrite My first two years of teaching were frustrating. I was exasperated by students’ poor writing, by their refusal to read assigned articles and, when they did read, by their inability to identify a thesis statement. Although I would never have admitted it at the time, I was in fact personally offended by these failures. Deep down, I believed students were simply not interested in understanding political-science research, or in writing good essays. Within a few months, I was muttering the kinds of phrases I had sworn would never cross my lips, such as “back when I was a student, we would never…” Looking back, the offense I took to my students was worse than unfair: it was hypocritical. In truth, my undergraduate myself had committed each of these sins. What’s more, it was almost never a lack of interest in learning that drove me to commit them. Quiet often, when I failed

J. A. J. Rich (B) Department of Political Science, Marquette University, Milwaukee, WI, USA e-mail: [email protected] © The Author(s), under exclusive license to Springer Nature Switzerland AG 2021 D. J. Mallinson et al. (eds.), The Palgrave Handbook of Political Research Pedagogy, Political Pedagogies, https://doi.org/10.1007/978-3-030-76955-0_38

447

448

J. A. J. RICH

to read an assignment or write a polished essay, I had actually spent an inordinate amount of time working on it (or staring at the assignment and surfing the Internet while paralyzed by panic). I was even occasionally offended by my professors, who had “set me up to fail” on assignments I did not know how to master. Gradually, my attitude toward students began to change. During a rare moment of calm, I would occasionally reflect on my own undergraduate struggles—remembering how many tasks that now seem so simple seemed, back then, beyond my abilities. Ultimately, I compiled these reflections into a new pedagogical mission: to demystify social-science research by giving students simple rules for reading, conducting research, and writing. Now, my favorite course to teach is “Writing and Argumentation in Political Science: The Politics of NGOs.” The title is rather unwieldy— a product of university bureaucracy—yet the description is apt. In this course I encourage students to think about nonprofits and NGOs in a new and more complicated way. But I also help students improve their research-writing skills by giving them the methods tools they need to succeed, and by giving them opportunities to apply critical feedback on their writing.1

The Building Blocks: Teaching Students How to Read Although the ultimate goal of the course is to help students improve their research-design and writing skills, I have found that achieving this ultimate aim rests on whether students can read and understand academic articles. Prior to conducting their own research, in other words, students must first be able to make sense of the research other scholars have already conducted. With that goal in mind, I dedicate time during the first several weeks of the course to teaching students how to read social science. First, I teach students a few simple rules for reading: rules for how to identify thesis statements, and for how to distinguish between key parts of an argument and supporting evidence. I start by pairing an academic article about NGOs with a “how to read social science” article. We then diagram the academic articles I assigned, to reveal general patterns in where argument summaries are placed, and in how the sections of an article are organized. Over time, I have found it often takes a few rounds

38

TEACHING METHODS IN THE CONTEXT …

449

of trial and error for students to begin correctly identifying thesis statements, and so I repeat this exercise twice, using two separate “how to read” articles that use different words to offer similar advice. Second, I help students build good reading habits by giving them time at the start of each class to take handwritten notes. For undergraduates who are taking five to six classes at a time, as they regularly do at my university, it may seem overwhelming and inefficient to write notes on every assigned article. Yet, for students to be able to retain more than the one-sentence summary of an article, note-taking is essential. After several frustrating semesters watching students fail to take notes on their reading assignments even after I made it a requirement, I came to realize that students, understandably, have a hard time navigating the trade-off between the short-term goal of completing all their daily assignments and the long-term goal of reading for retention. Ultimately, I decided not to force students to navigate this tradeoff but rather, to give them structured time in class for writing brief summaries of their reading assignments. At the start of each class, I now give them fifteen minutes to complete a single note-taking worksheet that provides space for students to summarize the thesis, describe the methodology, and add questions or comments. After I adopted this note-taking exercise, I discovered two additional advantages. First, I found that it lowers the psychological barrier to notetaking for students with perfectionist tendencies. By giving students only one page and only 15–20 minutes, students can train themselves to jot down whatever comes to their minds, without worrying about the quality of their writing. The second advantage I discovered, unintentionally, was that the exercise gave students time to settle their brains: to switch gears away from their last class and to remember what they had read the night before. After I began incorporating this exercise, I witnessed a surge in the percentage of students in class who regularly commented on their assigned readings. Only then did it occur to me that getting students to actively discuss reading in class may be as much a question of helping them focus their attention on the present moment as it is a question of getting them to complete the readings ahead of time.

Teaching Research Design I dedicate the next section of the course to teaching my students research design skills. After we have discussed a book and several articles together

450

J. A. J. RICH

as a class, I shift away from assigned readings and toward encouraging students to start independently reading, in search of their own research projects. I begin by asking students to complete a “soaking and poking” exercise, which requires them to use inductive reasoning to identify potential case comparisons; and I follow that with an annotated bibliography exercise, which requires students to use deductive reasoning to develop hypotheses about their cases. By seeing how each of these two exercises informs the other, students begin to grasp the relationship between inductive and deductive approaches to research.

Soaking and Poking I initially decided to start with a soaking and poking exercise because it seemed an easy assignment that would excite students about their research projects and leave them with an early sense of accomplishment. In the assignment, I ask students to choose 2–3 sets of two cases they might want to compare, and to define a basic research question. I require them to submit at least six pages of material, single-spaced, and I ask them to include, for each case comparison, one written page about the comparison that answers: • What question or questions will this comparison allow you to answer? • Why is this comparison interesting? • What kind of data would you need to answer your question? To guide them, I lead students through a basic lesson on Mill’s Methods, as well as a lesson on how to develop a research question. I envisioned this assignment as a clever way for students to practice and internalize the basic logic of comparison that many scholars use as the foundation of their research design. Because I do not grade this assignment based on the quality of writing but, rather, solely on the quantity of information students provide, I anticipated that this assignment would be an easy that would reward students based on the work they put in, regardless of the strength of their writing skills. In practice, however, I have found this exercise generates anxiety for many students. To be sure, some students complete this exercise without any apparent trouble, while others perform poorly because they begin

38

TEACHING METHODS IN THE CONTEXT …

451

working on the assignment the night before it is due. Yet, consistently, several students in every class reach out to me for emergency meetings at this point in the semester. Consistently, I hear from students that they “have no research question,” that they “can’t find enough information,” or that their cases are “too different.” In these meetings, I sometimes help students choose new cases for comparison. Often, I help students reframe what they see as fundamental flaws in their case comparisons. Despite the anxiety it tends to generate, I have kept this exercise in my syllabus, and I have kept it in place as the first assignment. I hang on to it because, in holding these “crisis” meetings with students, I have come to realize that many of the anxieties they face are, in fact, inherent to early stages of research. More importantly, I have come to realize that students, by working through this anxiety, develop two important transferable skills. First, when students persist and ultimately succeed in completing a research project despite their early fears of failure, they gain the confidence to work through similar moments of anxiety in future projects, in college and beyond. Second, students come to understand that there is no such thing as a perfect comparison in qualitative socialscience research, and they learn that justifying a comparative research design is as much a matter of framing as it is a matter of choosing the right cases.

The Annotated Bibliography The next assignment helps students learn how to develop a theoretical puzzle. After their initial foray into learning about potential cases, I ask students to turn back to the academic literature; but this time, with the specific purpose of assessing how well existing scholarship helps to explain the differences (or similarities) between their cases. To guide students, I demonstrate different ways their case comparisons can contribute to existing scholarship: either through creative disagreement with the literature (what we typically think of as identifying a “gap”), or through various forms of creative agreement, such as by offering additional evidence to support a claim in the literature or by extending a claim more widely. For this assignment, I ask students to create an annotated bibliography of eight to ten books or articles that relate to their topic. For each annotation, I ask them to write two paragraphs: one summarizing the argument of each article or book, and another making a statement about how the reading relates to their cases or potential research question. Finally, I ask

452

J. A. J. RICH

them to develop two to three hypotheses about their cases based on the literature they read for their annotated bibliographies. To guide students, I spend two class periods teaching them how to build hypotheses from secondary sources. To encourage students to put effort into their notetaking exercises, I allow them to include the readings I assign for class in their bibliographies. If I am perfectly honest, I had never quite understood what an annotated bibliography was until I developed this exercise. Of course, I knew how to write a bibliography; and I understood in the abstract what it meant to annotate something. As an undergraduate, however, I had never quite grasped the point of annotating a bibliographic reference. But somewhere in the course of developing this exercise for my students I had one of those a-ha moments: that an annotated bibliography is simply the early incarnation of a literature review. It is, in other words, an exercise to help students creatively identify connections across articles and assess where the literature may fall short. In fact, annotated bibliographies are quite often what students turn in as “literature review” sections of research papers. (Every professor knows, and dreads reading, that string of disconnected paragraphs listing every scholarly article that relates to a student’s project. What I came to understand in developing this exercise was that annotated bibliographies are a necessary first step in the process of writing a sharp literature review. Suddenly, I understood that I myself write what are essentially annotated bibliographies as part of the process of writing each of my research papers. First, I develop a list of summaries of articles and books related to my projects. Second, as my list of summaries grows, I start incorporating ideas about how each scholarly work relates to the other. Finally, when I draft the actual paper, I transform my list of summaries into a presentation of what I see as the main contributions of each existing body of literature, the relationship among them, and the gap I intend to fill. In thinking through my own process, I realized that undergraduate students, pressed for time, are only typically able to engage in the first step of the literature-review process. As I now explain it to my students, by drafting an annotated bibliography they are producing the content that they will then rearrange to create the “theory” section of their research paper. I explain to my students that the point of this exercise is to help them creatively identify “baskets” or “categories” of literature; to discover how those different categories relate to each other; and to draw expectations (hypotheses) about their cases based on this literature. To encourage students to

38

TEACHING METHODS IN THE CONTEXT …

453

expend effort on this exercise, I further explain to them that, through cutting and pasting, they will likely be able to use much of what they write in a later assignment that asks them to draft the theoretical section of their papers. I hold individual meetings with each of my students to discuss their annotated bibliographies and hypotheses. I do this to help students who may not grasp the exercise the first time around. But I also enjoy these meetings, because it is through this assignment that many students have their own a-ha moments. Through the process of engaging with existing scholarship, they often realize their cases indeed present a puzzle. Each semester, I witness a new spark of excitement in at least one of my students who had previously declared themself incapable of completing this project. Realizing they might actually have something to say, the students who were once the most anxious often transform into the most enthusiastic among my group of researchers.

Teaching Research Writing In the second half of the course, I ask students to apply their skills to a near-article-length research paper. However, I break the paper into its constituent parts and grade each as a separate assignment. I first ask them to draft the “case study” or “evidence” section of the paper. I then ask them to draft the “theory” section of the paper. For each section, I require students to write two drafts—one for a writing tutor, who gives them extensive written feedback and, three weeks later, another for me. I give them both a grade and written comments, and I ask students to incorporate my feedback in the final paper. By dividing the research paper into smaller segments, and by requiring students to write two drafts of each, I help students learn that good writing is an iterative process that involves writing, reflecting and, often, reorganizing.

Writing Case Studies For the case study assignment, I ask students to open with a thesis paragraph stating the argument they intend to make as it relates to their cases. I explain that the paragraph will probably explain how and why their two cases are similar or different, and I emphasize that this paragraph should also include a statement about why their explanation is interesting or different from what existing scholarship already says about their topic. I

454

J. A. J. RICH

then ask students to write a minimum of ten pages describing their cases, and I explain that they should focus these summaries on highlighting how their evidence supports the broader argument they want to make about the politics of NGOs. I implore them to think carefully about what information they need (or don’t need) to provide in order to convince the reader that their argument is correct. I created this assignment to help students overcome two problems that caused me suffering as an undergraduate. First, I wanted to aid students who, like myself, tend to get bogged down in the search for the perfect words before their ideas are fully formed. While I still struggle with this perfectionist tendency, I now understand that—especially in qualitative research—the act of analytic writing is often the same as the act of working through ideas. I now also understand (on my best days) that fleshing out the logic of an argument is an iterative process —that honing one’s own ideas through writing is its own separate task, which the writer must perform before they polish their written presentation of that idea for an outside audience. Second, I wanted to give students an opportunity to learn from the feedback they receive on their writing, and to be able to apply those lessons to new drafts. As an undergraduate, I distinctly remember the frustration I felt when I received cryptic written comments on a final paper. Comments scrawled onto the margins of my typed essays, such as √ “Why?,” “Elaborate,” or simply “!” or “ ,” were often incomprehensible to me. I distinctly remember thinking, as I would read these comments, “Elaborate on what part of this paragraph?” or “Does this ‘!’ mean something good or bad?” Often, these papers were due at the end of a term, just as students were packing up to leave for a long break. Therefore, I rarely had the opportunity to ask my professor to clarify their meaning. Even when I could understand the comments, I was never able to apply the suggestions because, with the end of the semester, this one draft of an essay marked the end of my learning about that subject. Even as I write these words, the undergraduate within me is irritated anew. “How unfair it is to give feedback without giving students a shot to learn from it!,” she whines. (In writing these lines I am, simultaneously, ashamed at how many times I have committed these offenses as a professor. A never-ending cycle of angst!) In class, I explain this assignment to students in three ways. I first describe the elements of a polished evidence section, and as illustration I have students diagram the evidence section of an article that was assigned

38

TEACHING METHODS IN THE CONTEXT …

455

for class. I then describe three common errors writers make in writing case studies: not enough detail; too much detail about their cases (without clearly explaining the relevance); and, organizationally, failing to make clear which section of the case study relates to which step in their argument. Finally, I tell students they will make at least one of these errors in their first drafts; and that this is fine because the purpose of a first draft is to “vomit” ideas and information onto paper. I explain, further, that the job of an outside reader is to point out these errors to the writer; and that it is the writer’s job it is to correct these errors in the revising phase(s). Students then receive three sets of comments on their case comparisons. The day the assignment is due, I conduct a peer-exchange exercise in class. I have students exchange papers in groups of three, giving them 15–20 minutes to read and provide written feedback on each of their peer’s drafts, and then 5–10 minutes for each student to receive verbal feedback from the group. I developed this exercise so students can compare feedback from their peers to the feedback they receive from the writing tutor. As a writer, I find it helpful when I can compare feedback because, when I receive similar comments from different readers, I often better understand the suggestions; I also tend to view the suggestions I receive as more credible when they are repeated by multiple readers. In talking with my students, I discovered another benefit to this exercise: by identifying writing errors committed by their peers, students are sometimes then able to identify these errors in their own writing. Two weeks after the peer exchange, students receive comments on their drafts from a writing tutor.2 Students then have one week to revise their drafts for me. In assigning grades to students, I compare the two drafts of their papers, and I comment on improvements as well as on continued flaws in their drafts. Typically, students continue to make common errors in the revised drafts they hand in to me—even those students who make significant revisions. In witnessing this pattern of only minimal improvements between their first and second drafts, I have been able to internalize a lesson based on my own experience of writing: that the act of revising is itself an iterative process. Just as with my own articles, which often undergo more revisions than I can count on both hands, student papers develop into polished presentations only over the course of multiple revisions.

456

J. A. J. RICH

Writing the Theory Section In the next assignment, I ask students to write a minimum of five new pages, presenting their theoretical argument and discussing how it contributes to existing scholarship. I also ask them to reorganize and revise their case studies to reflect the order in which the student presents the argument. I developed this exercise not in response to my undergraduate suffering but, rather, in response to my suffering as a professor (and occasional reviewer) when forced to read “literature reviews” that present interminable summaries of existing scholarship without clearly explaining how they fit together. To help me escape this particular form of professorial torture, I show students in class how to move beyond rote summarization and instead create their own categories of existing scholarship with the goal of identifying a gap. Together, we diagram two examples of good literature reviews from articles that were assigned earlier in the class, with the goal of identifying patterns in their writing, in their topic sentences, and in the order of presentation. Just as in the prior assignment, students receive three sets of comments: first from their peers, then from a writing tutor, and only after that, from myself. In contrast to the usual panic students experience when they work on papers that are due toward the end of the semester, students’ anxieties in this class tend to ease as they work on this assignment. By the time they draft the theory section of their papers, they have already spent weeks formulating their ideas through their annotated bibliographies and their case studies. At the same time, students are no longer confronted with the dreaded blank page: their annotated bibliographies provide them with prose to use, as do earlier thesis-statement exercises. I also find it easier and more enjoyable to critique student papers at this point in the semester. By this point, I am familiar with their research topics and have a sense of what they are trying to argue. By this point, moreover, the drafts I receive are often quite polished—sometimes quite easy to read and understand. The final paper assignment is the easiest: students stitch together the sections they already wrote and polish their writing. Reading these papers is often a joy, because I can focus on praise rather than critique, and because they are the fruits of hard labor. Many of my students leave the course proud of having accomplished a research and writing feat they had not previously thought possible; and

38

TEACHING METHODS IN THE CONTEXT …

457

they leave empowered by their discovery that long research papers are in fact chains of small, manageable tasks. Teaching this course excites and empowers me as well. I discover I don’t need to wait to “find” students who enjoy improving their research and writing; I can create them.

Notes 1. This course was inspired by and adapted from a sociology course that I taught as a graduate teaching assistant at the University of California, Berkeley, called “Global Society and Cultural Change: NGOs and the Response to AIDS in Sub-Saharan Africa” and led by the inimitable Ann Swidler. 2. Through a program in my university’s writing center, I was previously able to assign a tutor to my course. After funding for the program was cut, I began assigning my own graduate research assistant to this task.