Behind the Scenes in Social Research: How Practical and Personal Matters Affect a Project 1032386207, 9781032386201

Citation preview

BEHIND THE SCENES IN SOCIAL RESEARCH

Behind the Scenes in Social Research discusses the informal, adaptive, and real-life process of doing social science research. It complements the material in standard methods texts that describe the basics—how to choose topics and ways of obtaining and analyzing data—but in doing so miss out on many of the obstacles and practicalities of doing research. Researchers may find themselves adrift when they start their research and discover that what confronts them doesn’t precisely match exactly what is described in the basic textbooks, such as the obstacles that frequently occur, the logistical matters that must be handled, and the improvisations in research design and data gathering techniques that successful projects require. This book covers this material, while also paying attention to the ways in which the personal characteristics of those doing the research affect how projects are designed and data gathered. In addition, it explores the manner in which doing research affects the researchers themselves, affecting self-images, altering political or social views, or providing skills that extend beyond the research enterprise. Based on the author’s own experiences and interviews with senior researchers in a variety of social science fields, Behind the Scenes in Social Research explores the practical problems that arise in undertaking a research project while showing how these problems can be overcome through perseverance and improvisation. It will therefore appeal to scholars and students across the social sciences with interests in research methods and the practical issues that arise during any research project. Herbert J. Rubin is an urban sociologist, now retired. Most of his career was as a professor and researcher at Northern Illinois University. In addition, he has conducted research at a government agency and consulted on overseas foreign-­ aid projects. He is the author of Advocacy for Social Change: Coalitions and the Organizations That Lead Them and Renewing Hope within Neighborhoods of Despair: The Community-Based Development Model, and the co-author of Community Organizing and Development and Qualitative Interviewing: The Art of Hearing Data.

Routledge Advances in Research Methods

Silences, Neglected Feelings, and Blind-Spots in Research Practice Kathy Davis, Janice Irvine Social Causation and Biographical Research Philosophical, Theoretical and Methodological Arguments Giorgos Tsiolis, Michalis Christodoulou Participatory Case Study Work Approaches, Authenticity and Application in Ageing Studies Sion Williams, John Keady Behind the Scenes in Social Research How Practical and Personal Matters Affect a Project Herbert J. Rubin

BEHIND THE SCENES IN SOCIAL RESEARCH How Practical and Personal Matters Affect a Project

Herbert J. Rubin

Designed cover image: ©Getty Images First published 2023 by Routledge 2 Park Square, Milton Park, Abingdon, Oxon OX14 4RN and by Routledge 605 Third Avenue, New York, NY 10158 Routledge is an imprint of the Taylor & Francis Group, an informa business ©2023 Herbert J. Rubin The right of Herbert J. Rubin to be identified as author of this work has been asserted by him/her/them in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988. All rights reserved. No part of this book may be reprinted or reproduced or utilised in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers. Trademark notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library ISBN: 9781032386201 (hbk) ISBN: 9781032386218 (pbk) ISBN: 9781003345909 (ebk) DOI: 10.4324/9781003345909 Typeset in Bembo by KnowledgeWorks Global Ltd.

This book is dedicated to Irene Rubin, my partner for life. I dedicated my first book to her over 40 years ago and am pleased to able to once again dedicate what is probably my last book to my wife for over 56 years.

CONTENTS

1 Doing Research: Beyond What Is Taught in the Methods Class What Follows:The Flow of This Book 5

1

2 How Being a Researcher Affects you Personally Research as a Personal Growing Experience 9 Learning from the Broader Context 12 Challenging Assumptions and Changing Political Perspectives 15 Gaining Skills and Knowledge that Apply Far Beyond the Research Experience 21 Research Is Often Fun and Allows for Fun Activities 22 Conclusion and Lessons Learned 24

7

3 The Sources for Research Ideas How Academic Theories Suggest Research Topics 28 Spill Over from Prior Projects 29 Policy, Program, or Organizational Questions 30 Research Questions Emerge as Opportunities Present Themselves 32 Stemming from Your Personal Background or Experiences 35 Conclusion 37

25

4 The Evolution and Modification of a Research Design Influences on the Original Design 41 Design Needs to Mesh with Who You Are,What Makes You Comfortable 42 How You Are Viewed by Those Who You Want to Study Matters in Research Design 43

39

viii  Contents

The Impact of the Funders on Design 48 The Initial Design Is Not Carved in Stone 50 Serendipity, Luck, and Unexpected Help 56 Conclusion 63 5 Enhancing Data Gathering Methods: Learning New Techniques and Modifying Standard Approaches Why Are Some Research Methods De-emphasized? 65 Learning Methods (and Statistics) Is an Ongoing Process 67 Methods and Statistical Techniques Evolve;Though Be Concerned about Passing Fashions 70 When Standard Textbook Methods Do Not Work, Improvisation, Cleverness, and Adaptations Are Required 73 Learning to Manage a Team 76 Learning from One’s Mistakes 78 Why Are Mistakes Inevitable?  79 Common Mistakes  79 How Did I Recover?  80 Conclusion 82 6 Combatting Obstacles: Finding Sites and Individuals and Handling Logistics Access to Sites and Individuals 83 Access Is a Concern No Matter the Data Gathering Technique 83 Choosing a Site and Gaining Access to It  84 Becoming a Known Quantity  87 Letter of Introduction  88 Getting Permission in a Foreign Country  89 Getting By Obstacles 89 Handling Logistical Problems 91 Conclusion 95 7 Other Obstacles Faced in Collecting Data External Obstacles 97 Politics and Bureaucratic Games Can Create an Obstacle 98 Push Back and Attempted Censorship 101 Physical Danger 104 Ethical Concerns as They Impact the Researcher 107 Ethical Problems with Applied Research  113 Conclusion 114

65

83

96

Contents ix

8 Tricks of the Trade Facilitating Survey Research 115 Enhancing In-Depth Interview Studies 117 Reaching Out and Working with Contacts 120 Anticipating the Audience for Your Report 125 Conclusion 128

115

Index129

1 DOING RESEARCH Beyond What Is Taught in the Methods Class

Behind the Scenes in Social Research discusses the informal, adaptive, and real-life process of doing social science research. Standard methods texts describe the basics, how to choose topics and ways of obtaining and analyzing data, but often do so in an idealized, almost sanitized way. As such, researchers may find themselves adrift when they start their research and discover that what confronts them is not exactly what was described in the methods texts. What are missed are the curve balls, the obstacles that frequently occur. As a result, standard methods texts often omit guidance on how to overcome the obstacles that confront those doing real-world research. Hopefully, this text will fill that gap. Methods courses are chock-full of best practices consistent with systematic scientific approaches for doing research and that is to the good. They describe how data collection should be systematic and analysis rigorous, so that the findings are solid, reproducible, and meaningful. But in doing so, they miss some of the realities, the obstacles faced, and the improvisations that real-world research often requires. Sampling is taught, but what is missed is that appropriate sampling frames may not exist in the places where research is being done. Courses omit that during the research the social or political environment changes requiring redesign on the fly. The COVID pandemic dramatically illustrates how unanticipated circumstances can change how research is carried out. Or, texts rarely if ever discuss that researchers may have their jobs threatened by those who fear the results of their investigations. It is important to be aware of these issues and many other practical concerns to be able to design the research around them. Doing so might require improvisations in how data are collected. In standard texts, there is a tendency to de-emphasize the importance of the individual characteristics of those doing the research, their personalities, their family situations, their risk acceptance, their preferences for one research method over another, as well as their prior experiences. In addition, methods courses DOI: 10.4324/9781003345909-1

2  Doing Research

seldom focus on how the researchers themselves are changed by the very act of doing research, for example, the self-discovery involved in learning to deal calmly with those with whom the researcher is in serious disagreement. How does one cope with the long-term stress of day in day out data collection or, more problematic, living in dangerous conditions? Often missed are discussions of threats, sometimes physical, to the researchers themselves in doing research in troubled areas, in a war zone, in a prison, or in a high-crime neighborhood. Less dramatic, though important, concerns are also missed. Getting to some sites, persuading reluctant individuals to respond to questions, reconciling the researchers’ schedules with the schedules of those being studied all can prove a challenge. Obtaining resources to do the research can be problematic. Or, on a more personal level, being away from family for long periods of time is stressful. In short, methods texts omit many of the practical and pragmatic details that impact how research is accomplished. These missing issues are the subject of Behind the Scenes in Social Research. This book points out how it is possible to adapt to and overcome practical and personal problems that arise during the research. The examples provided of the problems, and often their solutions, build upon the experiences of this author along with those of several dozen senior researchers who were interviewed. These experienced researchers were chosen from a variety of disciplines—political science, sociology, sociology work, planning, anthropology, as well as those with experience in economics and business—as the problems faced and solutions are found in all the social sciences. Behind the Scenes in Social Research explores the practical problems that may arise as well as the ways in which these problems were overcome through perseverance, cleverness, and improvisation. It details how their personal characteristics of the individual researchers impacted the ways in which they went about doing research and conversely how doing the research impacted them individually. Their stories offer examples of what others should look out and solutions that can be emulated. All those interviewed for this book have had impressive research records. While many were ethnographers, that is those using observation and depth interviewing as data gathering tools, others relied on surveys, analyses of governmental statistical data, and lab experimentation as their methods for collecting data. Several used elaborate and sophisticated mathematical models. The topics of their studies varied greatly from those studying political parties, health care of minority groups, nomadic sea wanderers, development in low-income countries, psychiatric institutes, urban renewal, business development, industrial workplaces, sexual behaviors of students, policy making at the federal level, life in a psychiatric hospital, the impact of mining on rural communities, and the dense and crime-ridden slums of Brazil. Some worked in the mountains of Peru, others in Washington D.C., while others conducted surveys, ran social psychological experiments, or immersed themselves in examining government data while remaining at their home universities. No matter their research topic or data gathering technique, these individuals provided numerous examples of unexpected problems, as well as the joys and

Doing Research 3

benefits of doing research. They described how often they had to improvise in their data-collection techniques to overcome anticipated and unanticipated problems. They lamented that many of the methods courses they had taken omitted the specific data gathering technique required for a particular project, necessitating their taking time to learn the material on their own. They shared their experiences in handling the basic tasks required in collecting and analyzing research data. They made clear the importance of understanding the logistics of doing a project—how does one get to an island when there is no regular boat service; how to keep up one’s morale when many refuse to be interviewed; or accommodating the work to personal foibles, such as being shy yet having to contact and then interview strangers. They talked about how their own personal characteristics and needs influenced the research. For example, how did being female in a sexist environment impact the study? Or, how did the need to balance family and research affect the way the research was designed and carried out? They shared memories of mistakes made and ethical ambiguities faced with more or less success. Many pointed out how luck, helpful strangers, or serendipitous events shaped what they did. The interviewees indicated how doing research impacted them on a personal level. Some made it clear to me that doing research was a rite of passage, a growing experience, and a source of both fun and self-discovery. Exposure to unfamiliar cultures, different organizations, or people outside of the researchers’ experiences impacted their personal understandings, sometimes moderating their behaviors or even changing their political perspectives. Their overall sentiments on what is often missing in standard research methods course were summed up in a single sentence by one of the interviewees. “No matter how much you have learned in the classroom you really have to use your common sense when you are in the field. Many things texts don’t teach you about.” Throughout this book, I will merge their numerous examples and insights with my own research experiences. So, who am I and how and why did I decide to write this book? I’m a retired urban sociologist; now in my late seventies. For most of my career, I was a teacher and academic researcher, though I spent a year doing research at a federal agency, was an applied researcher half time for several years at my university’s research center, and later undertook a month-and-half-long consultancy for a U.S. foreign aid agency in the Philippines. Among other courses, I taught research methodology, both quantitative and qualitative. I authored (or co-authored) several research methods texts. I have been engaged in a wide variety of research efforts, varying from my dissertation done in an isolated area in rural Thailand, research on community development in poorer neighborhoods in the United States, land-use disputes in suburbia, urban economic development, community organizing and advocacy, to a decadeslong study of progressive advocacy organizations in the nation’s capital. I was a bit frustrated while teaching research methods that I had to spend so much time explaining technicalities—how to word a survey question, or draw a

4  Doing Research

sample, or conduct an open-ended interview—that I never got around to talking about the practicalities, the frustrations as well as the excitement of doing research. In retirement, I had a more time to reflect on my research experiences. Looking back, I remembered the practicalities I dealt with, such the need to draw water from a well in a rural village (we bought a pump), lack of refrigeration (we smoked meat over a cooking fire to eat later in the day), and the dog that lived in the house we rented who stole meat from the butcher, who wanted us to pay him for his loss. I had to manage interviews in a local dialect (I hired some young people from a language school) and learn to do research with a team. I remembered the frustrations, such as contracting hepatitis from contaminated food, and the dangers, having guns pointed at me and avoiding snakes, and wondered at my ability to adapt, to cope, to invent methods on the fly. I fondly remembered walking the halls of congress while studying D.C.-based advocacy groups or being appalled at the neglect of the poor neighborhoods that the community development corporations that I studied worked to improve. I was disappointed when I learned that people in housing cooperatives did not cooperate with one another. I took pleasure in the travel required to interview people in many different sites. And, I looked back with joy on the satisfaction of work well done and years of growth in learning about matters totally new to me. Nowadays, I consider myself an experienced depth interviewer and have coauthored a book (with my wife) on qualitative interviewing. Yet, in my years in graduate school, I never had training about in-depth interviewing, so I had to teach myself. No one had showed me how to set up appointments or get permission from a person or an organization to do a study, much less figure out how on a limited budget to coordinate schedules of very busy people whom I wanted to interview with the times I could travel without neglecting my teaching responsibilities. I love to interview but had not been prepared for the fatigue of day in, day out interviewing followed by a nighttime firming up notes. I had been taught how to carefully prepare a research design of what I intended to do with what methods and with whom only to learn that designs are not carved in stone and need to be modified in real time while a project is well underway. I also remembered the numerous mistakes I had made in my research. I recalled an amateurish survey I had prepared while working at my university’s applied research shop, a survey I inflicted on some mayors. Or, the embarrassment I felt when I thought I was recording an interview and never pushed the on button. I thought about my long-repressed reaction after I learned that the person who helped me most in one project committed suicide. Had I missed signs that he was depressed when I repeatedly interviewed him? I thought about how much help I had received along the way, much of it from people who had been strangers prior to the projects—a provincial governor in Thailand who gave me permission to work in an area usually banned for foreigners and the frankness of D.C.-based advocates in describing their encounters with the Washington power structure.

Doing Research 5

Thinking about my own research adventures made clear to me many of the realities of doing social science research that were not anticipated in research methods courses I had taken or, even more embarrassing, taught. I was aware of what nowadays is known as positionality theory, i.e., who you are and how you are seen, impacts the research endeavor; nowadays often taught in methods classes. It was obvious to me that being a middle class, Jewish male would affect the work I was doing in minority communities. What I had not anticipated, and had not mentioned in methods classes, was how doing research would impact me as an individual, changing how I saw the world and how I saw and understood myself. After a few successful research projects, I began to gain some confidence in myself, a confidence that extended to my personal life as a local political activist. I learned that university life that I knew and at been part of for decades was not the whole world. In listening to people in different jobs and locations with different perspectives, I moderated some of my own political opinions. In general, my reflections, along with those of the interviewees, made it clear that social science researchers are not white coated automatons but rather individual human beings. Who you are, your background, ethnicity, gender identification, and what questions are of interest differ from person to person. A statistical test is the same no matter your background, but what one chooses to research and what skills one brings to the enterprise vary with each individual. In doing research, who you are counts. I regret that I did not know what the individuals I interviewed for this book shared with me earlier in my career. I would have avoided many mistakes, not panicked when obstacles occurred, and learned many decades earlier that personal factors impact the research enterprise in numerous ways. Hopefully, this book will help the reader learn these lessons earlier on without waiting half a century as I did.

What Follows: The Flow of This Book One of the most important lessons of this book is how your background, history, and personal characteristics impact how a project is carried out. This theme is consistent with positionality theories that reflect the ways in which personal characteristics influence how the researcher is viewed by those being studied. But what is missed is how doing research impacts researchers themselves personally both emotionally and intellectually. I felt that understanding how research impacts you the researcher personally was important enough to place it early in the book in Chapter 2. The rest of the book examines step by step the realities of doing research— the pragmatic concerns, the personal elements, logistical constraints, doing so by building upon and then extending what is taught in most research methods books. Methods texts emphasize that research begins with testing an academic theory or filling in puzzles or missing areas of prior research. That is often the case. But, there are many other sources of research ideas a topic covered in Chapter 3.

6  Doing Research

Fruitful and intriguing research ideas emerge from personal experiences or family histories, are assigned in contracts received by a think tank or consulting firms, or are suggested by the need to solve a problem a government organization, business, or social agency confronts. Sometimes, good research ideas emerge from intellectual curiosity about a news story or even a chance conversation. Chapter 4 begins by repeating the important lesson contained in most methods texts, that is the necessity carefully preparing a research design. But then, the chapter argues that even the best designs are not carved in stone. Initial designs evolve, sometimes dramatically, as a project is underway. Flexibility is required in working out the initial design to allow for these changes. At times, it seems to me that what is later described in a report as the research design is what the researchers ended up doing, rather than what they started out thinking they would do. In undertaking a project, researchers rely on a variety of social science data gathering tools. The first methods course taken in colleges usually provides an overview of this variety of these tools with more advanced courses focusing on one or two data gathering techniques in greater depth. Still, some techniques may be ignored or glossed over. Researchers can end up having to teach themselves new techniques or those omitted in courses while their research is underway. Often, they may have to adapt existing techniques or perhaps invent new ones when they find the situations they confront don’t match the conditions outlined in methods books. These adaptations are the topic of Chapter 5. Chapter 6 discusses the most frequent obstacles that arise during the research, such as gaining access to research sites, convincing those whom you want to study to participate, and managing logistics such as travel, finding places to sleep, hiring help, working in teams, maintaining health, as well as physical and mental concerns of day in day out data gathering. In Chapter 7, I examine less frequent but still daunting obstacles that can occur mostly while research is underway. For example, exploring how changes in the social and political environment impact what it is possible to study, where research can be conducted and how data gathering might be impacted by these external factors. Researchers may encounter push back and censorship of their work or at an extreme, though not infrequent, physical dangers. Methods courses appropriately emphasize ethics; but in real-world research, ethical dilemmas are never as clear as they seem in the classroom. Examples of these more frequent but less often discussed ethical quandaries and how some were resolved are also included in this chapter. Chapter 8 offers an assortment of tricks of the trade, that is, ways in which experienced researchers have innovated to handle the numerous, often unanticipated, problems they face. Some of these tricks of the trade were responses to the researchers’ own mistakes; having caught them, they figured out how to fix them and prevent their reoccurrence. Most tricks of the trade that the interviewees shared are ways these experienced researchers have learned over time to cope in real time with recurring problems—getting individuals to talk on sensitive matters, convincing those in high-level positions to provide their perspective, and, most importantly, establishing and building upon personal contacts that facilitate doing the research.

2 HOW BEING A RESEARCHER AFFECTS YOU PERSONALLY

A core theme of this book is that the researcher’s personality, interests, and abilities are vital in shaping the research experience. Later chapters describe how personality and personal characteristics affect the ways in which research is done, the choice of topic, the overall design of the research, and what data gathering techniques are preferred. This chapter explores the other side of the process that is the way doing research affects the researchers in a quite personal and individual way. Engagement in a research project has multiple impacts on the individual. To begin with, for the novice researcher doing a project becomes a rite of passage, a growing experience. Successful research builds self-confidence, an awareness that, yes, I can manage complicated intellectual projects and come up with meaningful results. This increased confidence extends over to daily life. Research brings about introspection about who you are and what you want to achieve. I asked myself who was I, as a 25-year-old American, to make judgements about experienced Thai bureaucrats? Do I have the hubris to write a book after only a year-long study about people who have spent their lives rebuilding deteriorating neighborhoods? Why did I study social change organizations, greatly admiring the efforts of these organizations, but then not quit my academic position and work with them when they offered me a job? I’ve worried about these matters for decades. While the purpose of doing research is to obtain answers to specific, usually predetermined questions; in the very process of collecting data, the researchers learn about matters having little direct connection to the study itself. This incidental experiential learning can expand the researcher’s understanding of matters having little to do with the immediate project, encouraging an escape from a preexisting intellectual silo. Doing a project in a foreign country or at home among a cultural group different from your own teaches you a great deal way beyond what is being researched. DOI: 10.4324/9781003345909-2

8  How Being a Researcher Affects You Personally

In my case, in Thailand I learned about Buddhism and about mosquito-borne diseases, and the importance and scarcity of clean water. Drawing water from a well and carrying it home from the local monastery were lessons I have never forgotten, showing me the difficulties of living in a place lacking in everyday conveniences. In the United States, while researching how organizations redevelop lowerincome neighborhoods, I visited a day-care center built by the redevelopment organization I was studying; learning about that organization was core to my research question. But, I learned much more. The director of the day-care center pointed out the bullet holes in the wall, providing me with an unexpected lesson on the meaning of gun violence in a low-income community. In doing a project based on available data maintained by governmental agencies, a researcher can learn something about the agencies that collect and maintain that information. Early in my career, I wanted to examine zoning records that were kept at city halls; my research was on zoning cases not on government transparency. In one city, I asked for the documents and was escorted to the files, told to take whatever I needed home so long I left a note of doing so and returned them. In contrast, in another city, I had to go through a complicated procedure even to see these public documents. In seeking out these documents for my project, I learned that some local governments are much more open than others and became sensitive to differences in openness of various government agencies, something that I pay close attention as an activist with local governments. When wearing your research hat, you must learn not to argue with people whose opinions you find repugnant; you simply listen lest you offend those whom you are studying. While that was difficult for me, learning to listen before opining helped me over the years to interact with friends, family members, and on the various city committees on which I served. Yes, in these situations, unlike in a research interview, I did present my own views but only after respectfully listening to those of others. Carrying out a research project sometimes requires learning skills that carry over to activities having little to do with the initial research. In my study of a land-use controversy, I had to learn how city planners worked and that knowledge was immediately relevant when as a citizen volunteer I served on my city’s planning board. (Learning how to vaccinate a water buffalo while doing research in Thailand was less useful in my later life.) Finally, doing research can provide opportunities for fun. Ethnographers enjoy talking with people and those of a quantitative bent get a personal kick out of ferreting out patterns from what initially looks like a hodgepodge of numbers. Getting data and playing with them to figure out what they mean should be fun; research can involve intriguing puzzle solving. But in addition, research both allows for and requires time for fun, for socializing, for entertainment, and tourism. You should not try to keep up a continuous stressful pace of research without taking breaks, less one ends up rushing and misunderstanding what is being found or burning out before a project is done. These necessary breaks can lead to relaxing and enjoyable experiences. If not for

How Being a Researcher Affects You Personally 9

a research project that took me to Washington, D.C., I would not have been able to explore the numerous (and free) museums in that city between interviews and at the end of the day. The rest of this chapter expands on ways that doing research impacts researchers on a personal level, beyond the excitement of the discoveries made during data gathering.

Research as a Personal Growing Experience I grew up in a lower middle-class neighborhood in a racially homogeneous, socially liberal town. In my high school class of over 500, there was but a single African American. I had never traveled. I was a bookworm, a good student. I had studied mathematics until I started a doctoral program in political science. I was mostly apolitical, although the Vietnam war had made me aware of the importance of politics. I lived in a very pleasant silo. As a researcher, I was a virgin. I had almost no experience or first-hand knowledge or even second-hand knowledge of the world outside of my silo. That began to change in the graduate school. As part of a class project, I did a mini study talking with teenagers in an African American community about the racist actions by the Boston school board. I remember walking through the community and spending many hours talking with younger African Americans about their lives. I had read about racial discrimination but talking with those who were victims was an eye-opening experience, well beyond what I learned about the reaction to the Boston school board. For my doctoral research, I ended up in a remote area in rural Thailand, with only one summer of language training. I was suddenly immersed in a foreign culture, with inadequate language skills, and had to manage transportation, negotiate access, arrange a place to live, and learn how to shop in a place that required bargaining. I had to learn how to cook food over a wood-burning stove that looked like a clay flowerpot with a hole near the bottom for the removal of ashes. I had to hire bilingual people as interpreters and learn to interview through them while maintaining eye contact with interviewees. At home, I was critical of the U.S. policy in Vietnam; but in rural area of Thailand, I was seen as the American, the first many local residents had ever met. I suddenly found myself representing—and sometimes defending—the United States, a change that required reflection. It was one thing for me to be critical of my country at home, but apparently something quite different for someone in another country to do so. Rather than being treated as a student, I was seen as a foreign guest, a doctoral student from a prestigious university. I had gone from near the bottom of the social hierarchy at home to being treated with respect (and kindness), and was then responsible for the research and the ordinary aspects of maintaining a household with minimal electricity, and no running water. I had to adapt real time to totally new experiences. The analogy often used in such situations is—it is like drinking from a fire hose, in other words, just

10  How Being a Researcher Affects You Personally

overwhelming. But somehow, I did it. Over time, my self-confidence increased. I could do research and when necessary, could improvise when standard research procedures were not possible. While in Thailand, I learned how to talk with strangers and got a lot of practice in open-ended interviewing, something that I had not been taught in school. I had now traveled internationally, lived in in a non-English speaking country, learned to live in a totally new environment, and had solidified a young marriage. As a result of the research in Thailand and several projects, I became less of an academic snob. Previously, my world had been that of the highly educated and degreed people in my schools. During my research, I met, often interviewed, many people with far less education, who were wise, informed, and taught me much, all without textbook learning. This started way back in Thailand, when in an isolated village I met a barefoot doctor, a local farmer trained in simple medical procedures and medicines. I chatted with him frequently. I believe he had four years of formal education, but I learned more from him than I had learned from many of my teachers. I certainly was not alone in experiencing research as a growing experience. One interviewee described a project that had a profound impact on her identity and self-confidence. Somewhat shy, she had done little travel and had never managed basic tasks such as finding an apartment, signing a lease, and handling her own finances. Yet during an early research project undertaken in Washington, D.C., hundreds of miles away from home, she found a place to stay, handled her finances, and negotiated access to those officials, some in senior positions, whom she needed to interview. Doing research was a profound growing experience for her. She described The six months in Washington was … a learning experience. It was the first time … that I had lived on my own and that involved a whole set of tasks and capacities that I had never exercised. It gave me confidence that I was able to do it. I still look back and wonder … I’m basically a shy person and that I was able to make the contacts with people high up the bureaucracy and sometimes up into the political level of government, to make the contacts, to set up the interviews and go and do them. I still look back on it with a great deal of awe. Another person, a China scholar, detailed his own rite of passage as well as the increased self-awareness resulting from his involvement in overseas research: as far as that sense of independence and being completely being on my own in a place where English was not spoken and just diving into it. I learned a lot, I grew up a lot and I think I matured a lot. I’m trying to think more outside of myself. …. So, every time I came back I felt I was learning and growing more … Learning about how to listen and observe … And again, it wasn’t from what I got from my university in America. …

How Being a Researcher Affects You Personally 11

I remember …. I was staying with a villager and they don’t have water in the house and they have to take an outside bucket and go to the well area. So I was helping bring the water in and met an old guy there at the well and I said it was hard, life here is really tough and he just looked at me and said are you out of your mind …. what the hell is the matter with you. Life is so much better now, my grandkids are going to junior high school. We have electricity. … We are able to go to the market at least once a week, this is incredible it is so much better than it was when I was growing up here. You don’t know shit. … So, I just learned a lot from just listening and trying to observe and also learning not to compare with my own experiences but try to figure out what their experiences were now compared to what their experiences were before. Those were things I think helped me grow as a person more than just an academic. Similar sentiments were expressed by another political scientist. … I think every project that I’ve done in one way or another has helped me grow as a person. Not just as a scholar but as a person. I might not have been cognitive of that at the time but over time, your research I think can indeed have direct impacts on who you are a person. Another example was shared by person who studied sea nomads and found that her work taught her to understand: how far you yourself can go in terms of perseverance in terms of the ability to adapt. … you simply go with the flow of things. But when you look back you realize that it takes quite a bit of stamina. There were physical challenges, you know, things like you don’t have the kinds of bathroom that you are used to, and you don’t have the kind of electricity that you are used to and all of that. A sociologist echoed a similar perspective as his studies taught him: a general perseverance. Research creates a lot of obstacles. You have these goals you want to accomplish. And they are not easy in many cases and it requires perseverance. (now) I often sort of try to envision that the end product. I know where I want to go, how do I get there? How do I get this cooperation? How do I solve this problem and there are a million little decision points along the way in getting there, but its just a matter of persistence? Those comments resonated with my own overseas experience in which I had to handle difficult and unanticipated situations, such as the time local wells were polluted and the time when neighbors anxiously pointed at the poisonous snake

12  How Being a Researcher Affects You Personally

in our yard to get our attention. Then, there was the time my wife was accused of adultery because we signed souvenir photos with love from both of us. Somehow my wife and I were able to cope with the missteps, the misunderstandings, and the dangers. That knowledge that we could cope extends to the normal (and abnormal) vicissitudes of daily life. If I could cope with problems in a remote village with minimal infrastructure and few, if any, ways of getting help, I certainly could cope with events at home. In general, through doing research, you enter at least a bit into the lives of others, recognizing that your world (or world view) is not the only one and that people outside of academe are skillful and knowledgeable, and you routinely deal successfully with a world in which you are a novice.

Learning from the Broader Context In the process of doing research, you are exposed to social, economic, political, or cultural matters that are informative, though often not directly relevant to the specific questions you are researching. My Thai experience exposed me to a Buddhist society, with its underlying assumptions of impermanence and suffering, so different from my own world view. I was exposed to corruption and its consequences; while previously at home, I didn’t operate at a level where I would have encountered it first-hand. I also saw how honest public officials were able to negotiate change, a lesson in optimism that has stayed with me throughout life. Though it was not a direct focus of the study, I learned about reinforcing cycles of poverty as I saw small tractors introduced to the area. Those who could afford them would eventually displace the farmers who were still dependent on water buffalo to plough their fields. Those who were poor would be poorer still. This insight, too, has stayed with me, as applicable at home as it was in Thailand. I experienced a little of what it was like to live in a country where the military have a disproportionate influence on civil society. I shared the fear of the local Thai officials when military representatives from the central government came to the remote area of Thailand. Since those days, I have paid attention to the extent to which our military remains under civilian control, the extent to which our military is deployed domestically, and to the size of its budget compared with domestic priorities. I like to think I am a more responsible citizen at home as a result of what I learned in doing research abroad. In the states, I did several studies on economic development in middle-sized communities. In carrying out these projects, I attended many city council meetings, where economic development, the purpose of my study, was only one of the topics on the agenda. From this frequent attendance, and by staying at the meetings long after the economic development issues I was studying were discussed, I gained a broader understanding of the dynamics of decision-making at the local level and was exposed to the large variety of issues that local governments face. As an indirect result of my studies on economic development, I ended up becoming a local government junkie and, years later, wrote a book on the matter.

How Being a Researcher Affects You Personally 13

Similar incidental learning occurred in my U.S. projects in Washington. As a political scientist, I knew about lobbying in general; as a spinoff of my D.C. advocacy study, I got to watch lobbying negotiations first-hand. I discovered that the phrase “walking the halls of Congress” was more than a literary metaphor. The apparent openness of government to lobby groups helped me understand what democracy means. Not all lobby groups or coalitions of groups had equal access, which further shaped my idea of what democracy means in practice. A chance observation while doing this research in Washington made visible the imbalance of power between business and progressive organizations. The head of one progressive organization that I was studying was a small woman of modest means whose clothes reflected her income. One day, I was standing outside of an elevator in a congressional office building. The elevator opened and I saw her heading to a hearing along with an elevator full of taller men, all dressed in Brooks Brother suits, each representing organizations opposed to what the progressive advocacy organization was asking Congress to do. It was a visual portrait of the imbalance of power in D.C. that has stuck with me over these many decades. While I was an employee at the Department of Housing and Urban Development (DHUD) in a research shop, a senior political appointee in the agency asked me to spy on my division. Stunned and unwilling to comply, I said, “I’m not really a permanent employee; I’m just on loan from a university.” Because I was only there temporarily and would return to my university, I had the luxury of behaving in an ethical manner, but what about his subordinates; would they feel free to defy an unethical request from a political appointee? I learned something about the tensions between career officials and political appointees, and the pressures to conform and obey even unethical requests or commands. A male criminologist (gender is relevant) shared with me an encounter that expanded his knowledge of prison life well beyond the topics on that he had been researching. As he told me the story decades after it happened, he was still shaken and perplexed: Oh, I just have to tell you, this is one of the weird things that happen. They (prison guards) do what they call pocket searches … maybe a third of the time, put their hands in your pockets from behind and on one occasion this female guard officer … she put her hands in my pocket from behind and started to massage my genitals, … I went back and told the guys (the inmates he was studying) what happened. They didn’t think it was funny. They said, you think it was a joke. Now you know what our loved ones had to go through every time they came in. And if this happened to you, someone quasi-official, just think what they do to the powerless, with cavity searches. That was a lesson (for me); always remember the context and where you are. Sometimes the information that you pick up during research modifies, perhaps dramatically revamps your assumptions and predispositions about other people.

14  How Being a Researcher Affects You Personally

One of the interviewees for this book described an experience that shortcircuited his assumptions in this case about a liberal politician. …. When I was at the civil rights commission, the city of Cleveland enacted its first affirmative action plan. … so we set up a series of interviews with people [to talk about the plan]. So one of the interviews … was this guy George Forbes who was the President of the City council, well known and a major civil rights leader …. we knew going to talk to George Forbes that he was going to be very supportive. Now the affirmative action plan had goals for women and for race. When we walked into his office and sat down, the first question was some generic question about what do you think about your city’s new plan. And I’m there with a white woman who is a lawyer and African American woman who is civil rights (expert). … he looks at the white lawyer and puts his finger almost right in her face and first thing out of his mouth was I ain’t got time for you white women…. When we walked out of there, we were kind of stunned …. For the researcher, this encounter was a side lesson on the complexity of race and gender relations. Later in her career, a scholar who studied city planning in American cities ended up in Singapore and, while there, began researching the government’s approach to planning. In doing this project, she could not help but notice how this wealthy, modern country treated foreign workers: … the thing they (Singapore government) do worst is the treatment of foreign workers who all got covid … if you look at their plans … there is somehow no place in the map of Singapore in 2030 or 40 or whatever it is that shows even where these workers live; they don’t exist, and all the data you get from the government, it’s about the permanent residents and these contract workers are simply not included in their data. Her views on Singapore changed as she learned how they treated “guest workers.” She also learned to pay attention to what wasn’t there, since the absence of information about the guest workers was revealing. Another researcher, while interviewing public officials on local budgetary processes, described that after observing a poster on the office wall of an official, she picked up a secondary lesson that decidedly increased her empathy for those working in the public sector: One of the assistant city managers that I interviewed had a poster on his wall that said “I’m a mushroom. I know I am a mushroom because they keep me in the dark and the pour shit on me” and I realized for the first time what it felt like to be an assistant city manager, what it felt like to not be informed, how one extrudes and distances within a bureaucracy where

How Being a Researcher Affects You Personally 15

everybody is supposed to be working together, what that felt like. It wasn’t something that he intentionally conveyed to me. It was his self-image; it was on the wall. That insight was irrelevant to her research, but also impacted the way she later on taught her courses on public administration as well as how she better understood the concerns of subordinate public officials. Some side lessons learned while relevant to the research at hand, also extended far beyond the immediate research project. The scholar who studied Vietnam (during the war) described two such lessons, germane to his research but later affecting his broader understanding. The first was on the use of code words in language: You have to learn whenever somebody says the word ‘just’ they are covering something up, like asking a kid what were you doing in the library, we were ‘just’ studying. or when somebody says we just thought Ho Chi Minh was just a nationalist, they don’t want to talk about the other stuff. It’s like you learn certain words are really code words, like “oh it is just natural’ or ‘just happened’ or that’s all I wanted to be. Whenever somebody says ‘just’ the ‘reason is just x’ they are saying I don’t want to acknowledge anything but x. The other lesson involved his gaining a broader understanding of problems with U.S. foreign policy, brought about by his reflecting on his research in Vietnam. Whether it is military or foreign aid projects trying to change the power structure and the social structure, the single most important thing I learned … was that baby countries don’t have baby politicians. And there was this sense that we will teach them how to be good, as somehow we will teach them to be better people. Missionaries? [That is …] not what we were there to do. It was not about educating them it was about changing the incentives when we could. These two lessons learned during his research in Vietnam impacted his outlook on aid policy long afterward, when he became a political consultant advising candidates on a wide variety of topics having nothing to do with Vietnam.

Challenging Assumptions and Changing Political Perspectives One’s experiences doing research can challenge personal assumptions about policy and politics and move one further left or right, or closer to the center. With an in-depth look at yourself stimulated by researching others, you may be able to see the flaws in your own previous perspectives.

16  How Being a Researcher Affects You Personally

For me, an early research experience was seminal. I was concerned about how urban growth reduces open space and was studying a dispute between environmentalists and local government officials on that matter. I interviewed government officials who wanted to develop a large tract of open space for housing and commercial use, as well as the environmentalists who opposed the development. I found that the public officials (whose plans for the land I strongly disagreed with) were willing to answer any question posed, doing so in what seemed to be an open and honest way and quickly shared data when I asked for it. The environmentalists were more distrusting and less open, provided far less information and acted in a defensive way, even though it was clear I supported their side of the dispute. I understood why they may have become more distrustful as they had been burned by those in the public sector as well as by the press. Still as a result of that experience, rightly or wrongly I ended up becoming more cynical about environmental groups and more supportive of those in the public sector who see growth as a way of increasing the revenue needed to provide public services, something that the environmentalist ignored. A researcher doing work on the environmental impacts of mining also came away with a more negative impression of those protesting mining people with whom he had initially agreed: My research experience … shifted my politics which is a weird thing. And my politics I think since shifted back to a certain extent. When I was … doing the research … one of the things that stood out to me was how little the people who were opposed to the mineral development, how little technical understanding they had about what they were opposed to. … I agreed with them that mining was problematic … but I was really surprised by how little understanding they had … how scripted everybody’s responses were, how uniform everybody’s responses were and so I developed this sense … that the rhetoric was really being sent by outside advocacy groups … these north American and western European activist organizations were doing a lot of … workshops raising awareness about the dangers of mining, and I got the sense that the folks in these (village) communities really weren’t making truly informed [decisions] on their own. They were being subjected to essentially left-wing propaganda. … So, I wrote in my dissertation … this idea that foreign activists were replicating a sort of essentialist noble savage imagery of the folks in the communities and that this was not helpful. … it reinforced sort of North American white hegemony and inequality. I asked another expert on South America if her research changed her political perspective and behavior: I think the answer is yes … When I was looking toward grad school I was looking to do research that would, I thought, would have some meaning and explore certain questions about social justice or inequality. And,

How Being a Researcher Affects You Personally 17

so when I went to do my doctoral work in Peru I was looking at small scale marketers when [they were] in working in market places, often quite impoverished … when I was there … it was … the era of austerity and it was a year when the IMF was insisting on certain measures to cut back on inflation and let prices rise, depress wages, cut back on social services all those kind of things that typically are part of the austerity package. … And, so I think that kind of experience was something that I was quite open to and my ears were you know [attuned to] that kind of perspective and I certainly duly noted it. Which gave me more determination when I came home to continue to explore questions of social justice … and questions around race became more important in my work over time. … I began to see more of the ethno- racial identity issues and how they continue since the time of colonialism … in holding some people back …. I think what I took from the field and returned home with was a deeper commitment to that and a global perspective that would be very hard for me to have without ever leaving home …. Others found that their research tempered their political views. One interviewee was a former classmate of mine, who grew up in an era when those of the left strongly advocated for mass participation and community control of school boards. She agreed that her research: certainly has affected my political beliefs. …. I was really committed to the idea of community control. … I would see whether community control affected the learning of kids … Well, I started going to meetings of  … citizens’ group …. which would make recommendations to the school superintended. … subsequently I’ve seen many of the difficulties, the problems of participation. … looking at populism at the local level and how that affects decision making and the way in which in fact participatory mechanisms allow these people who oppose anything that would make their community more diverse or oppose efforts to deal with climate change, they are empowered. And, so having more community participation is leading to policy stalemate that as a result make the outcomes less equitable and which are again opposed to diversity and so I have a very mixed view because I do believe in democracy. I then imposed as a question: “So what I’m hearing is you have sort of developed a sort of tempered cynicism?” Yes, I guess you’d call it that. In general, I guess I’ve become more cynical both of people who are well, I hate to go along with the people who so opposed the 1960s lefties but I’m a little cynical about the so-called woke left. I see myself still as being on the left, as being committed to

18  How Being a Researcher Affects You Personally

redistribution, as being committed to racial diversity but at the same time I am cynical about many of the people, I guess I would say or judgmental about many of the people who avow these causes. Research can modify extreme positions, filling space between black and white with gray as an interviewee, a criminologist initially a participant in the leftwing activities of the sixties, opined: I don’t see things as black or white nearly as much as I used to. … seeing the world with more nuance, seeing alternate possibilities, what can go wrong (shouts Smash the State, what could go wrong with that?) and that you see things as a mosaic rather than like a split figure … The notion of a fluid multiple universe is one of the things I’ve picked up in research. Seeing things with more nuance was also a result of my own research. Here is how my perspective changed as a result of a research project of several organizations pushing to expand the amount of affordable housing for low-income individuals. Some of those advocates pushed for home ownership for low-income individuals, others supported subsidized cooperatives, while still others supported rental apartments for extremely low-income people. Though all the organizations I studied wanted more federal aid for housing, these organizations were often at loggerheads because they differed on the target groups they wanted to help and were competing for a limited pot of federal largess. This study made me aware of the intermittent and tenuous relationships that can occur within alliances of those who, at one level, support the same cause but, on another level, can seriously differ. As a result, when I’m now wearing my hat as a local political activist, I am sensitive when trying to build coalitions of organizations that seem to be on the same side of an issue to carefully examine which differences need to be bridged to enable them to work together. Another affordable housing advocate now working as an academic reflected on how his research affected his prior political views on what housing programs most benefit lower-income minority group individuals: I think [my research] also made me open minded enough to recognize that things change and maybe I need to rethink some things. … I did a ton of research on the subprime era and the damage of foreclosures and a lot of people came out of that era thinking … particularly home ownership for Blacks was a bad thing. … I did all this research showing how home ownership can be a bad thing, but I also know enough from being on the ground and being around in the nineties when kind of more responsible CRA was going on that it had been the right way to give people an option. His research on housing policies tempered his prior beliefs. Policies that he too had initially criticized after doing more research, he now felt were not all bad.

How Being a Researcher Affects You Personally 19

Under the right circumstances, they worked and, as such, he modified what he in his activist mode now supported. Another urban scholar agreed that his research led him to less extreme, less idealized views. I think research has helped me reflect and be open minded, [I ended up with] a kind of an evidence base for my values and my political positions. … I was very inspired by Michael Harrington (author of the very influential book The Other America) and … European democratic socialism is something that I see as a model but this a kind of fantasy world. [The conclusion that] anything that involves any kind of private market activity is always bad is something that I think my evidence base [from my research] has guarded against. I do think it kind of moderated [my political views] … A political scientist, most of whose work was based on surveys and documents, reflected that “I don’t think [my research] has influenced my core political values or anything like that. (but). the work I’ve done over the years has given me more appreciation of the role of activists, voters, bottom up processes.” In other words, he came to question his previous assumption that voices from below were not important. While I agree that local-level participation is important and can influence outcomes, my research led me to question how often that occurs. While I was studying city issues, I learned that few citizens paid attention to city hall. Those who did pay attention, acted primarily as critics, willing to complain but unwilling to take more positive action. These observations left me somewhat skeptical about the future of democracy at the grassroots level, contradicting my prior believes in the strength of peoples’ power. Other interviewees suggested that their research made them more tolerant, as they explored in depth and better understood the lives of others. An interviewee indicated his research caused him: not to be as judgmental, to be aware of the context that the other person is living in … such as when you are teaching someone comes in and says I couldn’t do my paper last night. It may or may not be bullshit, but sensitivity to others’ needs and even if they are bullshitting, assume it is true, because what are the costs to me, what are the costs to them. I think I would try to be more appreciative of and aware of others. A sociologist shared how his research experiences encouraged him to more often look at the multiple sides of an issue, to imagine himself in the role of the other, and to separate the real from the ideal. If you think about some examples of the recurring issues of police use of force. It is very quick and easy to say, yes police ought to have some

20  How Being a Researcher Affects You Personally

kind of restraint. … the way things ought to be, what is good and right and just. But as a social scientist, I appreciate the ability to … understand, well, when there is a threat to somebody’s identity in some situation where authority has to be exerted. … Being able to separate your morality, your [sense of ] right and wrong, of what ought to be and what is just from the way that people truly operate. From her study about federal agencies that were to be terminated, a researcher garnered a broader lesson on how to adapt to difficult circumstances that appear throughout one’s life: I was studying agencies that were terminating, were ending, …. And I got to know a number of people in these agencies. … What still sticks in my mind after all of these years is a description of what it was like to be in an agency that was closing down … What they said to me was “we go along together as a team. When one of us falls down, the other one picks the person up. And we continue to do it no matter who falls down. We as a group help each other up.” That stayed with me all these years …. as a personal philosophy. That you may fall down but if you are with a group of people who are like minded they will pick you up and you will pick up the person who falls, and that is your responsibility. I then asked, “Did you ever use that philosophy?” I suppose when you deal with somebody’s death, you try to pick each other up … But it is kind of an obligation to me to do that as much as I can with other people who are struggling. That is part of my personal philosophy, my personal set of obligations. It is part of an understanding of how people are connected to each other, what a family means, what a group of friends means, what a group of colleagues can but doesn’t always mean. Another individual indicated that his academic study of a political campaign was the nudge that made him run for office (and win). On further reflection, he indicated his willingness to run for office was not the most important personal impact from his research. He continued describing that he had been working on a book about a noted folk singer who had committed suicide. I’ve always been sensitive to the needs of my students but for the first time in thirty-three years, I had a student, I was convinced after I talked to her, she was going to hurt herself. What got me to make the call were these people (whom he interviewed for his book project on the folk singer) telling me how bad they still feel that they couldn’t do more or didn’t do anything for (the singer). And so I made the call to the school … and they made the call and the police went and did a wellness visit. … I don’t know

How Being a Researcher Affects You Personally 21

if I would have called to help that student even though I would have cared had I had not been talking to other people because of this research. The anthropologist studying sea nomads, immersed in research about people whose lives and work were radically different from her own, discovered the limits of her own knowledge and learned respect for those whom she was studying. It was a very humbling process to be working with these sea nomads … People often think that they are backward or primitive … But when I got into the field and situated myself in the Islands … while we take pride with all the advantages we have in technology and all of that it really requires a different set of skills to be able to live out in the islands and the open ocean and so it was really amazing to see how skillful they were and how much they knew and how little I knew. It was an absolutely humbling process for me.

Gaining Skills and Knowledge that Apply Far Beyond the Research Experience Sometimes in doing research, you have to learn a skill that later on proves useful in other parts of your life. Not all such skills port. In Thailand, I learned how to vaccinate a water buffalo (no kidding) and build a water-sealed toilet; I’ve never had an opportunity to apply this knowledge in my Illinois college town. But other skills learned to help in understanding what was being researched have had direct carry over from the research to my personal life. When studying economic development projects and housing built by community development corporations, I had to learn how to read budgets, spread sheets, and a variety of other financial documents in order to ask meaningful questions. That skill enabled me as a local activist to better interpret and react to city budget documents and developers’ proposals. As a part of my project on community development corporations (CDCs), I learned the nuts and bolts of setting up these redevelopment organizations. Years later, when I chaired a city task force set up to develop plans to renew the poorest neighborhood in my town, the knowledge from my CDC studies enabled me to provide information to the group on how to set up a local CDC, which they did. Some research skills port in unexpected ways. A scholar whose research was carried out in southeast Asia ended up working at a senior level at the federal National Security Council (NSC). The interpersonal skills he learned in Indonesia ported over to his role at the NSC, where he worked as the NSC liaison to Japanese ministries: Working at the village level in Indonesia taught me everything I needed to know about Japan, …, knowing how to give and receive deference, how to be courteous, how to make yourself somewhat indispensable.

22  How Being a Researcher Affects You Personally

Probably, more relevant for ordinary life are these research skills described by the Chinese scholar. I would say the [most useful] idea is being able to listen to other people and to ask questions about what they are doing and what they do and how they do it. … Pay attention to the people, I guess would be for lack of a better word, who are more unfavorable in the society. In China … just regularly living there … where everybody is talking to me in a restaurant, talking to waiters and waitresses trying to figure out where they are from, how long they’ve been there, are they migrants, explaining how they learned how to cook, they have these tools. … they light up as they explain it to me. So here, back in the U.S., I know how hard it is to work when you are  doing service work and … people, I noticed, were not very nice to the service providers. … And, back in the U.S. I’ve been doing that, and I’ve noticed that I’ve been more conscious of that, of asking and learning more about what people are doing. People I’ve met working in the food industry or people who are stocking shelves in the supermarket. I am always interested in who they are, where they are from … and I always notice they smile and they kind of like the idea that somebody is asking what they are doing and appreciate it. Social skills learned in doing research carry over to ordinary, daily life.

Research Is Often Fun and Allows for Fun Activities Research ought to be fun and often can provide opportunities to participate in fun activities. While one can have real moments of frustrations in a project, if doing research itself is not fun for you, you should find another occupation or at very least choose another project. While researchers are after the truth or a solution to a discipline or an organizational problem, for many, doing so is akin to solving a puzzle. How can poorly funded nonprofit organizations gain the clout needed to influence the Congress? Why do people who would benefit from the policies of one political party, steadfastly support another party? Why do some cities incur deficits even when they are illegal and how to they get away with it? Solving intriguing questions can be fun, sort of putting together a social or political jigsaw puzzle. Personally, I find traveling to and looking around research sites a source of fun and pleasure. Once there (and I’m always early), I spend the time walking around new places, exploring neighborhoods, and, on a few occasions, taking advantage of cultural amenities that I would never have visited if my research had not required a trip to a place new to me. Research, especially doing participant observation and depth interviews, can be physically fatiguing and mentally exhausting. Time-outs are needed to recharge and to continue with the work. This need to recharge is obvious for

How Being a Researcher Affects You Personally 23

those who do depth interviewing, but certainly occurs for researchers using other data gathering techniques. Fortunately, the normal flow of a depth interviewing project allows for, I would say requires, breaks to clear one’s head and the flow of such interviews allows for this to occur. Those who do depth interviews quickly learn that estimating how long any one interview will last is near impossible. Some are over in a few minutes while others last for hours. Busy people sometimes arrive late for interviews. Accordingly, when I’m interviewing, I always leave a significant gap of time between interviews (it also enables me to calm down from the excitement and mental concentration needed in doing an interview). Such gaps allow for down time and, on occasion, time for informative and fun explorations of new sites. In interviewing in D.C. when I had such gaps, I would visit the Smithsonian museums. Some activities core to the research itself are simply fun. At their annual meetings, the Washington-based advocacy organizations that I studied would arrange times during which members of the organizations descend on the Congress to lobby for their issues. I joined in as part of my research, taking notes; that work being core to the research. In addition, though I found walking around the halls of the Congress and talking with congressional staff, senators, and representatives exciting, informative, and a fun experience, especially when we as a group visited and talked with the then Senator Barack Obama. A planner and political scientist (and her husband in the same field) did much of their research abroad (often in Europe). Their work was successful as demonstrated in the books and articles they published. Still, they made sure while data gathering that they allow time for having fun: We walked the cities, we walked a lot and I looked at neighborhoods that were interesting and we also went to concerts, plays, in London, to the Theater, … once I was by myself, I decided to go the Royal Ballet. Having fun, learning unusual things, and learning about the research matter often blend into one another as the criminologist described: Ethnography is a lot more fun; it is so messy … let me give you one example. I was always curious about how people make alcohol in prison, how to keep it from the guards? And I spoke to a guy who was considered to be a really good alcohol maker … and he said want to come up for a drink, he had about a five one gallon jugs of orange stuff under his bed. How to you keep this from staff? You can smell it when you walk in. … he poured us some and a couple of more guys came in, pretty soon there is about six guys and this about three o’clock, staff are walking by and they look in and they see us, when staff walks by, hide it, they know you got it, you know they know you got it, it is the game. You have to show you are not flaunting … so we hide it down, staff would wave and move on. And it is

24  How Being a Researcher Affects You Personally

getting to be about now about six or seven and we had one guy passed out and about eleven o’clock a staff guy finally came up, we are all drinking, and said, party is over. Such observations were relevant for understanding prison life; but for the researcher, they were also fun, getting drunk with a bunch of convivial guys.

Conclusion and Lessons Learned Beyond gathering data to answer research questions, researchers have experiences that shape their understanding of the broader world and of themselves. Political and social views evolve, at times moderating preexisting perspectives. Skills are developed that port over to one’s ordinary daily life. Research efforts help researchers mature, gain confidence in themselves. In addition, research encourages introspection about who one is and how one can and should relate to others. Research can become core to your life, far beyond being an intellectual pursuit.

3 THE SOURCES FOR RESEARCH IDEAS

Research begins with a question in mind, a research topic, a puzzle you want to solve. Where do these questions, topics, and puzzles come from? As texts point out, research ideas are inspired by prior academic studies, filling intellectual gaps and testing theories. That certainly is an important source of research ideas and topics. In addition, there are many other sources of research ideas that receive far less attention but in practice are more often the source of research ideas. Before choosing a research idea, certain principles need to be considered. A research topic must engage the person doing the work; a research project that does not grip you can be a waste of time. Early in my career while recovering from a medical misadventure, I decided to investigate how rumors circulate by building a simulation and mathematical model; a topic that I could undertake without travel that for a while I physically could not do. I spent an inordinate number of hours working out a computer simulation to examine the research question of what causes a rumor to spread. By the time I was finished, I really didn’t give a hoot about the research question and dropped the project cold. In retrospect, I should have never started and quickly found a more engaging topic. Most students and professors have considerable freedom in what topics they choose to investigate. In contrast, those who do contract research or are working as a researcher in a business or governmental agency must follow the boss’s orders as topics are assigned or work on a project specified in a paid contract. I often worry if researchers are not interested in the topic they are assigned, they may not work as hard or as carefully as if they were fully motivated and intellectually engaged. When I was working in a research shop in the Department of Housing and Urban Development, I often heard the mantra “good enough for government work” when staff had been assigned a research question that did not interest them, perhaps an excuse for doing less thorough work. DOI: 10.4324/9781003345909-3

26  The Sources for Research Ideas

Even if a research question seems intriguing, two more questions should be answered. Would the findings be of interest to others, those in your field, journal editors, or policy makers? And, would it be possible to do rigorous research with the time and resources available? Some topics are of immense interest and importance but are so difficult to study and require so many resources that they are not practical. However, in such cases, it might be possible to break off a more manageable piece of the larger topic that is feasible to undertake. An interviewee offered a counter point on deciding what is worth studying. He wondered if some questions were of such importance that research on these matters is worthwhile even if there is not sufficient time or resources to do the project really well. He remembered …. a comment my graduate advisor made to me … which was something along the lines of ‘any research worth doing is worth doing poorly’ that is to say if it is useful and important research does it make a contribution to knowledge or policy by doing research even if you don’t have the best research design to do it. Are we better off for having done the research as well as one can do it even if it is not the best or up to snuff than simply letting the question lie? In my opinion, there is a third approach. It is better to break off a smaller piece and do it right than try the bigger project and do it poorly, as a project seen as done poorly ends up not being credible. Another respondent took a different approach by worrying whether some topics would be so narrow in scope that the research would have no impact beyond a handful of academics. He was concerned if the research questions he chose to examine really mattered: I sometimes ask if I am doing anybody any good by any of my research. And sometimes I don’t know whether some of the things I come up, with let’s say for example and I did in my study in US cities, I find that cities with inpatient psychiatric treatment capacity have lower rates of homelessness and crime and arrest so the policy implications of that are very straight forward. Do I have the ability to radically restructure our health care system? Definitely not. Can I try to advocate in some way and use my findings to argue for why we need a certain transformation or some policy change? Perhaps, you know. My sense is, this person was worrying about the policy use of his research a bit too much. Especially for an academic, so long as the research question is itself meaningful, you only need to do quality research and then wait to see if the research has policy implications. Researchers concerned about the policy

The Sources for Research Ideas 27

implications can present their findings at conferences that attract both academics and policy makers or prepare synopses of the research and submit them to journals policy makers do read. And those in policy-making positions, or more often their staff, do search for academic literature that can guide their policy work. Fifty years ago, I wrote a monograph summarizing some of the more practical results of my Thai research, publishing it through an academic research center for southeast Asian studies. Turns out that a major purchaser of the monograph was a foreign aid agency; inadvertently I had reached those who could act on my finding; they had been paying attention to academic publications. Still, matching the scope of the project to time and resources available takes some forethought. Throughout my career, I had a bad habit of picking topics well beyond the time and resources I had so midstream ended up reducing the scope of the study to a portion I could accomplish. An applied urbanist added another cautionary note on choosing a research question, worrying about the competition that does exist in the academic world: So (how) I do I come up with research Ideas? A lot of it happens such when I give myself space and time to think. Like when I am walking somewhere. I’ll say to myself you know that’s interesting; I wonder if that is a researchable question and whether anyone else has done research on it … and then the critical question. Oh gawd, has anybody already done that? And how do I find out whether this is virgin territory or whether I’m just reinventing the wheel and reinventing it in a poorer way than it has already been invented? I felt that concern was over blown. While simply duplicating what others have done might not be productive, building on existing research, exploring nuances that were missed, is a perfectly fine approach in choosing a research topic. Enough of a prelude. Let’s explore five ways in which research questions emerge, starting with those based on academic theories and then examining other ways that have received far less coverage in most research methods courses:

SOURCES FOR RESEARCH IDEAS Academic Theories Suggest Research Topics Spill Over from Previous Projects Policy, Program, or Organizational Problems Responding to Opportunities Personal Background or Experiences

28  The Sources for Research Ideas

How Academic Theories Suggest Research Topics If you are doing a senior paper, master’s thesis, or doctoral dissertation, you will be strongly advised to look through published empirical studies or academic theories on your topic that suggest unanswered research questions. This approach to coming up with a research topic is covered in methods courses, but in ways that create the impression that academic theories are carved on stone tablets rather than being ever changing. It is the very process of the evolution of a theory that can suggest further research questions. In his interview, an urbanist, described how emerging and broader theories on cities refocused the research questions on neighborhood change. Older urban theories treated neighborhoods as distinct and independent units, focusing research questions primarily on factors internal to a neighborhood and in so doing neglecting the impact on a particular neighborhood of characteristics of the broader metropolitan environment. Newer theories indicate that neighborhoods need to be understood within the broader urban context. This interviewee then described how this evolution in the classical urban theories caused a reformulation of his research questions on the causes and consequences of gentrification. He explained “that neighborhood stuff and even household level stuff (now is seen) within a context of neighborhoods but neighborhoods happen within a context of metro and city.” His research question was focused on which neighborhoods gentrify, which do not and why. Under older theories, he would have only relied on data descriptive of the individual neighborhood. The approach, stimulated by the evolving urban theories, suggested that the research question on how and which neighborhoods gentrified depended both on factors internal to the neighborhoods themselves as well as the changes in the overall metropolitan area. A research question on gentrification, a topic of interest to the interviewee, was modified as a result of the continued evolution of broader theories about urban change. In many ways, political ideologies constitute a type of theory, though ideologies often reflect a strong slant in one direction or another and too often contain within them predetermined answers. Still, political ideologies can suggest concerns that can then be investigated without presupposing the answers. An interviewee described how he formed a series of research questions based on Marxist theory without presupposing the answers but rather allowing the political ideology to suggest what questions should be pursued. As he described: I was a Marxist … and I felt it was important for a Marxist to understand what is going on in a shop floor … That means working with workers … and, so I wanted to study a non-capitalist workplace and so where was there to go? Eastern Europe was opening up. The Marxist ideology suggested the broader research questions on what workers’ lives were like on the shop floor. The researcher than focused that question

The Sources for Research Ideas 29

to compare worker behaviors between capitalist and socialist countries without allowing the ideology to presuppose an answer on which might be better. Treating political ideologies as suggestive of topics to examine and then working out specific research questions is a viable path to take.

Spill Over from Prior Projects A chain of research questions emerges when the completion of one research project suggests another set of research questions. That’s been the path my own work has taken me along. I had spent several years studying community development corporations (CDCs), small organizations that build homes in lower-income communities. My initial research question was what makes the CDCs effective or ineffective, a question that had been suggested to me from the academic literature on community decline and renewal. I finished that research impressed with what these small organizations could accomplish. Still the completed the research left me with numerous unanswered questions with one being how did these small community-based, underfunded CDCs obtain the knowledge and resources to accomplish their missions. That became my next research question. In re-examining my conversations with the heads of the CDCs, I learned that they were helped by larger national organizations, suggesting that my next research question would be to ask how the larger national organizations aided the CDCs. I started to work on this question and found it too broad for a single person to adequately examine but, among many other matters, learned that one way these organizations helped the CDCs was by advocating with congress and federal agencies for resources for CDCs. That in turn lead to a more manageable research question about the advocacy tactics these organizations worked out. An initial research question on how CDCs worked evolved step by step, finally leading to research questions on what advocacy tactics of progressive organization worked (and which failed) at the federal level. Sometimes, just keeping your eyes open while doing one project raises subsequent research question having little, perhaps nothing, to do with what the initial study. Such questions might emerge from events observed in the place where the initial study was being done, or even from incidental conversations. The experiences of an anthropologist illustrate this serendipitous pattern of coming up with the next research question. She “was going to Nicaragua because “(she) was interested in that process of sharp social/political change,” i.e., her research question (and project) dealt with political matters. However, while making repeated trips to Nicaragua to study the political question, she went into neighborhoods to talk about political concerns; but in doing so, ended up talking with people who were running small scale enterprises. I was interested particularly in women working in areas like women welders … and things like that small scale origins. Another group was seamstresses and then bakers.

30  The Sources for Research Ideas

That led to her to her next research question: figuring out how these micro enterprises mostly run by women worked. This same pattern of miscellaneous observations or conversations during one project leading to another project occurred later on for the same researcher, this time stimulated by casual conversations. As she remembered: I can say sometimes the kernel of one idea develops in the course of one project and then something totally new emerges from it. Right now, I’m fascinated … around the idea around of verticality in the Andes …. I am very interested in how altitude figures in the thinking of people about social identity and place. … this is an idea that kind of emerged over time after hearing people [when doing a previous project] tell me repeatedly things like if you want to find the real indigenous people you have to go higher up. The incidental conversations she had with those she was studying on one project suggested her next set of research questions on how the different heights in the Andes impacted the inhabitants.

Policy, Program, or Organizational Questions In working out research questions based on academic theories, researchers have near free rein on determining research questions, that is, being able to follow their academic nose wherever it might lead. For reasons, I have long since forgotten I was reading social psychology theories on what would make people cooperate or fail to do so. I mentioned this in class, and an older student indicated that she lived in a housing cooperative that would be a good setting to test out these theories on cooperation that I had been rambling about. That led to a project with the research question asking how those in housing cooperates did (or unfortunately did not) work together. (The study showed they failed to cooperate with one another.) As a tenured academic, I could pick and choose research questions that intrigued me at that moment. In sharp contrast in the business, governmental, organizational, or political worlds, the path toward a research question is far more curtailed, with the researcher often having little if any choice on what to examine; your boss tells you what to do or you’ve signed a contract with someone who is paying you to explore the question he or she has provided. Rather than having free rein, in these cases, the in-house researcher or gun for hire researcher is assigned a research question. The researcher has little discretion on the matter as the topic and research question are imposed by immediate needs of the organization. Such questions can be quite important, why people do or do not get vaccinated against a spreading disease is no trivial matter, but the researchers are assigned the topic rather having a choice on what to examine. Agency for International Development (AID), the federal agency for international development, was wondering why a program it funded in one of the

The Sources for Research Ideas 31

poorest regions in the Philippines was accomplishing very little; finding out why was the research question given to a team of consultants of which I was the junior member. We had no say in formulating the initial question, nor an ability to modify it. The agency assumed there were problems in implementing the development projects they funded and that is what we contractually were obligated to examine. I was frustrated because private conversations with local Filipino officials suggested that the problem was in communications between the capital and the rural areas, rather than on the projects themselves, suggesting to me an entirely different research question on bureaucratic behavior but one because of the contractual obligations could not be explored. Back in the states, another of the interviewees, while on loan from his university, was working for the Department of Housing and Urban Development (at a totally different time than I was there.) The research question he was assigned was to figure out the extent to which insurance companies discriminated against particular groups or neighborhoods and then use the findings to recommend changes to the Fair Housing Act, a governmental regulation setup to reduce discrimination. He was certainly interested in that area, but the framing of the question came from up above. But doing research for Uncle Sam, he learned that the research often had a political slant. When the research began, the Democrats who were in power and were strongly supportive of having the Fair Housing Act cover insurance companies; the Republicans opposed. The researcher completed the project but to his frustrations by then the “the Republicans took control of the House … that whole enterprise came to an end.” The findings of the research didn’t result in policy changes, but the experience made it clear that in a partisan world, whether or not particular research questions are posed can take on a political slant. His research didn’t end up impacting policy because of the political realities. However, the story ended well, at least for the person I interviewed. He became aware of the problem of discrimination by insurance companies and so, once he was back in his university, he continued to research the matter leading a series of books on that topic. In this case, the research question that fascinated the interviewee resulted from a question initially posed by a government agency that when political control changed forgot about the matter. The origin of research questions can follow improbable paths. This pattern, of research questions initially being stimulated by policy matters that then suggest questions that interest academics, is not is not uncommon. Another interviewee had learned that public health agencies were concerned when low-weight babies are born and they asked her, a sociologist/social worker, to explore the topic. In working on this applied research question, she then learned that lower numbers of babies in minority immigrant communities had low birth weights than the general population, but this occurred only in the first generation of immigrants. This intriguing finding suggested an academic research question on what it was about the immigration process or perhaps assimilation to a new country that impacted the health, diet, and work conditions of

32  The Sources for Research Ideas

the mother during pregnancy that led to this change in birth rates. A matter of policy interest, this time to a health agency, led to both policy and academic research questions. On the one hand, when working for a business or government agencies, researchers end up having to study what higher ups, or perhaps those footing the bill want done. Still such policy questions can intrigue and open up a whole array of research questions that a student or faculty member might want to explore.

Research Questions Emerge as Opportunities Present Themselves Sometimes, an opportunity presents itself strongly suggesting a research question just asking to be examined. Being there, perhaps having easy access to a research site or perhaps a data set, sets up a research opportunity too good to miss. An interviewee described how early in his career that a set of opportunities, along with his personal experiences, lead to a research question for his dissertation. This person had been a Peace Corps volunteer in Guatemala. While there, he met a woman whom after dating for a while he asked to marry him, and she accepted. He returned to the United States for his doctoral studies, while she continued living and working in Guatemala. Back at his university, he associated with some students from Guatemala who wanted to organize against a mining project in their country. What he learned from the students suggested to him a research question on the impact of the mining industry on poor rural areas. But how to approach the topic and gain access? Turned out his fiancée’s job gave her numerous contacts in the mining industry, contacts that she could share with her now beloved. And, so I’m doing the solidarity work in Madison with these folks who are saying we would like to organize against mining and at the same time I’ve got my fiancée down in Guatemala getting all these jobs writing environmental impact assessments for mining companies … And so all of these things kind of converged and I realized that this is something, I can get (in) the … early part of the curve and this is something that is going to become a bigger issue. His research question on the impact of mining on rural villages capitalized on the ideas he learned at school from the students from Guatemala and the opportunity made available by the networks open to him because of his fiancée. When I was working at the Department of Housing and Urban Development (DHUD), an opportunity suggested a research topic. A colleague and I had been working at a research unit at the DHUD, where our research topics were assigned to us, some quite boring. During walks after work, we shared our concern that important topics for the agency were being overlooked because of bureaucratic rivalries within the DHUD. As temporary employees, we could do

The Sources for Research Ideas 33

nothing about this problem, but the very problem suggested a research question, that is, exploring the ways in which status games and rivalries within an organization could shape research agendas. We were already there, the data presented themselves if not daily on a frequent basis as we witnessed the rivalries between the several research shops. We took advantage of the opportunity documenting examples of decisions made because of the rivalries completed and wrote up the research once back in our home universities. Sometimes, the opportunity as well as the research question is stimulated by little more than a casual conversation, in fact in the example shared by an interviewee through several conversations. This economic development planner recalled: I was talking to an engaged scholar Richard on the phone one day about a Cleveland bank … that was going bankrupt because of the bad investment they made in Colorado oil shale area. And Richard made the comment, “I could just see the money flowing out of Cleveland to take care of problems in Denver” and I said for the fun of it, Richard I can model that. So, I built a spread sheet model of the interstate distribution of wealth triggered by the savings and loan bailout. Pretty cool. That is the research question suggested by the chance conversation was to discover why money flowed between parts of the country, a question explored through an econometric model that the interviewee designed. Subsequently, opportunities to test and refine the model continued to present themselves, again based on chance conversations. The interviewee described his model to his graduate classes. A year or two later, he got a call from a former student now working for a manufacturing association, with the student saying, “I think you are doing stuff that the manufacturing association really has to know about.” This former student asked him to present his model to a board retreat of the manufacturing association. After explaining what he was doing, the planner received a contract to use his model in an applied research project examining the flight of businesses from the local metropolitan area, an expansion of his research question to a larger matter one with practical implications. The academic research question that led to building an econometric model was refined for applied research. A research question emerged from a casual conversation with an activist scholar and then was refined in a research opportunity facilitated by a former student. Others described how research topics, in fact a full career of such topics, emerged from casual conversations. A criminologist described that a student in one of his classes: was an ex prisoner at Stateville (prison) … this guy was brilliant … I said, “where did you learn all this stuff” and he said, “you think I’m a good student. You ought to see the guys in Stateville, they are the Einsteins, I’m the

34  The Sources for Research Ideas

idiot.” He started urging me to go over to the university office in charge of arranging for teaching in a prison. This student is really the one who got me into prison research. Once teaching in prison, the nascent criminologist befriended the prisoners and from chatting with them came up with a series of research questions dealing with informal social controls within a high-security prison. A chance conversation in a class lead to an assignment to teach in a prison that then brought about a series of significant research questions on prison life.. Research topics can also be inspired by newspaper stories as illustrated in the experience of a sociologist. As he tells the story, he was reading the morning paper: mostly for the sports stories and in the front page there was a story about NPA and NTIC (two community organizations) challenging Allstate for their redlining practices and a representative of NTIC referred to the president of Allstate as an ‘asshole’. I said to myself, there is someplace you can actually make a living and call the captains of industry names like that. I need to learn more about this. He took the time to meet the leaders of these organizations and as a result of the conversations he formulated a series of research questions on redlining (systematic discrimination by drawing red lines around particular neighborhoods and not providing insurance within those boundaries) by insurance companies. Other times when academic research questions niggle, data available from social service and governmental agencies provide opportunities for examining these questions. A sociologist explained I’ve also used a lot of survey data that people (in social service agencies) have collected for … an applied purpose that I have seen an opportunity to kind of exploit or take advantage … (the agencies) want to help people, they want to change policies but I’m more interested in more theoretical processes of behavior or community dynamics or what have you. And I take advantage of the data they collected for one purpose … have not fully exploited for … theoretical purposes A political scientist ran for a local public office. The events and opportunities occurring during the campaign suggested a research topic about what happens when academic social scientists run for office. As he described: my run for city council … turned into a research project … basically about the campaigning, and its take was political scientists who actually engaged in real politics. And, I turned … my city council campaign basically into an after the fact participant observer research project.

The Sources for Research Ideas 35

Sometimes the opportunity to investigate a second research topic occurs while doing another project. One researcher was tracking the emerging feminist movement in Nicaragua and while doing so she observed a new movement forming to advocate for the rights of people who were lesbian, gay, bisexual, and trans. The opportunity was there, and after she completed her current project, she moved on to examine the research question about the rise of the LGBT movement in Nicaragua. As she said, I “was sort of following my nose and getting interested in some dimension of activities that I hadn’t anticipated would go that way.” Funding opportunities also suggest research questions. I had asked a former graduate school classmate why he had ended up exploring a series of research questions on urban poverty. He answered explaining how money (in this case big money) moved his university in that direction: … Ford [Foundation] had looked at American universities … And decided … that their expertise was needed to address the emerging problems of urban America, … Ford set out a series of challenge grants …one funded the program at my college. Still funder-inspired research can be problematic. I asked a scholar who had worked in both think tanks and universities about concerns that could arise if funders had strong expectations about what the researcher should find. He responded by describing his own approach to avoid the problem: I developed a standard answer, [when dealing with funders] which is essentially I’m going to do this research and I’m going to find what it finds, I’m not necessarily going to find what you want it to find. If you want to make use of it in any way I don’t care as long as you don’t misrepresent it HJR How did they respond? almost always the response was, oh we understand that … A couple of times they …. didn’t say straight up but I understood what they were saying. If you don’t find what we want you to find, we are going to deep six it or in one case essentially, they are not going to pay. In that case I’m not going to do that.

Stemming from Your Personal Background or Experiences Everyone’s personal life—the relationships formed, adventures experienced, jobs held, and political or social concerns—can be suggestive of research questions. In addition, personal and family matters and at times a search for adventure can impact the choice of a research question and subsequent project. My own early career exemplifies how this might come about. During graduate school in political science, I concentrated on two distinct research areas—one on mathematical social science (I had two math degrees) and the other rural

36  The Sources for Research Ideas

development with an emphasis on southeast Asia. Come dissertation time, I had worked out research questions for each of my two interests—one on how voting rules impacted board decisions, a question in mathematical social science, and the other on rural development in Thailand. Both interested me and I wrestled with the choice of which to pursue a decision that ended up being made for personal reasons rather than academic ones. I was young and newly married. My wife and I had never traveled, and we were living within a few miles of both sets of parents. I had an offer from a well-known, academic, who later won a Nobel prize, to guide my research on mathematical voting models, but I decided to undertake the research on local economic development in Thailand because it got my wife and me away from our parents and allowed us to travel and see some of the world. I had worked out appropriate research questions but admit the choice was based far more on personal matters than the intellectual questions involved. Decades of being an advocate (i.e. a pain in the butt) with local government led me recently to formulate a research project expanding on my personal experiences and ask as a research question which lobbying tactics work best with local government and which ones fail. Though inspired by my experiences as a local advocate, the research question did deal with broader political science issues examining when and what types of citizen participation in local government actually have an impact. The project eventually led to a book on how to advocate with local government based on several dozen interviews with mayors. A political scientist at a smaller university also had a side gig as a radio commentator on local government. Wearing that hat, he was covering the election campaign of the person who became the city’s first African American mayor. His experience in covering the campaign as a part time radio commentator suggested a research question of how a new and first African American mayor would govern, especially on matters dealing with race. As the interviewee described: “after he got elected … I basically asked him if I could start following him around. I thought it would be kind of interesting to see what a new mayor would do.” His part time gig in as a radio commentator suggested both a research question and as he had met the mayor to be from his commentator role the opportunity to study it. A personal search for excitement can also lead to research topics, perhaps without the foreknowledge of the potential difficulties involved. I wondered why as a young woman, then an anthropologist doctoral candidate, took on the difficult task of researching sea nomads doing so alone living in remote areas that few had visited. I asked her and she replied: … Having grown up in Singapore. … having grown up in a highly organized city, what really attracted me was to look at rural communities … the other thing was that Singapore is an Island so I grew up basically surrounded by the sea, so I always had a fascination with the ocean. I really enjoyed being out on the waters. … So being young and looking for adventure it all sounded very exciting. And, I think maybe it was good that I was naive

The Sources for Research Ideas 37

because (laughter) I think if you think too hard and start to consider all the challenges you might actually not have the courage to go into the field. … It is just wanting to know something new, to break new ground and do something different and you realize that there is so much out in the world that you don’t know. I think that the spirit of adventure, the spirit of curiosity was really important. Favelas are shanty towns in Rio de Janeiro occupied by migrants from rural communities. Since my graduate school a very long time ago, I wondered why a classmate had decided to do her research in these reputedly dangerous communities. So, for this book, at long last I asked her. Her answer added insight to how early personal experiences might shape a research question. She described that when she was an undergraduate student … There was a … summer field study program in anthropology, and they picked six undergraduates … to go and spend the summer doing field work … I spent a summer in a village studying child rearing and how the young people developed their aspirations for life and their world view. … I started seeing how very isolated the people were and then in the middle of the summer there was a … cart that was pulled by a donkey that they sell things out of this cart that went around the countryside. And they were selling a transistor radio. And the people in the village combined their money to buy a transistor radio. … so before, all the young people wanted to do what their parents did. Some wanted to fish, children of the agricultural workers wanted to work with hoes, and everyone wanted to be whatever their parents and grandparents and great grandparents had done. Then after that, all the young people wanted to go to the big city, they wanted to go where the action was. Because they heard all these like advertisements on the radio. … And so, I was around eighteen then and I thought, well, holy moly, my life span on this planet is going to coincide with people all over the world wanting to leave the countryside and go to the big cities for more opportunity and to have an unknown, exciting future. And I had decided right then and there, that would be something I’d like to do and see what happens when the people from these kinds of villages came to the big city. So, years later she ended up with a research question on the impact of ruralto-urban migration, studying in the favelas of Rio de Janeiro where rural migrants now lived.

Conclusion Research topics are often chosen to test theories and expand theories in the social sciences. Graduate students seeking committee approval for their thesis or dissertation proposals are acutely aware of this fact; and among professors,

38  The Sources for Research Ideas

academic careers can rise or fall as their research tests, corrects, or expands existing theories. In addition, research ideas emerge from many other sources. Researchers working for government and social service agencies explore research questions relevant to the agency programs, without regard to academic theories. The availability of funds or chance opportunities might make exploring a particular research topic both a possibility and a priority. Finally, research topics emerge from personal experiences and histories. Such topics can and hopefully will ultimately link up to concerns within a discipline, but it is a personal matter that initially suggests the research. No matter where a research topic comes from, if the topic does not interest you, if it doesn’t intrigue you intellectually and sounds like fun to do, assuming you have a choice, pick something else to study. Original research is challenging, demanding of your time, your creativity, and, at times, your physical strength and adaptability; no matter the origin of the ideas, a personal element should be present in assuring that the topic strongly appeals to you.

4 THE EVOLUTION AND MODIFICATION OF A RESEARCH DESIGN

You have chosen, or perhaps been assigned, a research topic. What’s next? Following the sound advice in research methods textbooks, you prepare a research design. Research designs are meant to guide the entire project. Few students would start data collection unless their designs were approved by their faculty mentor, thesis, or dissertation committees. If funding is sought, granting agencies will insist on seeing a fully fleshed-out design. Funders want to know not only if the project is important, but also whether it is viable, whether the project will produce meaningful findings. Designs should make clear the purpose of the project, the appropriateness of the proposed methodology, as well as the time and monetary costs. To demonstrate viability, proposals need to include the number of researchers and how much time they will commit, how access to data will be achieved, and how any legal or ethical problems will be handled. A research design describes the research problem, why it is important, where you will do the research, what data you will use or what people you will talk to, as well as the data gathering and the analysis techniques you plan to use. Designs for academic research often include hypotheses, that is, formal statements that will be tested with the data obtained. Even if you don’t need approval for a project, it is still a good idea to prepare a design. I follow my own advice. I sketch out a design for a project and then pretend I’m a funder or a dissertation adviser and critique my own design. That conversation with myself forces me to question the logic and feasibility of what I’m planning to do. In the lead paragraph of a formal design for a proposed master’s paper, a student might state: “for my thesis I’m going to examine why college students choose particular majors, using an e-mail survey of students currently at Illinois state DOI: 10.4324/9781003345909-4

40  The Evolution and Modification of a Research Design

universities. Such information will be helpful in allocating university resources.” Similarly, the beginning of the design for an applied project undertaken for a private sector company might begin by stating, “(Company name) requires insights on how to improve its products. I will set up focus groups of customers to get their take on the matter, relying on a list of registered users who now live within twenty miles of our headquarters.” A doctoral student’s or a professor’s project design would show how the proposed research examines questions relevant to the discipline. An academic’s proposal for research on legislative behavior might begin as follows: The state legislature just passed a bill to combat climate change, a bill environmentalists strongly supported and fossil fuel companies opposed. The proposed research will examine how such controversial legislation got passed, what compromises were needed, and describe the role of lobbyists. The research will include depth interviewing of legislators, officials in relevant government agencies, and advocates pro and con. The work will also examine documents detailing the development of the wording of the bill, the amendments proposed, and e-mail received from the public and from interest groups. The project will expand upon existing theories of legislative behavior. In a research design, a researcher for a public health agency might indicate that the purpose of the research is to determine if the agency’s program to encourage individuals to adopt healthier eating practices is succeeding. “To check how well the healthy eating program is working, I will interview program recipients asking about changes in their food choices and combine that information with hard data measuring before and after weight, blood pressure, and cholesterol using medical records that participants have given us permission to use.” A research proposal for a social action agency concerned with lending discrimination might state “By law, banks are not allowed to discriminate in their lending by geographic area or by race. Using, data in bank reports to their regulators, this project will examine how well banks are complying with these federal laws.” In each case, the opening sentence or two describes the topic of research, how data are to be gathered, from whom, and when. These matters are further elaborated later in the proposal. A fully fleshed-out research design also points out the limitations of the research. For example, information on how students choose majors might apply only to current students in Illinois. In the hypothetical focus groups study about a product, reliance solely on those who had purchased the project might limit the how generalizable the results would be, because those who didn’t buy the product or who didn’t register it might be very different from those chosen for the focus groups. Textbook discussions on research designs offer a good starting point, but they often miss important points. Those missing pieces are what are covered in the rest of this chapter.

The Evolution and Modification of a Research Design 41

The most important of these missing elements is making clear that a research design is not cast in concrete as most project designs are modified in some way as the project proceeds. Sometimes, the initial assumptions turn out to be incorrect; other times, promised access does not come through; frequently, the initial research question evolves as more is learned and what is really important to learn about becomes clearer. Designs might have to accommodate to changes in the social, political, or physical environment in which the research is conducted. Political upheavals or, as is now occurring as I write, pandemics can render the best planned designs obsolete. A design must be flexible enough to accommodate to external changes. A second major point often missed is the role of luck, perhaps because it sounds unscientific and uncontrollable. Luck means that you gain access to individuals or a site that had been closed to other researchers; it means that an unanticipated helper has stepped up and pointed out where important events are occurring or providing you with insights that better enable you to understand the data collected. I’ll discuss these often-missed matters about designs roughly grouping the  material into two categories—those factors that influence working out an initial design and those that later can lead to changes in the design while the study is underway.

FURTHER INSIGHTS INTO RESEARCH DESIGN Influences on the original design: accounting in the design for matters that make the researcher comfortable recognizing how you are seen by those being studied, impacts a design sources of funding can shape a design Influences on design changes over the course of a study: adapting to changing conditions taking advantage of unexpected opportunities responding to what one learns in the earlier stages of data gathering

Influences on the Original Design Three elements that should shape a design are frequently overlooked. The first is the need to match the design to researcher’s specific skills, personality, and interests; an approach that would work for you might not work for a classmate. The second is the need to anticipate, in the design, how you are perceived by those you are studying. And the third is the need to get funding and negotiate with the funders on the overall design.

42  The Evolution and Modification of a Research Design

Design Needs to Mesh with Who You Are, What Makes You Comfortable Successful designs need to consider the individual background and personal characteristics of the researchers. The researcher’s age, health, and energy level can impact what is studied and how. You need to think about your family obligations, how far from home you can travel, and for what periods of time. If you have children, how will you take care of them if your project takes you to a remote place? These personal elements must be compatible with the proposed work. Several of the women professionals described how their design was impacted because of being mothers. One anthropologist whose research was in South America described that because she had a young son, I made short trips, I made multiple short trips and I took him with me sometimes in the summer and a couple of my most frightening experiences were with him where he had, he was in some danger. Another woman, a social worker doing research abroad on one occasion had to place her children in a day-care facility, necessitating taking time out from data gathering to bring them there and later fetch them. As she said, “that made life pretty complicated.” On another occasion, she left her children back home in the states with her husband. She told me, “that was tough on all of us. I would get on the phone with the kids, there was no zoom or anything, and the two-year-old would just cry, like I hate this.” She had to mesh her data collecting with her parental obligations and it was not easy emotionally. What about your attitudes and beliefs? Are you comfortable doing studies of gays, transgendered, or nonbinary individuals? What about criminals, those who have stolen from the rich, or the poor, or the elderly? What about those who have been stealing other people’s identities? How about people who have raped women or boys, or beaten their spouses or children? If such folks disturb you, will you be able to do the research without biasing the results? Such questions need to be considered in designing a project on sensitive topics. Your social and political attitudes can affect your ability to carry out the work. Can you successfully do research in a group whose values you despise? Or, on the other hand, will you only see the good side of those with whom you agree? I worried about this when I decided to study organizations whose missions I supported. Those in these groups seemed to bend over backward to describe the nitty-gritty of how they worked, flaws, and all, perhaps as a way of unburdening themselves to an outsider, who was on their side. However, even though I did ask tough questions, I still worry that what I heard was too roseate or that I may have presented them too positively in my reports as I didn’t want to negatively impact their organizations with what I wrote. Your skill set as well as your comfort with various methodological tools can impact a design. If a project design requires a survey, I’m perfectly capable of

The Evolution and Modification of a Research Design 43

working out the questions, but I lack the ability, or perhaps the will, to interview from a fixed-answer survey form because I so desperately want to have the person elaborate on the matters. If surveys are required, if I can’t hire someone more skilled at asking survey questions than I am, I simply can’t do such a project. On the other hand, if you can’t deal well with uncertainty or layers of meaning that must be unraveled, or if you get angry if someone lies to you or keeps information back, you probably won’t be good at using in-depth interviewing as your data-collection technique.

How You Are Viewed by Those Who You Want to Study Matters in Research Design Sometimes, how you are seen by those you want to study not only will influence the design but, in some circumstance, might make it impossible for you to do certain projects. When I was interviewing African Americans involved in community redevelopment work and asked about obstacles they faced, initially few would bring up their concern with racism to this white guy. I had to introduce this sensitive topic myself. However, once I did, the interviewees elaborated on the matter in detail and in the second interviews had no hesitation in introducing the matters. Still, I wonder how they would have responded to an African American interviewer. Gender identification can matter. Although still relevant, a generation or two ago, gender was a particularly important element in designing and conducting research. A female researcher was worried about working as a solitary woman, so she and her husband designed projects they could do together: When my husband and I got married … I was afraid if we did things separately, that he would succeed, and I would be a nobody. And so, we agreed to write everything together. (after doing the research as a team) Gender still influences design as it is difficult for females or males to get access to some locations, such as single-sex dining clubs or locker rooms of male or female sports teams. Getting around after dark in some cities is probably still a problem for female researchers, an issue that needs to be accommodated in the design, especially if going to nighttime meetings or interviewing in the evening is required. Are you a member of the group or organization that you want to research? And how might that matter? You might know more about the group than an outsider, yet what you know or think you know might impact what you are told or slant how you understand what is said or observed. If those in the group or organization are divided, will you be seen as supporting one faction and will that impact what you can learn?

44  The Evolution and Modification of a Research Design

At times, researchers outside of the group are rejected simply because they are outsiders; on the other hand, as one informant opined “sometimes you can get stuff as an outsider you won’t get as an insider because you are safe.” He elaborated: you can say the things that nobody else dares say and can raise the topics that people may be afraid to really address whether they want to or not. My value has been to be trusted enough so that I could say what I thought needed to be said and ask what about this or why aren’t we talking about this? On the other hand, those being studied might want to hide matters from outsiders. In the area of Thailand in which I did my research, there were armed battles between the government and tribal groups. One night when I heard conversations about the fights from the Thai officials I was studying and then asked about the shootings, the response was an immediate denial that anything had occurred. Had I wanted to study the conflict, my perceived status as an outsider would have prevented me from doing so. It made me wonder what other matters were being hidden from me. Perceived social status can matter. In Thailand in the late sixties, American academics enjoyed considerable respect. This relatively high social status gave me quick access to high-level officials such as provincial governors, for instance. In the states, getting on the calendars of state governors would take forever to arrange if it were even possible. A senior political scientist, a Nigerian by birth, now a long-time American citizen, shared an opposite experience when he got caught up in status games doing research in Nigeria. … because you are coming from the United States there is a need to put you in your place, right? Because they want to show you that even though you are coming from the United States Mr. Big Shot. No, we too are important. … So, the guy is constantly trying to show you, if not your equal, maybe we are your superior. And that seems to affect the quality of the interaction that you are having with that person. When interviewing, he had to allow more time as inevitably he would be made to wait as part of the status games. In designing a project, remember that those whom you are studying are often also studying you—who is this person who is prying into my life? You need to build into your design a way of allowing those whom you are studying to recognize that you are a legitimate researcher. If you are doing a telephone survey, will the person answering the phone, see you as a neutral academic, an advocate for a political party or issue, or one using the ruse of a survey to make a sales pitch or even worse? People want to know, and your design should include a way of letting them find out.

The Evolution and Modification of a Research Design 45

The anthropologist studying the sea nomads told me that early in her research, “the people whom I was going to study were also studying me.” That certainly was my experience in Thailand, especially in the most remote village. People came into our house, walked into our kitchen, and looked into our rice pot. Food was linked to regional identity and social status. People from the central region, associated with higher social status and authority, ate steamed white rice, while locals from the north ate sticky rice. It was important for us to eat like those in the area we were studying. My research design necessitated choosing a life style (and type of rice) that would mesh with those being studied, something that I quickly added to the design. The local bureaucracy that I was studying had a status pecking order. Where would I fit in the hierarchy? I decided (rightly so I think) to peg myself as more or less equal to a young Thai college graduate just starting on a bureaucratic career. Such practical interpersonal concerns need to be considered in the design in terms of how you introduce yourself and the way you interact with those you study. Ethnographers can easily be mistaken as reporters, as both interview people take notes and observe events. People in the political realm are often leery of reporters and less frank in what they say. One researcher indicated it was important to say right off: I’m not a journalist, I’m not a reporter and I’m not looking for dirt, don’t tell me. I’m not interested in it. And that would relieve them and start the interview off on a better foot. In my project on zoning matters, I attended the same meetings as did reporters, but did not want to be mistaken for one. Fortunately, my design involved leaving time to attend numerous meetings over several months early in the study so that those whom I was studying would see that what they said and what I saw did not appear in the newspaper. I had evidence that the tactic worked. One day, I was in a small conference room observing a meeting between a zoning official and several politicians serving on the zoning committee. The zoning official (a core person to my study) intended to share with the elected officials disparaging comments about the petitioner who was appearing that morning. I was conspicuously sitting in the corner of the room visibly taking notes, when the zoning official indicated to the elected officials that he was glad that there were no reporters present so he could speak frankly. My being there meeting after meeting to engender trust apparently had worked as I visibly continued to take notes as he spoke. The social worker working in a low-income community in South America described the variety of the ways in which she was seen, perceptions that if not countered would have made research difficult. As she described, “some thought we were a proselytizing Protestant group, another thought we were an

46  The Evolution and Modification of a Research Design

oppositional party and there was a third thing, I forget what the third thing was.” She had to make clear over time who she really was and how valuable her work and her group’s work was to those she was studying. That required building into the design a period for her to credibly introduce herself. Many people have little understanding of what research is about and who a researcher might be. In developing a research design, you need to work out ways of making your role apparent to those whom you are studying. Sometimes, how you are perceived is something that is fixed and does affect the research. In such situations, you need to understand how that might limit what you can ask or learn about. An anthropologist described the difficulty she encountered because of her national origin. I was doing research in Venezuela because I am Venezuelan. I can go to other places in South America and ask people questions and they would not ask me questions on where I stand in politics. But as a Venezuelan I’m hardly afforded that. Are you with Chavez or against Chavez and that might determine whether we can continue the communication. They do not accept so called objectivity of the researcher …. it they think the researcher is part of their community. The status of being a Venezuelan required having a view toward Chavez, a view that could impact how those being studied would respond to questions on totally different matters. This constraint would have to be factored into the research design, perhaps limiting what topics could be studied. A sociologist indicated that in studying of the police, he had to consider the police perception of sociologists. “When I went to study the police officers, you could tell they have a certain skepticism. Here is this sociologist, which of course makes people think automatically it triggers a whole schema of liberals, critical of the police and so forth.” As a result, he moved away from further research questions dealing with the police. It does make me sort of reluctant to … study police again because they are notorious for being somewhat insulated and they don’t like outsiders scrutinizing them and now especially there is a lot of criticism, some of it justified some of it over arching [overdone?] perhaps, but there is a sense of what is this guy going to do to criticize us. Is there any way to approach police that would give them more confidence in who you are and what you think about police? Perhaps, you can circulate some of your prior research to demonstrate your lack of bias, or maybe you need to get other officers you know to vouch for you and introduce you. Or better yet partner with another researcher who had previously been an officer. In any case, you need to anticipate the problem and work out a solution as you design the research.

The Evolution and Modification of a Research Design 47

Being a student doing research for a term paper, thesis, or dissertation generally provides an acceptable entry for a study as the student role is well understood. What being a student appears to do is to change the role of those being studied from subjects of the research to become teachers of the less-experienced student. This situation necessitates that the student assumes a more passive role that of a curious learner rather than as an informed researcher. A professor is likely to be seen quite differently from a student. Those being studied probably understand that a professor will write a book or get brownie points at the university for what he or she learns, and might wonder what’s in it for them. A design of a project done by a professor must include ways of showing the benefit to those being studied, at times a direct benefit, other times publicizing (and perhaps legitimating) the world being studied, or maybe showing how what is learned will be shared with those being studied in ways that help them with their work. Far more awkward is the perception that the researcher/professor is somehow testing those being studied. If that image persists, it can seriously harm, if not destroy, the study. In this case, the research design needs to work out how to balance the advantages of being a professor, that is, someone who is curious and does research so is legitimate researcher, with the negative perception of professors as a person who is evaluating and testing others. When faced with such a situation, my strategy has been to allow enough time to be with those whom I’m studying so they see me as a curious outsider wanting to learn rather than a professor evaluating them through grades. Upfront time is planned in the design for just being there without asking too many questions until those being studied accept my presence. Some perceptions held by those being studied can be threatening to the researcher. Being away from home might make the researcher vulnerable. Female researchers have reported awkward social sexual situations. But, males too may encounter sexual advances. I (a male) once was propositioned, which I rejected. The person who propositioned me was on the board of the organization I was studying and later declined to be interviewed. It is not clear how you design to avoid or navigate such situations, but awareness of them is important so as not to be taken unawares. Ironically, in some research situations, appearing to be a vulnerable outsider and powerless can help. When a female political scientist was young, she was researching in a South American urban shanty town that had a reputation for violence. I asked her about her safety. She explained that as a young American female she was seen as “too exotic” and even though it [the research site] was considered a very, very, very dangerous and violent place, I did not experience it that way … And one of the reasons my work went so well I think it was because I wasn’t any threat to the macho hierarchy or to the alpha males.

48  The Evolution and Modification of a Research Design

Perceptions of the researcher can evolve over the course of a research study, as a female anthropologist studying the sea nomads explained. … initially they saw me and wondered if I was some kind of official so they would kind of suspicious. They were a little worried if I was going in there to convert them to some kind of religion … But when they saw … that I was trying very hard to learn their language and that I was willing to live with them. And I’m very very small I’m very petite. So, in the end …they needed to protect me, they needed to help me, so it was really good to be able to come under their wings and to be protected by them. You cannot make being small a part of your research design, but you can make it clear that you are not a missionary, and that you are not a threat, but in working out your design make sure you understand that such actions might be required. When you are studying groups or organizations that have internal schisms, the concern that you are perceived as linked to one group or another needs to be factored into your design. A sociologist, who in college did anthropological research, reflected: Often you are in a situation where you are caught between two opposed forces. And I always use the example of what to do when you go to an Indian village. Do you live with the untouchables or do live with the Brahmans? Of course, most people live with the Brahmans and so what was produced was a Brahman sociology. You have to live with one or the other. You can’t live with both. That was just impossible the way the villages were organized, and the caste system is organized. He described how he handled a similar situation later when studying factories in which labor and management were often divided. “I work in a team, so there were two of us. One with the managers and I would be working on the shop floor.” Such arrangements need to be anticipated in your research design. Perceptions of the researcher can dramatically impact the quality of the data obtained. Years ago, I had a former professional football player in my interviewing class. He did some fantastic interviews of the varsity football team, in part due to his interviewing skills, but in part because he represented to them their fantasy of playing professional sports. You cannot suddenly become a pro football player to interview football players, but you can pull out and emphasize those elements of your prior experience and identity that are most like those whom you are studying. Planning for how you will describe yourself is part of a research design.

The Impact of the Funders on Design To what extent does funding shape a research design? In some ways it must, you don’t design a four-country study without money to pay for the air fare, hotels, etc. On the other hand, do you want your design to depend on the

The Evolution and Modification of a Research Design 49

whims of a funder? Ideally, you put together a terrific research proposal that you are interested in and some agency or foundation decides to fund it as is, but that rarely is the case. Obtaining funding takes work, checking out directories to see what granting agencies will support, and then presenting a proposal for funding that in effect is equivalent to a research design. Many books have been written on how to obtain grants, but some simple rules help. For example, most funding agents have a list of projects for which they have provided support. It is well worth checking out what has been funded in the past to see if your idea fits into the funder’s current set of objectives. Check out if the funder restricts itself to projects done in a specific geographic area. In one of my learning experiences as a younger researcher, I located a foundation whose preferred projects meshed closely with the research I was planning on neighborhood renewal, missing the fact that the foundation concentrated in the Chicago area and I was planning to cover a larger territory, to include cities with far less supportive infrastructure for renewal programs than Chicago had. Then, there are the nitty-gritty matters. Forms for requesting grants contain questions that may seem irrelevant to what a researcher is designing and so the researcher may feel it is fine to ignore these items. That is a mistake as explained by an interviewee who had worked at a granting agency. One of the things I found and learned, if a research organization tells you it wants a, b, c, and d, you damn well better provide a, b, c and d even if it sounds meaningless…. One of the ways I learned that, was the years I spent on as an academic on loan to government [though the Intergovernmental Personnel Act] at the Department of Health and Human Services… I couldn’t believe how rigid they were at looking at proposals. Care is needed to balance what funders are willing to pay for and the research questions that of are interest to you. Cynicism was expressed about funder-driven research by a person who felt pressured to get a lot of grant money. He complained, “I did raise about a dozen grants over the years, but they were always on stuff that was my least impactful work.” Additional care is needed to assure that the funder does not predetermine the conclusions of your research. One of the interviewees stated that he refused to distort his research design to come up with what funders wanted the results to be. But, he then described that some other researchers: and research firms were willing pretty much to shape their research to what the funder wanted to find. And they were more willing to do that the greater the amount of money available. … there is more of that out there than I thought there was. … So that’s a problem.

50  The Evolution and Modification of a Research Design

His advice on avoiding such problems was to be proactive. Rather than responding to ideas put forth by the funders, he suggested taking the initiative: … you call someone up and say I’ve got an idea what do you think of it? I found myself much better at persuading people who were persuadable at doing what I was interested in than simply going through their standard research protocols. … You need to find out who is the research director … talk to them to try to figure out what are they interested in, what is the best way of proceeding, and so forth. Using such contacts can work, but even when the funder and researcher are in sync, there can be a downside if long-term research is dependent on a single funder. A social worker had been helping a university in Ethiopia get grant money to work with a poor in an innovative ongoing research program. I asked her how she convinced the foundation to support the idea in the first place and she explained, emphasizing the role of personal contacts and of being proactive in putting forth a proposal: one of our students was … the program officer, And, I said here is what we want to do, we got this idea with this poor community. But then selling it to them was I think based on the fact that we educated them about … the principle that the people in the community know more than we from the outside. We can be catalysts, the stimulus, the facilitators … he was our big champion … He was fully in support of this creative idea, … foundations like the kind of better mouse trap. The funding continued for several years but then it suddenly ended. (the) Foundation … got a new president and they went through what the foundation wanted to pursue and that was bigger projects between countries … so with that we couldn’t go to the next phase … The implications for design are that you should not plan long-term multi-phased research that requires continual funding to produce anything of value. Instead in a design, phase the research so that each stage offers some useful information. The design should show how each step presents increasingly valuable information, even if the project funding disappears earlier than preferred.

The Initial Design Is Not Carved in Stone Certain matters are almost always worked out in an initial design: Is it better to more intensely study a narrowly focused question or broadly cast the net? Are data best collected through surveys, available data, depth interviews, perhaps through a focus group? How large a sample is needed and with available time

The Evolution and Modification of a Research Design 51

and funding is that sample size feasible? Are the results meant to be generalizable or limited to a particular case? Where should the research be done? Still you are not trapped by your initial design. A design evolves as research progresses. As an interviewee reflected: I have found in my own work that it doesn’t matter how much time you spend reading and theorizing and coming up with the most defensible hypotheses. When you get into the field, everything is going to change. What you need to be able to do, is to pivot, not be rigid and … hear the data and listen to it and follow it and follow it in the direction that it takes you, regardless of what your proposal says. A political science researcher on China remembered that he had worked out a design for his dissertation but with the realistic understanding that it was flexible. My committee said, do you know what you are doing? and I said yeah, and they said good. They said you should have a research design, everything set up, but I kind of knew once I got there we would have to make some adjustments. That was kind of up to me. For my dissertation proposal, I worked out a conceptual design my committee accepted but once in Thailand a lot of the design changed. My research topic was on how local economic development projects were accomplished. I planned to watch and ask about the physical projects that the Thai officials were building consistent with what I had included in the initial research design. But, after a short time, I refocused the study to examine bureaucrat-villager relations and relations among the officials themselves as it became clear to me that these relationships were the core in determining the success or failure of a development project. I revamped the interview questions for the local officials to mesh with the revised design. The change in focus also made it apparent that I would also have to survey villagers, something that was not in the initial design. In his original design to study the evolution of the mining industry in a Central American country, a political scientist included doing depth interviewing of the mining company officers as well as government officials. Once the project was underway, he discovered that he had missed an important component: the response of the villagers who were impacted by the mining operations. He had to add to the study that had initially only included ended interviewing, a survey that was not something that he had originally intended to do. Throughout my career, most of my projects were substantially redesigned. As an example, after completing a study on community development corporations (CDCs), I designed a project to study how larger national organizations helped these small local CDCs. I chose a number of these national organizations, I think close to a dozen, to study and started interviewing their senior people, following my prepared design, asking about the various forms of help they provided to the smaller CDCs.

52  The Evolution and Modification of a Research Design

From the early interviews I quickly figured out that my initial design of examining all the types of help provided—funding, staff training, technical assistance on projects—was too broad for a single researcher to accomplish, so I had to narrow down the focus. Fortunately, I learned from these initial interviews that many of these larger organizations advocated with Congress and federal agencies for changes in federal policies, laws, and regulations to help the poor communities that the CDCs served. On the fly, I redesigned my project to focus on this single topic of how these organizations advocated. I also figured out that I lacked the time and energy to examine how all of the initial group of national organizations conducted their advocacy efforts. I focused on only two of the national organizations, those that seemed most responsive to the local organizations, and each of which had a clearly defined advocacy goals, and as such so modified the initial design to a more manageable level. In general, designs change as compromises are made between what would be the ideal project, and what time and funding constraints make possible. A sociologist reflected on the accommodations that need to be made: I’ve learned to try to approach any project with flexibility …. I often think of having a gold package … what you really want to do. You want to get this particular sample, maybe it is a larger nationwide type of sample but rarely can I ever get the big dollars necessary to pull that off so I only think about how can the … idea be executed … on the cheap, …you know, what instead of the gold package what’s the silver package or plan B. … you’ve developed a proposal which you go to great lengths to craft … but you can’t exactly execute it in the ideal way … so how can you can still pull it off to make it practical and accomplish at least some of what you set out to do. Redesigns that require a change in focus or in the research question might require the researcher while doing a project learn new and unfamiliar material. When asked about what redesign her work required, a respondent indicated that her recent project required her mid-stream to learn material quite distant from her training in urban studies. … (the change in focus) required me to do a lot of reading of social theory and especially of philosophy … I needed to know much more about formal philosophy than I did in order to talk about theories of justice. I took a course at the New School. During the data collection stage of research, unanticipated events sometimes require a major modification in the design. During my research in Thailand, the ranking local official who was corrupt was transferred out of the district and replaced by another senior person with a reputation for propriety. There was no way that event could have been anticipated in the initial design, or even in the updated design on villager official interactions, but it provided an opportunity to

The Evolution and Modification of a Research Design 53

watch how local officials responded to the change. I quickly modified my design to explore how this transfer in power impacted the local officials in how they related to their boss and, more importantly, to my revamped question of how they interacted with the villagers. Sometimes there is no alternative but to change an initial design when the access required is unexpectedly denied. During one of the many times universities were in financial trouble, a dissertation student designed a study to examine the strategies universities followed when cutting back their budgets. Who lost out, who gained, who made what decisions? Her initial design proposed a comparison between private and public universities, mistakenly assuming that universities were receptive to researchers. Whoops. The private universities refused to let her see their budget documents and would not participate in the study. She had to redesign the research, to focus on public universities, which by law had to provide access to their budget data, and, in doing so, revamp her initial questions accordingly. (She also learned a lesson that in her future projects check out access prior to formalizing a design.). Unfortunately, at times unexpected denial of access is not recoverable. A team studying in China had designed a study on land use, thought they had received permission from the higher government authorities for the project, were already living in the communities being studied and had started the work when the local police started knocking on their doors … and asked (if they were) doing the project on land and they said yes and the police said no. Stop. And it was their way of telling us if you go a step further somebody is going to get arrested. They didn’t want to do that, they just wanted to warn us not to do it. So … we stopped it If access is critical to a study, the researcher should not assume pro forma approval will result in access but realize that each piece of the research may need to be negotiated separately. The key is figuring out what topics might be sensitive and why. A land use study in China would necessarily be sensitive, since land use is an embarrassing source of corruption by local officials. In designing a project that might have such sensitivity, it is well worth taking the time to make informal inquiries before starting the actual work. Research designs, especially those of graduate students, are likely to reflect the substantive concerns and preferred methodologies of their academic departments and, of course, their adviser. But once data gathering has begun, reality can set in, contradicting this initial approach, as a now senior urban researcher remembered from her graduate student days: When I did my dissertation … I was really committed to the idea of community control [then in fashion among the urban faculty of her department]. … I had framed it in … sort of before and after, quasi experimental design, [again a preference among her department’s faculty] so that I would

54  The Evolution and Modification of a Research Design

see whether community control affected the learning of kids and in the community school districts. Well, I started going to meetings … (of a) planning governing board … a citizens’ group … which would make recommendations to the school superintendent. … and I interviewed the members and I realized that my thesis could not be about before and after because it wasn’t going to have that effect, but rather it would be about social movements and the redesigned research … turned into how does a social movement get started and what makes it continue. The major thrust of her initial design—examining community control—was tossed and replaced by a study on the origin of a social movement to reflect what she was seeing. Fortunately, her graduate program was well known for its tolerance of such a change. Sometimes what throws of the original design is a change in the political or social environment, making it difficult or perhaps impossible carry out what initially was intended. In her initial project design, a graduate student was going to study the new capital of Brazil; she had already moved to Brasilia and had gotten a teaching position in that city. …. my original research proposal … it was supposed to be on Brasilia [the newly built capital], about how the idea was implemented, and I got various fellowships to teach in the University of Brasilia … When I got there, it was the middle of the dictatorship in ‘68. For one thing they were trying to close the anthropology department and all the social sciences and for another thing they had a big book burning. Research on her topic was politically no longer viable. She had to change topics. Other times, though, unexpected political changes can offer an opportunity to modify the original design allowing studying the before and after effects of the political change. I was examining efforts by advocacy groups wanting to change housing policy in ways that would better benefit poorer people. During the study, the party in charge of the national administration changed. I extended the length of the study to see how this change in administration affected the strategies and effectiveness of the organizations I was studying. The party change provided an opportunity to add the impact of party turnover on the lobbying tactics. Sudden changes in political or social events can change people’s responses to a survey already underway. Imagine the impact of the 9/11 attacks on a survey begun but not finished before the attack toward expenditures for national security. Or, for example, how a major, well-publicized event—violent police encounters with minority group members—could impact the answers to questions in a survey underway about the need for reforming police. When there are concerns that external events might impact a project, see if that possibility can be anticipated in the initial design. For example, while doing

The Evolution and Modification of a Research Design 55

surveys on current topics make sure that the dates of responses are carefully noted to be able to note if the external event impacted the responses. Sometimes, it is not the external environment that changes, but the researcher’s understanding of the environment. An interviewee illustrated such a situation. The initial purpose of the research was to evaluate the impact of a curriculum change on student learning, requiring data on student performance before and after. However, once the project had started, the researcher noted the broader environment in which the changes were introduced and felt that what occurred might have been influenced by factors separate from the students’ responses, such as how acceptable the curriculum changes were to the teachers, or how much support the changes received from the elected school board or parents. In such a case, the design would have to be modified not only to include changes in student performance, but also to note what actions were taken by other participants in that reform effort. At the extreme, a detailed design sometimes might not be possible prior to the beginning of data collection. The anthropologist who studied sea nomads was unable to work out a detailed design in advance as there was very little information on the nomads available and what was there was dated. She described her early observations that were crucial to incorporate into the emerging design I didn’t know that there would be rival groups of sea nomads (in the same territory) So it could be rival sea nomads in adjacent islands. (the) network of territories and these networks of territories are spread out. So, it doesn’t mean …. that the islands closest to you would be people from your group or your tribe, … your related territory could be really far away … I realized that oh my goodness there are rival groups of sea nomads who don’t get along with each other … but then everybody was trying to pull me to their side because they were afraid the enemy would hurt me … So, in the end my solution was that I would stay in different communities in order to be able to be on good terms with everybody, but that helped a lot to be able to get insights into different perspectives of what was going on. Her initial design was to study the sea nomads assuming they were but one group. She had to modify her design (as well as where she stayed) to accommodate once learning that her initial impression about the social environment was wrong. Because designs should be flexible and adaptable, it is possible to recover from mistakes or improve upon them by doing a series of projects building on one another. Early in my career I was interested in how cities carried out economic development. I began by designing several telephone surveys asking mayors about their goals for their city’s economic development and quickly figured out that the answers were superficial. I concluded that I had surveyed the wrong people. It would have been better to have asked questions to the local officials in charge of economic development rather than the less knowledgeable mayors.

56  The Evolution and Modification of a Research Design

However, I knew little about these individuals and what they did. There was no way I could initially survey them. Instead, I picked a series of cities within driving range of my home (sometimes with an overnight stay), asked who was in charge of economic development and then in face-to-face interviews asked these individuals about their perspectives on economic development, how they did their work, and the obstacles they faced. Doing so, of course, involved a redesign from the initial inadequate mayoral survey. Fortunately, the series of interviews produced interesting results on their perspectives, concerns, and approaches to their work. So far so good. However, I was now interested in how general the perspectives of the dozen or two people whom I interviewed were. I wanted to be able to generalize my finding more broadly. Based on these interviews I prepared a brand new survey instrument, miles ahead, of the one I had used with the mayors. The evolving design, that step by step had eliminated past errors, involved surveying the economic development professionals in a random sample of middle-sized cities, a restriction I added having learned that the process of promoting economic development in the larger metropolises was a different game than in smaller locales. I drew a random national sample of middle-sized cities, made an inquiry through a post card survey of who the economic development person was in each city, and sent out a now far more meaningful survey to these individuals. Over three significant stages, the design improved leading to an increase in quality and detail of the information obtained. The lesson to take out of my experience is unlike Athena who emerged full grown and very wise from the head of Zeus, a fully worked-out design does not emerge magically and without error from the head of a researcher; designs are developed step by step.

Serendipity, Luck, and Unexpected Help In research methods classes students repeatedly hear the mantra “follow the scientific method.” I certainly included that saying in my classes. Yet, science alone does not guarantee good results. In fact, at the opposite extreme, and not covered in methods classes, is the realization that serendipity, luck, and unanticipated help, often from strangers, can be vital for project success. Certainly, you cannot anticipate the role of serendipity, nor include a helpful stranger in the research design. Writing in a design saying this project will work because I’ll be lucky sounds silly but allowing sufficient flexibility in a design to take advantage of good fortune when it occurs is a wise move. For some of the interviewees, their major, continuing research focus emerged from a chance, lucky event. The China scholar had no intention of being such. However, while in college, he needed: to take a foreign language class … Spanish class was full, so I started taking Chinese classes … So I got interested in China … I lived there for two

The Evolution and Modification of a Research Design 57

years, and I made some very good friends who started doing very ad hoc kind of studies, just hanging out in rural life. By chance he fell into what ended up as career long set of research topics. I had a similar experience of falling into a topic by luck. I was designing a broad study comparing zoning boards in a variety of cities and had been doing preliminary work, spending night after in different cities watching zoning committee meetings. During this preliminary stage, I heard about a controversial zoning dispute, one of the most significant in the nation, that involved the town in which I lived and thought it would make an interesting case study. Luck, rather than planning, led me to an important case study in a place in which I already had contact with the officials, unanticipated when I had started examining zoning processes. In my Thai study, I received unanticipated help in gaining access to the perfect comparative site for my research. Following my initial design, the first phase of my research took place in a sedate area that created little concern (and interest to) the central government. For comparison, I was seeking a second site with sufficient turmoil to be of concern to the central government and was having little luck. I made an appointment with a provincial governor, described to him the type of place I was looking for, and without hesitation he said, you should do your work in a district he named. I had already noted the suitability of that place; it was perfect other than it was red marked on a security map, meaning foreigners like me were not allowed. I mentioned that to the governor and he responded, “I’m in charge here, that’s where you’ll go to do your research.” That unanticipated support enabled me to compare two areas that differed in terms of central government interest as had been initially planned in my design. I was just plain lucky. An urban researcher now noted for her comparative urban studies described how simple good luck and chance both enabled her to improve her initial design and at the same time helped her gain access to overseas sites: The project in Amsterdam, this is what you call pure luck. The research question in my mind was why is it that other capitalist cities provided more welfare and more planning than American cities and so I thought I’d go to Stockholm … I mentioned this … in class and this guy said don’t go there, that is just so homogeneous, what you should do is go to Amsterdam which is really diverse and economically very successful. It happened that his family lived in Amsterdam and ran the most important public relations firm in the Netherlands. … So he said, if you come to Amsterdam, we will put you up and my parents will introduce you to everyone who matters, and so I went to Amsterdam, … they gave me introductions to all the people I’d want to interview within the planning department, within community organizations. They knew everybody … So, it was just a piece of luck.

58  The Evolution and Modification of a Research Design

A young professional was working for the Civil Rights Commission. As part of his job, he was meeting with community organizations. That linkage led to the beginning of his long-term research into red lining by insurance companies. The community organizations: had just done a study looking at the refusal of insurance companies to write insurance in Chicago’s working class, northwest side, white neighborhood. And I thought the obvious thing for me, being at the Civil Rights Commission, was to look at the same phenomenon in Chicago’s Black neighborhoods. The local organizations were very interested in collaborating with us. So that is what sort of got me going. The luck of meeting with these activist community groups encouraged him in focusing on a research topic and provided him with the access. In another case, the interest and kindness of strangers as well as bit of luck enabled a young scholar to jump start his career and obtain the data that led to his writing what ended up as well-received book. He had been thinking about doing a study on the nature of professions but had little data and no sense of where to obtain it. By chance, an academic article he had written got the attention of two established scholars who taught in a nearby major university. They ran the seminar on professions and invited me to attend. Week after week, I had these ideas, out on a new profession, a new situation, that kind of stuff. I just was incredibly fortunate. There are a bunch of things that happened that were maybe accidents, that (the two senior scholars) read the article … a lot of things just broke well. That exposure led to a detailed study of the professions and his now classical book on the topic. Of course, luck, chance or unexpected help are not enough. As one interviewee wisely stated, “It is the serendipity of your situation that sends you on a trajectory, but the challenge is to take advantage of the situation you find yourself in, to make something out of it.” Luck sometimes occurs in casual conversations, as once happened between my wife and myself. Both of us were interested in why some cities were more likely to provide subsidies (incentives? bribes?) to businesses to locate or expand in the cities. We had not figured out how to get the data for such a project. At that time, we both were working part time at a university research institute. I had been assigned a project that involved learning what subsidies the cities in our state provided to businesses, doing it through a phone survey. In my work I did not inquire on background characteristics of the cities. My wife was doing a separate survey ascertaining on water rates in cities in the state seemingly unrelated to what I was doing. Her survey though did ascertain background characteristics of the cities, many not available from census data.

The Evolution and Modification of a Research Design 59

We were talking at dinner and wondered if we had an overlapping set of cities, as the background characteristics from her study might possibly explain the types of subsidies described in the study that I had done. We were in luck. Many of the same cities were in the two data sets. We merged the two data sets that by luck enabled us to link decisions city governments made on providing subsidies to the broader background characteristics of the cities. When carrying out a research project, you might encounter someone willing and able to help you out, with advice, with access, or explanations of what you are finding out. Such facilitators vary from colleagues, friends, helpful strangers, and even those whom you are studying who want you to get it right. These helpful strangers might take time to explain a technical matter that you don’t understand relevant to what you are studying. I’m a political sociologist/ urbanist not an accountant so I was befuddled by pro forma and spread sheets describing the finances of the projects build by CDCs. I needed to understand them. Several people whose organizations I was studying took the time to teach me how to read the documents. In doing so, they changed roles from being my interviewees, that is, subjects of the research, to technical experts teaching me on financial matters. In my zoning study, a question arose about the extent to which growth in the city with the zoning dispute was fortuitous or part of a long-range preexisting plan. I had no idea of how to check out whether growth was happenstance or carefully planned well in advance. A helpful city bureaucrat suggested to me that I visit the sanitary district. I had no idea why that would help but it sounded like fun. I did so, mentioned my concern about growth being planned years ago to the engineer in the sanitary district. That engineer explained that if growth were being planned the city would make sure years before that sewer pipes were in place. He showed me how to read a sewer map that clearly showed that the growth had been anticipated years before. A bureaucrat advised me to whom to consult and the person I consulted took the time to explain the reality of planning for growth. Luck along with two helpful people helped me fill in a missing piece in my study. A social worker wanted to study how Mexican villagers in remote villages used rural health clinics but had no way of gaining access or credibility with the villagers. That came about through the good will of strangers, in this case public health employees. People know people … you just can’t go in like a cowboy … you can’t go into a village and start asking questions. … I spent an entire year getting official permission … but it is also a way to get the work done, because once I had that official permission, the local officials … facilitated it. I then went to all these clinics way out somewhere where I would never have found them… I was able to do that because I accompanied the medical supervisors who go out delivering medicines and making sure that everything is working.

60  The Evolution and Modification of a Research Design

A political scientist and folk music buff was working on a biography of a famous folk singer whose career peaked two generations ago. He described how strangers helped him throughout the project along with the good luck he had in finding these individuals. I found one guy from an e-mail address that had to be about ten years old … he had been … a young teenage intern at the old Broadside folk music magazine, where a lot of guys’ music got published … I contacted him and he said, ‘yup I’m the same guy, happy to talk to you.’ And he has read every chapter I’ve written so far and given me feedback. This researcher continued describing how later on help came out of the blue, with a little aid from social media. Social media has … a Facebook page on the singer… I’ve posted something up there and it … this one guy writes me his name is CP and CP had been a left-wing journalist in the seventies. … he said the singer used to stay with me when he visited D.C. and …. CP even sent me an interview he did with the singer that no one else had ever seen … you have full rights to use it, he said… he is starting to go into serious Alzheimer’s and so he left me a voice mail the other day … …I talked with him and he said I want to give you whatever I find in my files on the singer. I want to give you, I want you to take care of the stuff. A senior sociologist reflected on his early career and how it had been helped by a staff member at a mental health hospital where he, as a graduate student, had been observing but had no idea how to design a project. The attendee told him how to fix his study. He was very kind about it and realized that I didn’t know what the hell I was doing, I was lost and I was completely unwilling to admit that. He said, you do a comparative thing with this mental hospital. In so doing, offered a jump start to the design of what eventually became his dissertation. Once a project is underway help might come from insiders who take you under their wings and guide you. A social worker described how important it was to have a cultural guide when doing studies abroad. In doing international work, there is the importance of that cultural guide, that person whom you trust with your life and who also can help interpret like, things you can’t know or see. You learn a lot … but there are still pieces that you will never know and he was the one who … you know worked with the community on ongoing basis and understood

The Evolution and Modification of a Research Design 61

all the concepts. So, he was essential I think in the success of the project and how it actually worked in practice. I don’t think I could do that myself as well. Cultural guides are probably needed by any researcher in entering any new environment to point out proper behaviors that facilitate the work. A criminologist engaged in a multiyear study of prison inmates, early on befriended some of the inmates who in turn became his cultural guides making him aware of the norms of prison life in ways that better enabled him to collect meaningful data. For example, they taught him how prisoners exerted control over guards and how they could get away with making alcohol in their cells. Others made it clear to him what he had to do to be better accepted. You learn socialization in so many goofy ways. I observed about six inmates one time in a cell and they were sitting around laughing. I started chewing gum, when he leaned over and whispered to me, I just want you to know when you do something like that [gum chew] for yourself you have to make sure you have enough to share. And he left. About ten minutes later [he returned]. It was a hot day. With a glass of iced tea. Here brother, you look thirsty, I get it. Lesson learned. It was just the little things like that. Major faux pas, in norm busting, not following or appearing to know the norms, those are the kinds of things. Learning how prisoners are socialized. I’ve had such spontaneous help in my studies in the states. In working out my research design I attended meetings of the national trade associations of the organizations I planned on studying. At those meetings, in casual conversation, people asked me what I was doing; I explained my preliminary ideas as I had not worked out a full design. Inevitably these strangers would point out questions I should ask my interviewees while suggesting which organizations would be well worth studying and which were not worth the time. In addition, they clued me to some of the tensions in the field, namely between African American and Latino organizations helping me to later avoid some major faux pas. Many, including myself, when starting our careers or working in a new research area discovered that established senior researchers would lend a hand if asked to do so. Both formally and informally, they acted as consultants in working out a design or offering ideas on what questions to ask in pursuing a project. The social worker who was examining birth weights among the babies of Mexican immigrants was puzzled why the birth weights dropped among the second-generation mothers and had not a clue from her own research. She reached out to a sociologist, a stranger to her, who was expert in the Mexican migration. From his own research he offered her a potential explanation and by doing so suggested to her some variables she could include in her then in progress research.

62  The Evolution and Modification of a Research Design

Earlier in her career this same person was concerned about the way she was designing a project and wondered if her approach was adequate. She wrote to a senior professor who was at the School of Social Welfare at Berkeley. I wrote to him early on because I was working on theories of human services policy making and he had written a lot on that. I wrote to him … out of the blue, I never met him before … I got a five page letter back, discussing this big issue it was so helpful and encouraging. You’re a young academic and not sure of yourself. It was like okay, maybe I can do this. A political scientist now in his sixties but remembering years ago the support he received from a stranger, then the leader in his field. The initial contact was made by the then young researcher leading to a long-term relation in which the senior person acted as a mentor and provided advice as future projects were designed. When I was doing my very first book … I was struggling whether I wanted to include an appendix on my methods … took the courage to call up [name of senior person] who is probably my god in political science. That  … was one of the best things that ever happened to my career … I called … introduced myself, we got talking. He said … if you want to do it, you should. … and after that he became very supportive of anything that I did … He was always supportive and encouraged me to do it. … he was so gracious with his time that particularly given his stature to be willing to show interest and taking in a brand new political scientist at a small liberal arts school always touched me. I had a different experience where a stranger facilitated my obtaining information I sought but had not be able to obtain. I was wondering how to find out if county bureaucrats were providing behind the scenes help to environmentalists fighting for open space whose efforts were opposed by the elected officials who wanted development on the land and not so incidentally were the bosses of the bureaucrats. I had raised the issue in an interview with a senior official who basically avoided discussing the matter as to answer my question would be to admit to inappropriate behavior on his part. However, immediately after the interview, one of his subordinates walked me to an empty conference room, handed me a thick folder, and said, the director (the person I had interviewed) doesn’t want you to see this; he winked as handed me the material. I’ll be back in an hour or so. The folder contained correspondence showing how the administrators in the forest preserve district of the county behind the scenes and against the preferences of the elected officials were aiding the environmentalists. Later, when I was interviewing the director of the forest preserve district, I asked about whether he was working with the environmentalists. He offered a mild denial. I then summarized the content of a letter that had been in the

The Evolution and Modification of a Research Design 63

folder that the mid-level planner had shared with me. The director first said, how did you get that and then said, the heck with it let me tell you what we are actually doing explain his role in both sub rosa and more open aid of the environmentalists. A bureaucrat whom I never met before supplied me with material that subsequently enabled me to get answers to a question core to my research. Later, as part of a project focusing on questions dealing with effective ways of advocating with mayors and city councils, I was interviewing mayors and asked them for examples of controversial local issues that would illustrate what to do and not to do in making an argument to city officials. One mayor provided me with a fascinating case, but when I looked for videos of the meetings on this matter so as to be able to quote exactly what people had said, it turned out that the issue described had occurred prior to when the city started making videos. I corresponded electronically with the bureaucrats in that city, explaining the nature of my project, describing my design in nontechnical language and pointing out the problem from the lack of the appropriate videos for the case the mayor had mentioned. Once again, a helpful person at city hall assisted going beyond the call of duty. She first queried me to make sure she understood what I needed to answer the questions posed in my design. With this information at hand, she pointed out a recent dispute in which advocates on both sides were quite vocal that demonstrate different tactics of advocacy, the core question of my design; the verbal arguments were fully documents on videos. She sent me links to the videos and, in addition, links to where written commentaries on the dispute had been stored. That dispute turned out to be the most interesting of the case studies perfectly illustrating questions I had posed in my design. This person, a total stranger, took considerable time to help with my research. (I did write a thank you e-mail and did copy it to her boss.)

Conclusion This chapter extends textbook discussions about research design emphasizing that the researcher’s own history, interests, contacts, skills, and personality matter when it comes to designing a project. Further, researchers’ age, sex, social status, and ethnic identity are likely to influence the way in which those being studied respond and as such should be incorporated within the research design. Where or for whom you work can impact a design. What a senior well established, funded, tenured professor at a prestige university can research will differ dramatically from what a novice doctoral student might be able to do. And both will experience fewer constraints in choosing topics than would employees of a governmental agency whose work might be impacted by political pressures. The core lesson of this chapter is that research designs must not be treated as rigid and unchangeable. Designs must accommodate, sometimes with significant modifications, to what is learned as data gathering is underway. If initial ideas

64  The Evolution and Modification of a Research Design

prove problematic, they must be amended; if unexpected obstacles appear, they need to be overcome. Designs need to be flexible enough also be able to accommodate to changes in the social or political environment. Design is about planning but there are some things which you cannot be planned; instead, a researcher can only react, and hopefully take advantage of them when they occur. Chance, luck, and the help of strangers cannot be counted on, but in retrospect can have a major impact on how researchers actually carried out their work. To me at least, while starting out with a design with a carefully worked out design makes sense, at the end, a design as described in research reports seems more like what the researcher actually did rather than what was initially planned.

5 ENHANCING DATA GATHERING METHODS Learning New Techniques and Modifying Standard Approaches

Methods texts and classes describe by the book techniques for data gathering. Yet, in some research situations, standard data gathering techniques simply don’t work. Some methods classes overemphasize one data gathering approach while minimizing or even ignoring other approaches requiring researchers to learn these other approaches on their own. By the book, descriptions of data gathering often presuppose conditions that might not be found in all research settings. In such situations, improvisations, adaptations, and cleverness are required to overcome the obstacles for data gathering. Finally, while mistakes are made in doing research and should be recognized, these mistakes are rectifiable and provide learning opportunities for the researcher. Personal characteristics and the skill set of the researcher impact the choice of preferred data gathering techniques, but caution is advised. No difficulties occur when the preferred technique of the researcher is appropriate for the research question at hand. What must be avoided, however, is when researchers rely on their preferred data gathering technique irrespective of its appropriateness for the research question being examined.

Why Are Some Research Methods De-emphasized? Sometimes, research courses fail to cover important data gathering techniques, or, at the very least, prioritize one technique over another. Why does this occur? There are many reasons, with lack of time being one. In addition, there are factors that to me are less acceptable. An interviewee suggested that the failure to expose students to the full variety of data gathering approaches reflected the schism between departments that emphasize quantitative research—that is, gathering numeric data and using

DOI: 10.4324/9781003345909-5

66  Enhancing Data Gathering Methods

statistical techniques to analyze the data—and those that endorse qualitative research—relying on depth interviewing and participant observation. This unnecessary narrowing in methodological skills leads to several, avoidable, problems. The least serious is that researchers limit themselves to projects for which the data gathering technique they were taught is appropriate. Little harm is done other limiting the topics on which research is done. There is more concerning if only a handful of techniques are mastered handicapping researchers when they find midstream in a project that a different data gathering approach is more appropriate. In the worst case, researchers use the techniques they know even if inappropriate for the project at hand producing data that are irrelevant or, worse yet, erroneous. The best approach is to choose a data gathering technique appropriate for the problem at hand. An alternative for those who strongly prefer one data gathering technique in lieu of another is to choose a research question for which the personally preferred data gathering technique is appropriate. Those most comfortable with survey research might want to work on projects where counts are important, for example, how popular a political candidate is or how much support a particular policy has. On the other hand, a person whose strength lies with depth interviewing might chose a question in which detailed probing is needed, for instance, interviewing former spouses on why their marriages fall apart. Years ago, I witnessed an illustration of a problem caused by being trapped in a methodological silo. A sociologist was enamored with the then fashionable small space data gathering and analysis technique, and had been applying it to a variety of situations. He decided to research the power structure within my hometown using small space analysis and ended up with elaborate (and pretty looking) diagrams, all readily publishable. Leaders in the town who over decades had observed who was important and who wasn’t in local decision-making felt that the study’s conclusions missed the local realities. The data collection and analysis technique didn’t match the research question. But, appropriately matching the data gathering technique with the problem at hand is but a beginning as other factors influence the choice of data gathering tool. For instance, the choice of a data gathering tool can depend upon a self-awareness of one’s own skills as well as where skills are lacking. An urbanist reflected: In terms of methods, I have found that I am very good at research that involves talking to people. I’m good at talking with people. I’m good at asking questions I’m good at follow ups, I’m good at doing it as a form of communication rather than going through a standard research design which bores me silly. And, after a little practice, I found I could do that while taking notes and writing at the same time. So, I can think and write at the same time. That was very useful. Still, he was aware of the limitations of only doing what he was good at. He argued that there was only a certain set of questions that he could pursue by talking to people.

Enhancing Data Gathering Methods 67

Sometimes, though there is no choice of which data gathering tool to use as it is mandated in a contract or in the applied world by the boss’ order, at times frustrating the researcher. For instance, while I was working at my university’s research center, we received a contract to do a public opinion poll to ascertain issues of concern to Illinois citizens. That contract specified that a survey was required. We developed a survey questionnaire asking respondents to rank their concerns, but not asking the why and wherefor. Several of us felt that more meaningful data would be obtained using several focus groups allowing for a far better understanding of the reasons for the ranking, but our hands were tied by the contract. When working at the Department of Housing and Urban Development (DHUD), my boss asked me (that is told me) to write a report on how a federally funded urban renewal program faired in Detroit using the documents available in the office. I complied but was frustrated that I was not allowed to do phone interviews of the city officials who were involved in the projects enabling them to better explain why the choices reflected in the documents came about. The rest of the chapter presents further thoughts of those whom I interviewed on the interplay between research questions, their choice of data gathering techniques, and their personal preferences and experiences with a variety of methodological tools. The chart below summarizes their insights.

INSIGHTS ABOUT DATA GATHERING TECHNIQUES Learning methods (and statistics) is an ongoing process, both supplementing what was not taught in school and accommodating to new challenges. Methods and statistical techniques evolve, though it is important to be concerned about passing fashions. Research is often undertaken in teams requiring accommodation to the separate skills of team members and courses simply don’t discuss team research. When standard textbook methods do not work, improvisation, cleverness, and adaptations are required. Methods can be improved by learning from one’s mistakes

Learning Methods (and Statistics) Is an Ongoing Process No matter how exhaustive methods courses might be, techniques that you might need simply might not have been covered or were de-emphasized. Several of those interviewed indicated that courses they took emphasized quantitative research in lieu of other approaches; so, when their research topic was best examined through qualitative techniques, they had to learn these techniques on their own. To a lesser extent, those whose course work emphasized qualitative techniques sometimes faced a problem best done through a numeric approach and again had to self-instruct.

68  Enhancing Data Gathering Methods

A planner’s reflections were typical: I … didn’t have to take a qualitative research methods course. We had to take quant courses … so I had to learn qualitative stuff on my own in reading books and looking at how other people did work. But I continued to learn quantitative stuff, too … I went through grad school before a lot of stuff that is being used now regularly was even taught. … and of course, there have been lots of new methods. Other interviewee described similar experiences. in the PhD program there wasn’t really a whole lot on how to do field work and the textbooks didn’t have anything about doing it in other countries, let alone non democracies, so I yes I did learn a lot. And it was with my Chinese colleagues that we really picked up how to make it work. Self-learning was also necessary for those who do survey research. In my generation, what was taught about survey research focused on face-to-face, or at times, telephonic interviewing. Those who proposed to do mail surveys had to learn the techniques on their own, while, with the advent of cell phones and the increase in the reluctance of people to respond to phone interviews, whole new procedures had to be mastered. There are a variety of reasons why continual learning is necessary. One reason, suggested at the beginning of the chapter, is that departments tend to lean in the favor of specific methodological approaches and discourage the use of others. Another reason is that data gathering procedures continue to evolve long after you’ve left school. As a sociologist described: I tell our own graduate students that a lot of what you learn in order to become a researcher takes place outside of the classroom. You learn a lot from your fellow students, from your fellow colleagues about the nitty gritty in the process of doing research … things that don’t come up in your classes and in your training and part of that is because everybody’s research is somewhat unique … you can’t possibly train somebody for all those kinds of situations …. And, especially for me, it’s an issue of learning different kinds of statistical techniques because depending on who your mentors were … where you went to school, when you went to school, you are going to get trained in certain kinds of statistical techniques. But those are always evolving … there is whole other kind of slate of popular statistical techniques that I was not trained in … And it takes me personally a while to figure some of this out and then also see how they apply to my particular problem. I asked a former graduate school classmate who developed a national reputation for constructing political polls how he learned to write questions. In graduate

Enhancing Data Gathering Methods 69

school, we both took courses from faculty who at that time were national experts in survey research. Yet, my classmate answered that it was experience, not the classroom instruction that counted: How did I learn to do what I do? Trial and error. One of the things in a (political) campaign, you might get to write a couple of hundred surveys, so you have a lot of opportunities. And then you see when something really strikes home and you get something that you hadn’t understood. His comment reminded me how long it took me to develop skills in open-ended interviewing, something for which nowadays I am considered sort of an expert. I had no training in interviewing in graduate school, and initially thought that depth interviewing was just an extension of a normal conversation. I’m a talkative person, but quickly discovered that being talkative isn’t a good way to conduct an open-ended interview. I had to teach myself to keep my mouth shut and listen, and just encourage my interviewees to keep talking. And, I had to learn how to balance the flow of the conversation between what my conservational partner felt like discussing and the topics I as a researcher needed to be covered— far different from an ordinary conversation with a friend. Other respondents emphasized that method courses routinely failed to describe the social skills that are necessary for doing research projects. Researchers need social skills in order to negotiate cooperation from individuals and organizations doing so in a way that convinces them that being part of a project is worthwhile. Sometimes, researchers have to sweet talk access to archives especially those that are private or that limited hours. Sales skills are needed. Those who do depth interviewing need to convince people to talk to a virtual stranger and then be willing to engage in what can often be a long, probing discussions. Survey researchers do pretests and then have to encourage those answering the rough draft questions to indicate problems they found in the questions.. All these situations require social skills rarely mentioned in methods classes. A respondent who studied insurance companies reinforced the importance of learning social skills, commenting that such matters were missing in his graduate school training: [I needed] to work with people from many different walks of life … I think that is the most important thing I feel I had to develop that we didn’t even come close to touching on in graduate school…. (an ability) to “schmooze” that is handle casual conversations. The political scientist who worked in China commented on how in doing projects: “I learned a lot was what to say and what not to say. So. working with my Chinese colleagues when we were talking with government officials, there was a certain kind of etiquette and protocol that you have to be careful of,” that is, cross-cultural social learning was required.

70  Enhancing Data Gathering Methods

He mentioned, quite seriously, a matter of social skills that was even less likely to be taught in a methods class, except in those classes taught late on Friday afternoon that might adjourn to a bar, he: had to learn about drinking stuff, so some of my best interviews I can’t remember. And learning drinking games, there are things you do with your hands. … my friends knew it and they taught me how to do the drinking game. And that got me incredible access with the village leader, village doctor and just the average farmers. I think none of the top academics could have gotten access without learning some of that colloquial—not just language, but kind of habits and games. In addition, there are project-specific sets of knowledge required for successful data gathering that, because of their near endless variety, could not possibly be covered in social science research methods courses. I already mentioned that in my community development project that I had to learn how to read business plans and spread sheets to be able to ask intelligent questions about community development work. As another example, a public administrator who studied organizations experiencing fiscal stress had to learn how to read the budgets and audit reports of universities, cities, and federal agencies. Doing so was all self-taught. A political scientist trained in quantitative techniques, later in his career, decided to write a biography of a well-known folk singer who had profound psychological problems. He described that to do this project: “I had to train myself in psychology, at least enough to do that project.” Reading pro forma documents, public audits, and psychological profiles all were vital in different projects certainly could not be covered in a standard social science research course.

Methods and Statistical Techniques Evolve; Though Be Concerned about Passing Fashions Part of the reason that researchers have to keep learning methods is that ways of collecting and analyzing data are constantly evolving. For example, during my dissertation research, a frightening to me 55 years ago, I had recorded hundreds of pages of handwritten notes. I needed to sort these notes into the topics to be covered in my thesis. To do so, I had to laboriously make multiple copies of all my notes. I then had to cut up these copies and put paragraphs dealing with the same topic together in one pile, doing so in the living room floor of my motherin-law’s house where we stayed on our return from Thailand. I had to do this over and over, one pile for each theme in my dissertation. I suspect I omitted topics simply out of the tedium of the task. Halfway through my career, personal computers became available, so all my interview transcripts and observation notes were kept in typed computer files. I had to code the material, but in one pass, I could code on a wide variety of

Enhancing Data Gathering Methods 71

topics. Once coded, I could use specialized software to instantly sort the data topic by topic in less time than it has taken me to write this sentence. When I started recording interviews, I had to use a cassette tape recorder. The longest tapes tended to break, so I limited myself to short tapes. That required that during an interview, I had to watch the recorders and change the tape, distracting my attention from the interview. Flash forward to our modern era with large capacity digital recorders that can record all day long on a single internal mini drive allowing an interviewer to ignore the recorder and fully pay attention to the person being interviewed. Over several generations, survey research changed dramatically. The initial major change was from nonscientific polling to true random sample surveys, as a political scientist reflected. Gallup started surveying in the nineteen thirties … the sampling that Gallup used back then was what was called quota sampling which is not up to contemporary, not random sampling and doesn’t approximate it. … it basically allowed interviewers to go around to fill the quotas however they wanted so they would get the most approachable people within some categories that tended to be wildly unrepresentative on things like education. Surveys moved then to random selection of dwelling units for face-to-face interviewing and to random-digit-dialing for the telephone interviewing. These changes in technology have required further modifications to handle the predominance of cell phones and techniques to handle the lower response rates. A number of survey shops now use panels, sometimes carefully selected, sometimes self-selected, who answer different surveys. Using archives for documentary research used to be a challenge, sometimes just in knowing that a particular archive existed. Specialized archives had limited hours. Nowadays, many significant archives are accessible on the web, as explained by a sociologist known for his archival research. It used to be, up until kind of everything went digital which is basically in the mid nineties— but the real change doesn’t come full circle until about 2005—It used to be the basic problem was to find stuff. Finding archives required knowledge of other peoples’ footnotes describing where archives were and how archives have been split up between here and there and the other place. [Imagine] a very small, invisible college where people are talking about stuff, where people pass around, this is where so and so’s paper are. There was a national catalogue of manuscript collections, but it wasn’t very complete and it never had a cumulative edition. It wasn’t until the early eighties that a cumulative name index was created. The point is in those days up until roughly 2000, you had to know esoterica, who the right person to get in touch was. And you needed to be able to butter up archivists to get them to keep the place open late or make a deal with you about access.

72  Enhancing Data Gathering Methods

The ready access to computers along with advanced statistical analysis packages has greatly simplified quantitative analysis, while bringing sophisticated statistics within the easy reach of those not expert in statistics as the math was all done within the packages. As an undergraduate when I did my first study that required calculating a correlation coefficient, I had to write my own correlation program. A social psychologist, a friend, my age, described that when he was in graduate school, students were paid to use mechanical calculating machines to do factor analyses day in, day out. Both our experiences were long before computer packages for quantitative and qualitative analyses were available. With online computing and more powerful personal computers, along with the availability of canned statistical programs, data analyses rapidly moved from examining two-by-two tables, to correlations, to regression analysis, to path analysis, to something called Lisrel, and to whatever has been invented since I last did a numeric study. All this change reflects not only on technology, but also on fads that can distract. One interviewee compared research to fashion. Things become hot and fashionable and they go away and then they come back … some methodologies tap the topic … they become hot for a while. People get all worked up like this is the thing. But eventually it is going to be some other thing. Being concerned about the impact of fashions in data handling, a planner worried that data science was focusing more on the data science and less on more meaningful concepts. This interviewee argued that whiz bang data visualization and machine learning approach were taking over from focusing on fundamental concepts, such as concentrating on what we are measuring or how we are measuring it. It is possible to spend too much time on learning the latest fad in data analysis, and hence you might end up paying too little attention to the meaning of the data. As an example, the planner described some of the current trends as fetishes, such as too much focus on statistical significance, and too little on the size of the effect, that is, whether something important was happening and visible in your data. It’s good that (to) pay more attention to the effect size than to statistical significance…. I can just make a bigger data set to get greater statistical significance if I want, but if the magnitude of the effect is weak [there is nothing there.]. I remember doing an analysis of regulators’ Community Reinvestment Act [CRA] ratings of banks. [These are evaluations of the extent to which banks were investing in poorer communities where they did business]. I had a fairly small data set, it was like a hundred exams, but it showed that Office of the Controller of the Currency [OCC a bank regulator] … was giving easier CRA exam marks than the other regulators. And a colleague asked me, what was the statistical significance on that? And I said, well it is only about .05, because I had a small sample size, but the size of the effect side was huge.

Enhancing Data Gathering Methods 73

The conclusion that a particular regulator was far more lenient toward the banks than were other regulators was an important finding with public policy implications; a conclusion that could have been obscured had the emphasis on statistical significance dominated over substantive findings. A further difficulty appears when the situation in which research is being done makes following the standard approaches problematic. For examples, in doing surveys, random selection of a sample of respondents is important and ways of drawing a sample are assumed to be readily at hand, perhaps from a list or doing randomdigit-dialing. However, in doing some studies obtaining such a list from which to sample can be difficult or impossible while and, in poorer countries, many still lack phones. Improvisation might be required as I shall describe in the next section.

When Standard Textbook Methods Do Not Work, Improvisation, Cleverness, and Adaptations Are Required One problem that occurs in studies done in more remote settings is that the basic conditions for conducting a modern random sample survey don’t exist. People may not have telephones, which makes random-digit-dialing phone interviewing impossible. They often don’t have street addresses to allow randomly drawing houses from which to draw a sample. Alternatives are needed to approximate what would be considered to be a proper random sample. When I was in rural Thailand half a century ago, I needed to draw a random sample of individuals from those in several villages, long before the inhabitants had phones. I discovered that homes in the villages did not have street addresses. Fortunately, I learned that the health officials who sprayed homes to kill malaria-bearing mosquitoes chalked numbers on houses and those numbers became the list from which I drew a sample. But how to do draw a random sample was the next concern. In those days, we used tables of random numbers to assure that the choice was mathematically random. I had not carted statistical tables with me to northern Thailand. I improvised. I cut up paper into equal-sized small pieces and put on each piece a number matching the numbers on houses placed by the health people. Then, I  put the pieces into a wide brim hat. I shook the hat for what seemed like forever, dumped out the numbers, repeated the process until thoroughly bored and all the scraps were thoroughly mixed, and picked the scraps of paper off the floor. The first so many (whatever was required for the sample size) slips designated the homes to choose for interviews. Another classmate was working in the urban slums of Brazil and had to improvise to obtain a sample. She described that: There were no identifying no addresses, no street names. We took these aerial photographs that I got from the military and traced the houses on that and then we numbered those houses, counting from each intersection. Then we had a whole methodology for getting a random sample.

74  Enhancing Data Gathering Methods

A political scientist more recently working in China described: There were no numbers on the villagers’ homes … (however) in China everybody has a household registration card so the village accountant would have a list, literally a paper list of everybody’s copy of their household registration. [To get a random selection], we would roll the dice and whatever the number on dice was, say it was five, we would start in counting five, and then one every ten, and pick one [from the accountant’s registration list]. So, we were able to get random samples, but there was no list to work on. An expert on rural Guatemala described how he garnered a sample among individuals not used to being surveyed, though, in this case, not a random sample. First, he had to get permission from the local authorities, then he had to hire interviewers and train them, and had to come up with a way of encouraging villagers to be interviewed. The logistics of administering the survey in rural parts of the developing world are challenging because you’ve got populations in most cases that don’t have phone numbers or at least not land lines that show up in a directory. They don’t have addresses where you can reliably send them a piece of mail and in most cases, they don’t read or write. And, so what I had to do for the survey was. I would go into these towns, and I would meet with the Mayor and/or members of the town council and …. would say can I work here; can I study this town and some would say yes and some would say no. And those who said yes I would say okay what communities, what rural communities should I look at and they would give me some names and I would go to these places and I would go to the schools because in every one of these rural communities there is … a public, government-run elementary school … the teachers are all high school graduates, and they can all read and write. And so, I would go to these schools and I would meet with the teachers and I would say, I’ve got a little project here. I’d like to pay you a couple of hundred dollars and they’d say, okay. And we would do a day of training that I would lead on how to administer a survey without sort of biasing the results, how to just kind to read the questions … without leading them … And, then we would convene these community assembly meetings and I would get together with some member of the community who cooks, because there is always a big informal economy in these places and there is always somebody who makes street food … So I would find that person and I would get that person to cook up a whole bunch of stuff and we’d convene a community assembly and get the word out. And, we would have anywhere from 20 to 70 people show up. And we break them off into groups and I’d have all the teachers from the community … take a group and we administer the survey one by one to all the people in the group and then we would have food and that

Enhancing Data Gathering Methods 75

would be the end of it. And, I did this in I don’t know maybe 15 different rural communities …. I collected over 600 survey responses with this method. No mail, no telephones. Every single survey was administered face to face, was read and recorded by hand. And, that worked out pretty well. Not quite a textbook approach, and not a sample that could be generalized to the whole country, yet still providing important data on the localities studied. Changes in research techniques over time make it difficult to compare responses, especially when the data were collected using decidedly different methods. In surveys done in different eras, the exact wording of the questions might change, as well as understandings of the same words over time. Surveys may look comparable but not actually be so. Even more complicated are situations in which older survey data collected prior to modern random sampling have to be compared to current random sample surveys. Handling such comparisons requires cleverness, persistence, a willingness to improvise, and lots of patience. An expert on political opinions and political parties was studying perceptions of racial matters over the decades. He described the contortions required to compare survey data collected prior to random sampling with more recent surveys: The older data was in really bad shape, with weird codes and strange formatting of the code books. In addition, the older surveys were done with quota sampling. That meant that there needed to be a certain number of interviews from each predetermined category of respondents and interviewers could fill the quotas any way they wanted, often interviewing those who were most approachable, resulting in a wildly unrepresentative sample, say for education. The people interviewed were just not typical of the group they were targeting. Southern Black people were almost entirely left out. The researchers were stuck with the problem of how to make the results more representative of the population at the time and similar to recent surveys. The usual way of making a survey more representative is to weigh the answers of some more heavily than others to offset under counts and reduce over counts. If there were only a few African Americans in the sample, their answers might count several times, to make up for the under count. Doing so is not a perfect process, but it helps make a sample closer to the proportions in the larger population. The problem was that the research team didn’t know what weights to use in working with the older surveys. The solution was ingenious being based on an examination of information that was NOT used to choose the quota but was present in the original questionnaire. For example, the older Gallup polls did not have a quota for people with telephones or for people who owned cars, but the polls did collect data on whether the interviewees owned phones or cars. Cars and telephones, especially at that time reflected social class and income. If the researchers could figure out how many people at that time had phones and cars, they could compare the numbers of people in the survey who had

76  Enhancing Data Gathering Methods

phones and cars and see how far off the data was by adjusting for phone ownership within the national population. But … …. you would think knowing the percentage of Americans with a phone and the percentage of Americans with a car, it would be trivial, … Turns out, especially in that period, it’s much more complicated and so I spent tons of time going through old census reports, old AT&T reports, calling AT&T’s archives to get documents. Ideally, we wanted phone ownership by state or region. Changes in methodology create problems comparing studies over time, but with cleverness, those problems can be reduced.

Learning to Manage a Team Doing research is (and should be) a humbling experience. You are unlikely to know everything you need to know for any given project, especially if you are just starting out or beginning a project in a new research area. You may need help learning what you don’t know, or you may need to recruit others who have the expertise you lack and work with them on the project. Working with a team has additional advantages. Teams can help extend the research, to add new cases and conditions and permit talking with more people. When doing work abroad or in culturally different communities, having people on the research team who are closely linked to and have the language skills of the communities being studied is extraordinarily useful. Teamwork can help in reducing many problems in data gathering and analysis. Designing, recruiting, and managing teams are important research skills, but ones that are rarely, if ever, taught in methods classes. I asked an anthropologist, who nowadays runs a research center that relied upon team-based research, how she learned to run a team. She responded, “I wished I had training to do that. I think it is something that I think I learned from doing it, making mistakes and from practice.” A person who studied poor communities with high-crime rates required a team for the project and reflected that supervising a team is a “skill you certainly don’t learn in graduate school.” Even more complicated is that leading a team can create awkward situations for the team leader, often unanticipated and certainly not discussed in graduate school. An interviewee who led such a team shared one such incident that required immediate but unanticipated action: There was a woman [on my research team] … she was very good looking and she tended to wear very sexy clothes and she went to the neighborhoods doing things like wearing short shorts, a low-cut blouse and licking a lollipop. Well, it wasn’t long before I started getting these phone calls from people in the neighborhoods, asking who am I sending out there. And so, I was forced to fire her.

Enhancing Data Gathering Methods 77

If she had prior training in running a team, she might have thought of issuing a dress code, and had her team members agree to it before going out into the field. As it was, she learned from experience. As a novice researcher in Thailand, I had to figure out how to hire a team, act as a boss, giving out assignments, making sure every member of the team knew and understood what was expected of them. I needed to be tolerant of their mistakes and explain patiently how they needed to fix them; they had to be tolerant of mine and learn how to point them out to me. I had to handle problems that occurred, whether in their personal lives or in the research. I was responsible for my team members and they, in turn, were responsible for me and, of course, none of those skills were even hinted at during my graduate program. Team research can be quite helpful in doing a project. Team members can be chosen to have complementary methodological skills as well as a variety of personal preferences for how to approach a research problem, as described by an urban scholar who like many whom I interviewed had to learn skills in running a team on his own: Research teams generally are a more productive way of doing research than doing it myself. So, I could assemble a research team that had a variety of appropriate skills. One of my skills, one of my duties, was to assemble that team and I then found that I liked working in research teams and I was decent at leading them. So, there was a whole learning curve on that … I found out that I was good at conceptualizing what the research was …, explaining to both the team and the people we were interviewing why it was important and what we were going to do with this. I was not good at all in writing it up, … I  quickly found I had to identify someone on the team who would write up, to the extent possible, everything that we had learned and that would in effect include interviewing members of the team on what they had learned. I found similarly that I am not well organized. So, when I would do quantitative research, for example, working on a data set, I was horrible at labeling the data set, I was horrible at keeping track where it was, archiving it … it took me a while and a lot of frustration with myself before I understood that I just had to assign someone else to do that for me. I couldn’t do it well myself. There were other things that I could do better. One of the things I found out about research teams was the need to find out what people can and cannot do and have them do what they can do rather than expect them to do what they can’t do. … I found that was true of me as well. A similar expression of the value of working in a team was expressed by a sociologist who, though skilled in statistics, still felt that: it helps again if you have some kind of partnerships, some methodological guru … not somebody who is just skilled at some advanced statistical

78  Enhancing Data Gathering Methods

technique but somebody who has the skill to make it clear to other people … I find myself in need of a guru also on occasion and to the extent that you have those kinds of people … that is important. In observational and interview research, partnering in a team can allow you to obtain complementary perspectives, something that might not be possible alone. A sociologist who studied workplace behavior in factories explained: you can be a participant observer with management or with workers, but not both. I resolved this … I worked in a team, so there were two of us. One with the managers and I would be working on the shop floor. … this was quite crucial in my understanding of how the enterprise as a whole worked, in the context within which the shop floor was organized. Teams can be formed with researchers who have quite different interests. An academic who studied mental health problems described: partnering up with people who have more an applied, practical orientation. They want to help people; they want to change policies but I’m more interested in more theoretical processes of behavior or community dynamics. Team research when well managed provides complementary skills in undertaking research but is one of those methodological skills that researchers need to master on their own.

Learning from One’s Mistakes No matter how careful one is or how hard one tries to follow best practices, mistakes are inevitable. Textbooks and courses don’t dwell on the mistakes you could make. They offer the right way, or at least a good way, to do things and do not envision the obstacles involved in strictly following textbook advice. There is a nearly unlimited set of possible mistakes you could make, and no methods course could possibly cover them all. Fortunately, as opposed to ruining a project, if you catch your mistakes, fix them, and learn from them, that can be to your long-run advantage. I used to worry a lot about the mistakes, and I’m embarrassed to admit how many I have made but was reassured by one interviewee who indicated making mistakes had a positive side. Most of one’s mistakes turn out to be opportunities for understanding things more deeply …. The more mistakes you make, as long as you reflect upon them, the more insights you can often get. Maybe the most humiliating moments in my field work have been the ones that led to the greatest insights into what was going on. In that sense, if you have not been humiliated and you are not having feelings … you are not doing good field

Enhancing Data Gathering Methods 79

work. … That’s what I believe. Yah, so I make mistakes all the time but as long as you reflect on them and at the time you make them.

Why Are Mistakes Inevitable? No study of a complicated topic is likely to be perfect. If you work in a team, you cannot control completely other people’s behavior; you cannot train everyone so that you include all the situations they might encounter. Some mistakes by team members can be caught by double checking. The researcher found in her study of the favela community in Brazil her team members were only interviewing people at the bottom of the steep hills in the area missing those higher up. She quickly rectified that situation. Qualitative interviewing involves listening and responding to spontaneous responses thinking on your feet while formulating the next question. You might miss an opportunity to follow up, to figure out what is meant, or hinted at without elaboration. You might not have done sufficient preparation to know what questions or topics are likely to be sensitive and stumble upon one by accident, shutting down an interview. And you can always mess up the technology—or I prefer to think the technology might mess you up, such as when I failed to turn on a recorder, and thought I had a recording when I didn’t. There are a lot of ways of messing up quantitative data too, including getting existing data sets in a format that you cannot read or interpret, making a coding error, or leaving off an answer category such as “I don’t know.” In any kind of study, it is possible to present yourself in ways that shut down rather than open up communication. You could describe your research topic in a way that is viewed as threatening, or you could present yourself as a student and ask to be taught, and then reveal the level of your training or expertise in an effort to elicit respect contradicting your persona as a naive student. You should assume you are going to make mistakes. Life, in general, and research, in particular, are not about not making mistakes; they are about catching them and doing what you can to fix them so they do the least harm, and then learning from that process. Think about what it meant when a question you asked was more sensitive than you thought it would be and you got a superficial answer. What might an interviewee be hiding? What does it mean that the question you thought was clear was misunderstood? Thinking about your mistakes this way can lead you into a deeper understanding of the data.

Common Mistakes I have made my share of mistakes doing research projects. Though sometimes embarrassing for the moment, I was generally able to recover and finish the project with success. I offer a few examples here to give you an idea of what can go wrong.

80  Enhancing Data Gathering Methods

In one study of organizations that advocated for affordable rental housing for the poor, I foolishly used questions I had prepared that were suitable for another group I was studying that was advocating home ownership for the poor. The questions completely missed the mark, and screwed up an interview.

How Did I Recover? When I realized how bad the interview was, I apologized and asked if I could talk to the interviewee a second time. Fortunately, I was able to do a do-over. I have always been impressed and grateful about how tolerant many interviewees are of my mistakes. Sometimes, interviewees correct mistakes real time, during the interview. In studying the land-use dispute, I mentioned earlier, I wanted to interview the environmentalists pushing to preserve open space. The city officials wanting development with whom I had talked with had been forthcoming and I hoped for even more openness from the environmentalist, as politically I was on their side. I had not done sufficient background work and had totally missed that those from the environmental organizations had been burnt before, I think by the press, and were very distrusting. My interviews with them were worth very little. That was not a recoverable mistake. Had I realized in time what was likely to occur, I would have spent much more time negotiating trust before trying any interviews. I generally try to earn a bit of trust by sending copies of my prior work on a related topic to my next set of interviewees. That way, they would know that I  wasn’t a journalist out to embarrass anyone, and I had done my homework, spent time learning their vocabulary and their issues, and was really interested in getting deeper into the material. I was signaling the level of my knowledge and how I worked. Mostly, that worked well. However, in one case, though, that process of signaling turned out to be a mistake. I had written a book on community development corporations (CDCs) from the perspective of the local organizations and in it I quoted those whom I had interviewed about their views about their national funders. Those I quoted used some very harsh language criticizing some of their funders and I did quote them verbatim. My next project included talking with the very funders who had been criticized. I sent my prospective interviewees a cover letter and included a copy of my book on CDCs. I waited a reasonable while and called for an appointment, only to find one of the funders was so upset that I had included the strongly expressed complaints of some of the CDC directors that he, in no uncertain terms, refused to have any dealings with me. Luckily, the other major funder was also upset but still wanted to talk and I had a first-rate informative interview with him, so the harm to the project was minimal. Everyone makes mistakes in both quantitative and qualitative work. Entering data erroneously is a common error, rectified by checking and double checking. Losing raw data is another hazard, so it is important to make sure the raw data are backed up. And backing up the coding sheets is also important.

Enhancing Data Gathering Methods 81

A survey researcher lamented: There definitely have been times where I didn’t back up codes and I had to try to replicate what I did … you get close and are not able to do it exactly or you don’t remember how you coded something and things like that certainly happened in my research. To me, a major mistake is not protecting your data; there is always a need to keep multiple copies. In my interview projects, I quickly back up recordings to my computer (and in doing so, double checking that the recorder worked) and then compulsively back up my interview recordings to numerous (don’t want to say how many or you will laugh) backup drives. When the interviews are not confidential, I also back up to some cloud services. More compulsive people than I, and yes there are some, do an extra weekly backup and store the drive with the backups in a safe deposit box. What happens when you are not careful enough with your data was shown by a senior scholar who remembered a mistake he made when studying political parties in which he and his research assistants spent months of tedious, though necessary effort, to locate historic party platforms as part of their raw documentary data: We did all this work on the state party platforms and when I moved cross country—I started the project when I was in the East—I was there for three years and moved back to west coast …I wanted to make a copy for my research assistant who was still back east, so we would each have a full copy of all the files. I hired somebody to Xerox and somehow in that process a whole bunch of them got lost. Let’s say we had 1000 platforms and 150 or so got lost. And I didn’t check before we moved, … I assumed that they were all there and I got here and they were gone and my (research assistant) he didn’t have them. The first thing we should have done is scan but back then (several decades ago) you couldn’t really scan, the quality of scanning wasn’t the same, but yah that certainly was something that sticks out in my mind as kind of costly mistake … When you make that kind of mistake, you learn what you should have done, and are much more careful going forward to not make that kind of mistake again. You cannot anticipate all the mistakes you could possibly make, but knowing that some are likely to occur, you need to build into your routines ways of catching errors see if there is any way of rectifying them, and devise ways of avoiding those mistakes in your next project. For a qualitative interviewing project, you should reread each interview shortly after you have done it, looking for weaknesses and erroneous follow-up questions. The public administration researcher whom I interviewed routinely sent transcripts of the interviews to the high-level officials she interviewed,

82  Enhancing Data Gathering Methods

asking them to correct any misinterpretation or mistakes of fact that she had made, and letting them elaborate on points where they had been brief or evasive or they had misspoken. Building in a second interview with the same person also allows for corrections of mistakes, and having a backup, that is, choosing someone else who knows the same or similar information in case one interview doesn’t work is a handy way of avoiding a serious error. For quantitative work, it is important, in preparing surveys, to check out each question with a set of informants with whom you can discuss the question, see how they understand it, see if there is enough context provided to make it understandable, and if it captures what you intend it to capture. You also want to know if the answer categories you provide feel right to the interviewees. You should build in computer checks for errors in data entry, and yes, keep multiple copies of code books.

Conclusion This chapter presents some issues not routinely covered in college research methods courses on data-collection methods. Some techniques are ignored in courses, for example, when those departments with a quantitative bent gloss over qualitative research methods or department that emphasizes that qualitative work offers less guidance and fewer courses on quantitative analysis. Research methodologies evolve, so what you have learned in school can be out of date. New statistical procedures come about frequently, while datamanagement tools are constantly improving. Research is often conducted in teams, yet how to reconcile the skills (and personalities) of team members is not taught, much less what to do when you oversee a team. Even when the texts and courses are clear and comprehensive, it is not always possible to follow their advice. Assumptions about doing samples maybe be irrelevant when people don’t have street addresses or phones and phone numbers. Sampling frames need to be improvised. Cleverness is often needed when gathering data as situations totally unanticipated in classes occur. Textbooks generally under emphasize the possibility of making mistakes and, hence, do not dwell on ways to recognize them, recover from them, and learn from them. Making, and learning from, mistakes is part of a researcher’s professional growth. Knowing that successful researchers also make mistakes and have usually recovered from them should help build confidence that you can also do so. Learning how to do research is continuous and the longer that you are at it, the more skilled you will become. The core lessen from this chapter is that while textbook learning is important, it is incomplete, so improvisation and cleverness are often necessary and learning is never finished.

6 COMBATTING OBSTACLES Finding Sites and Individuals and Handling Logistics

You’ve defined your research question and have worked out a design. You have figured out what data gathering tool to use and are now eager to get started. But one more step is required and that is anticipating what obstacles you will meet and, more importantly, how you will overcome them. This chapter and the next describe some of these obstacles with offering solutions that have been suggested. The immediate problem encountered in much of social science research is that neither people, nor institutions are eagerly standing by, waiting to be studied. You must figure out how to gain access to the research sites as well how to elicit cooperation from individuals whom you wish to study. Next, you face various logistical problems—getting to research sites and reconciling your schedules with those whom you wish to study. Unlike on Star Trek, one cannot simply be beamed to a research site. In this chapter, I’ll describe problems in gaining access to sites and individuals along with some of the logistical problems researchers face. Chapter 7 examines less common problems mentioned by the interviewees.

Access to Sites and Individuals Access Is a Concern No Matter the Data Gathering Technique In doing surveys whether by phone or face-to-face, you have only seconds to convince someone to respond. Doing so has been made much more difficult by scam artists posing as survey researchers, by the increase of push polls (polls for political campaigns intended to persuade, not to learn what people think or feel). You have to present yourself in a way that distinguishes you from the scammers, sales people posing as researchers all the while convincing potential respondents to participate … DOI: 10.4324/9781003345909-6

84  Combatting Obstacles

Even more effort is needed in encouraging individuals to participate in a depth interview study as these interviews can be time consuming and often involve several sessions. In this case, researchers need to show the potential interviewees that what is being studied is of value to them, not simply to the researcher. People do not have to say, “go ahead and take several hours of my time.” Access problems can occur even when the research involves using in public documents stored in government buildings. At first glance, gaining access to public documents would seem straight forward. Public means public. However, some public data sets have constraints on their availability. For example, in this internet age, most documents produced by the U.S. Census Bureau are readily available through a few mouse clicks; yet, some census material is protected as the census bureau assures that individuals cannot be identified from public census data. For some projects, a researcher might need to use micro data that could reveal individual identities. The census bureau has worked out a system that grants permission to use such data to some who swear in a checkable way not to abuse it. Getting that permission is not easy but is possible but can be an obstacle demographic researchers face. Similarly, medical information that can identify individuals is and ought to be confidential but legitimate public health concerns might require a micro analysis in which identities could accidentally be revealed. Again, for people with suitable credentials, ethical ways of doing such research exist. But, getting permission requires jumping through hoops (metaphorically). Most other government documents are by law, with a few specified exceptions (private personnel records or defense plans, for example), are available to any citizen, though sometimes only after a designated waiting period. Still, recalcitrant public officials might block researchers (or advocates) from seeing these available documents, perhaps by not admitting that they exist. For most government organizations there are laws called Freedom of Information Acts (FOIAs) that require the release of requested public documents within a reasonable time and at nominal cost. Newspapers use FOIAs all the time. But even with those provisions, you cannot ask for a document unless you know it exists. At times, researchers can make an informed guess. When a city has contracted for a major road construction project, the contract, the record of payments, and any penalties for nonperformance all ought to be available, even if tucked away in some bureaucratic cubby hole. You can reason after seeing the road repair that such records must exist. At other times, a hint is required from a helpful public employee that such documents exist. In an earlier chapter, I described a situation that occurred in my study of zoning in which a helpful bureaucrat leaked information to me that I never would have guessed existed but technically was public data.

Choosing a Site and Gaining Access to It Several relevant sites for a project might exist. In selecting where to study, you might want to factor in the costs in time and money of getting there along with

Combatting Obstacles 85

reckoning how much time you can devote. The choice is more difficult when a preferred site is further way or more expensive to access. In my study of community development corporations (CDCs), the most interesting ones were on the two coasts too expensive for me to repeatedly visit. On the other hand, there were only a handful within a short drive of home. My compromise was to visit CDCs no more than an overnight drive from home, at a cost, but nothing like repeated coastal flights, yet encompassing a large enough area to provide a variety of organizations engaged in different projects. After you have chosen what seem to be appropriate sites or organizations to study, you need to gain access. If access is denied, you must go back to the drawing board and decide if there are different sites or organizations germane to your study and try again for permission. If there are none that meet the requirements for your study and are feasible for you to reach, you may have to seriously alter your research question or simply change what you are researching. Access can be a critical choke point. One of the interviewees described her frustration when access was denied. She had worked out an exciting project on government budgeting and it would have been even better if she could use the archived papers of a former high-level budget official. To her dismay, she found out that the former official, now at a university, only allowed access to his papers to his own students. Sometimes, a simple redesign is possible if only part of the access is blocked. You can reduce the scope of the study if the remainder will produce meaningful results. As described in a previous chapter, the doctoral student who was studying university budgets could not gain access to private universities, so redesigned her project to focus only on the public ones which by law had to allow her access and, in doing so, dropped her questions comparing public and private universities and added those germane for the public universities. Seasoned researchers have worked out routines they follow to minimize the threat of denied access. The most often mentioned was using personal and professional contacts, people who know the people you are trying to reach, and can vouch for you and the importance of the research. If you start a project without a recommendation from someone known to those whom you want to study, other approaches are required. One way is to you hang around in settings in which those you want to study are present, as I did by attending conferences of those whom I wanted to study. If you chat with people, if they get to know you, you are more likely to be accepted. Better yet if you are part of the community, or you work in a place they know, if they know your boss or supervisor, those whom you want to study more readily will be willing to work with you. If in seeking access, no one responds to letters or phone calls asking, keep trying, doing so in a distinct, polite, and persistent fashion. When contacting busy executives, be willing to accept an appointment some months off if they say they are now quite busy. Some of those I interviewed when studying executives in larger organizations spent time cultivating the administrative assistants of the higher who then facilitated setting up appointments with their bosses.

86  Combatting Obstacles

A survey researcher, recognizing the increasing difficulty of getting people to agree to answer survey questions, underscored the importance of persistence. You just be persistent, you know, you leave one voice mail or one e-mail and you don’t hear back and you try again another week, then you leave another, you keep trying once a week for as many weeks as it takes to either get somebody to say ‘yes’ [or “no”]. If you get someone and they say no, I’ve changed my mind, then you respect it. But to make that [initial] contact is often much harder [than it used to be]. It takes persistence. In requests for access, it is important to show respect to both the individual and the organization. In a study of municipal budgeting, one researcher included in her letter of request for an interview that she knew the city was famous for the quality of its budget document, and so wanted to know how they had worked out such a budget, along with what obstacles had been encountered, and overcome. She phrased her request in a positive, complimentary way indicating that she wanted to talk about matters with those who were informed on the subject. None of her sites refused her access; none of her interviewees said no. More generally, you should tell your potential interviewees what is special and of interest about their work and their sites, specifically what you want to find out and why they are the appropriate people to ask. Doing so takes some background work. In my requests to interview directors of community development organizations, I indicated my interest in specific projects that I’d named that they had already accomplished as reasons why I wanted to talk with them. That meant first reading annual reports of the organizations or looking for newspaper articles, or at times, asking others in the field what a particular organization has accomplished. In seeking access, you need to indicate the value of the project, perhaps personally to those with whom you want to talk, or suggest what they share with you will provide a guide to those working in their field later on. Other times, individuals and organizations want publicity for their work and through your reports and publications you can provide that publicity. You also need to how that you are a credible researcher who will complete a project in ways that are helpful to those being studied and that would cause no harm. There are several ways of doing so. If you have published articles or books, you can refer to them, or share them, both to demonstrate that you are a serious scholar and that you are not a journalist out for scandal. Doing so shows that you respect anonymity when it is requested. Previous reports and writings can also indicate that to some extent you understand the field being studied, so in talking with you or allowing access to observe they are not opening the door to some ignoramus. Your underlying message should be that you know something about their world, your work will be helpful to them, or their organizations or their profession. An important selling point is in communicating the importance of the research as well as the value to those being studied, not just to yourself. Sometimes, the

Combatting Obstacles 87

value to the participants is an opportunity to teach, maybe even to boast, to talk about their work and the value of their work or present their side of a dispute. At times, for the potential participants, there is some status gained in being selected. Respect counts in gaining access and traveling to distant sites for the research is a way of paying respect. The researcher may have to spent time and money to get to the sites but doing so suggests commitment and sincere interest. It is easy to put off someone who lives around the corner and can come back at any time, but harder to turn down someone who has flown across the country to talk to you or, in my case, driven several hundred miles. You need to think about what you could offer to those being studied. Material rewards have been used to encourage participation in surveys, focus groups, or social psychological experiments. Another possibility in gaining access is hinting that your work will enable their organizations to receive publicity. I believe that is why I was able to gain access to little known community development organizations or progressive organizations in Washington. In fact, once my study was underway, leaders of the organizations explicitly told me that that was the reason they were cooperating. These points were made clear to me after I had published a book on advocacy organizations for the poor. I had shared the book with many I had interviewed. The book did point out difficulties the organizations face but it still shined a positive light on the organizations’ flaws and all. A senior officer of one of the organizations quickly wrote a review of the book and had it published in major journals in their field, convincing me that I had gained access because of my potential to publicize their work. Sometimes access is possible simply because people are flattered that someone care enough to ask about what they do. People do like to talk, perhaps brag about what they have accomplished, or vent about the obstacles they have faced and overcome.

Becoming a Known Quantity If people know you or know of you, they are more likely to comply with requests to participate. As I’ve mentioned, I would attend trade association meetings to which the organizations I wanted to study belong. At the meetings, I would introduce myself. Later on, I would e-mail or write letters to those I wanted to interview and follow up with a phone call. I would remind them I had met them at such and such a meeting. I was a known quantity to them. And during my meetings with them, I would share stories of what had happened at the meetings we both had attended. Being seen also helped in my studies involving local government officials. While city council meetings are public, few citizens attend. For years, I attended city meetings if even they did not concern issues I was researching. Being in the audience for several meetings makes you noticeable to the elected officials and the staff who may ask what you are doing there, giving you an opportunity to

88  Combatting Obstacles

introduce yourself. Later, when you approach them for an interview or ask for some documents, they will be more receptive.

Letter of Introduction Prior to the prevalence of e-mail, I’d write a letter on university letterhead to request appointments. The letterhead suggested that I was a legitimate scholar and indicated that I was embedded in an organization whose reputation I had to protect. Later on when I used e-mail to introduce myself and the project, I always used my university account with the @ showing the university name, rather than my personal g-mail account. In the letter or e-mail, I explained the project in nonthreatening terms, described why I needed to talk to them personally, and provided contact information. To bolster my credentials as a scholar, if I had done prior research on a related topic and had copies of publications I would enclose them in the initial package along with the explanatory letter. I once did that with a book, though more selectively as books are expensive even for the author. Surprisingly, those who received the book read it or at least skimmed it. The articles or the book ended up being part of the introductory, get to know you, part of the interview. The articles also demonstrated that I did protect confidentiality of those whom I had previously studied. At times, those whom you want to study may be hesitant to talk to you or allow you to observe their observations because they fear you might not be on their side. Suspicion can be triggered by a misunderstanding of the purpose of your proposed study; clarification is required. A researcher working for the Civil Rights Commission described such a situation: I was having difficulty setting up appointments … we were trying to get an appointment with the fire department. And, they were being very resistant to talking because they were always getting criticism … And, finally one day in our communication I indicated that the focus of our studies was the insurance industry not the fire department … when I made that clear, they opened up, they welcomed with open arms, they invited us in and they said they hated the insurance industry because, from their perspective, all the home insurance industry did was price their product to whatever the fire losses were. In other words, they weren’t doing anything to try to reduce losses. The fire fighters thought that they were out there doing God’s work, trying to keep people safe, and the insurance industry was a cost plus business and so if there was a million dollars in fires last year they would charge what be the appropriate amount next year. … It was pretty clear once they understood what the focus of the investigation was. That made all the difference in the world. Getting those whom you want to study to read your request and respond can be difficult. People get a lot of snail and e-mail and have learned to ignore what seems less important to them.

Combatting Obstacles 89

One time I wanted to interview mayors face-to-face but was concerned they might ignore an e-mail request from a total stranger and not a constituent. I wrote letters describing the purpose of the project, my credentials, and the value of the project for citizens, but wanted to make the letters distinct enough to stand out. I purchased heavier and larger envelopes, had my laser printer print the letters on heavy bond paper, and purchased stamps with interesting photos; the package caught the attention of the potential interviewees and almost all agreed to an interview. I routinely followed up a letter or e-mail with a call to answer questions the person might have. Usually, the call results in a scheduled appointment. At times, when I reached the person whom I wanted to interview (rather than a gate keeper), the follow-up call itself would turn into a preliminary interview. When that happened, I paused and asked permission to take notes or record. Of course, luck counts. Your reputation may precede you; your interviewees may recognize your name. I got some interviews with administrators in local government because they knew my wife who had been their teacher; she got some because she was the editor of a journal that many whom she studied received, and as such they recognized her name.

Getting Permission in a Foreign Country Gaining permission to do research in a foreign country can be a whole different, sometimes problematic, game. Many countries maintain offices whose approval is needed prior to outsiders doing research. Paperwork is required and background questions may be asked. For my work in Thailand, I had to apply to a special office, describe the purpose of the research and my background, and then wait. And wait. And wait. That wait had to be factored into the research design, as it reduced the amount of time I would have in the field. The upside was that once my proposal was approved, that office provided help, for example handling visa requests, allowing me to avoid the reputedly bribe-seeking immigration office. A researcher in Indonesia described that in order to get permission, she had to go to: Jakarta (where) you are given a couple of hours or days to get certain` official papers in place and if you are not able to do all of that than you have to start all over again, the process was very bureaucratic, and then you have to repeat the same process in the provincial capitals. It can be a pain in the butt, albeit a necessary pain.

Getting By Obstacles At times those whom you want to study seem hesitant to cooperate fearing your intentions or being concerned that your background is quite different from theirs. This can be a problem for researchers in studying within minority

90  Combatting Obstacles

communities or nowadays among immigrants, assuming that you are in neither group. It also can occur because of a misunderstanding of the purpose of your work. The example provided above of the troubles a researcher for the Civil Rights Commission in accessing a fire department illustrates such a problem and its resolution. An organization or individual who has been criticized or burned by someone who took advantage of their willingness to help, is more likely to refuse access. In that case, you may need to prove your credentials, identify convincingly that you do understand their world and want to learn more. One approach is when you are accepted member of the group you want to study. Another is to find someone in that group or at least accepted by that group willing to act as a sponsor. A person interviewed for this book, a straight person (that’s relevant) described such an experience in her studies with LGBTQ groups: Often community organizations can be very protective of the things that they do. For example, when I was doing research with … organizations that support LGBTQ rights and I was not from that community, they did not allow us to formally, to consult with them or interview them. So, my colleague and I had to go in circles in trying to interview some members … from the LGBTQ community who were not part of the organization who let us know about [our research questions]. So we went around the difficulty of not able to fully gain the trust of the organization. Because they had historical trauma and because they had been attacked, they are constantly under attack, and they have become overly protective. Sometimes access is harder to get simply because people are busy with other matters and being researched is not a high priority. A sociologist who studied police found a way of reducing this obstacle by finding a setting in which responding to the researcher was not an inconvenience. He explained: I did this study with police officers. They take these training classes … police procedures, legal issues. And police officers have to come from a variety of departments … and they go to this one central location and so while it would probably be much more challenging to get several municipalities to come to their police departments and administer the surveys, here you have an opportunity where they are coming from various locations and they are in one place right and so if you can get the cooperation of the people who do the police training at the beginning of each class. Here is the professor …from (university name) he would like to take a few minutes, … and, they are right there, they are captive … Instead of trying to get the permission from multiple cities, the researcher needed only to get one from a police trainer when the officers were away from catching bad guys.

Combatting Obstacles 91

Handling Logistical Problems Data do not magically appear on your computer drive or in large piles in your office. You need to locate research data, at times requiring physically getting to different locations. The most dramatic logistical problems in getting there were those of cultural anthropologists wanting to do research in some remote area, but logistical problems also occur in many other less esoteric projects. Telephone surveys require a trained staff and a facility with phones and recording equipment. Putting these together can be a challenge in logistics. Even when a survey lab is in place, researchers need to assure that trained interviewers and supervisors are present during the hours at which people are most likely be at home, something that can be difficult when a national study crosses multiple time zones. Bringing together scattered documentary data requires logistical skills, as described by a political scientist who was working on a book on the history of political campaigns: HJR In getting this data, did you face any particular obstacles in finding it? Yah, … the state party platforms …. collecting those, … Each state has its own situation, each state party and so basically it involves like detective work with myself and [research assistants] … for each state and party … Well we will call the state library first, and often they don’t have it, they’ll have three platforms but then you go to the parties themselves and they almost never have good records. Go to newspapers, go through old historical newspapers and just like that …, and you get the platforms and have to scan and digitize them and so on and just track it and make sure you are not duplicating effort and so on. Putting together scattered documents can create a challenging problem. For anthropologists, getting to the research site can pose significant challenges. The anthropologist who studied sea nomads (individuals who moved from Island to Island) had to overcome daunting transportation problems: … I was a student in England, so I flew from England and then went to Singapore (her family home) …. before flying into Jakarta and from Jakarta you are given a couple of hours or days to get certain official papers in place … and then from to the capital of (the) archipelago which still is very far from my (site) and then get a couple more papers, report to the university there and then from there fly into the … archipelago and then from there I had to search for fishing boats that could take me to the outer island for the sea nomads. She continued, noting further logistical difficulties in meeting those being studied: … it’s not possible when trying to do this project to work on a kind of schedule that we are used to. Like over here you know, say eight o’clock, I’m going

92  Combatting Obstacles

to do this each day, one o’clock I’m going to do that. It is really not possible. So, for example, in trying to catch a ride from a fishing boat from one point to another point. There are no fixed times, it goes with the tide. And whether they are finished loading the fish or not. So there have been times where I had to wait at the harbor thinking it is going to leave now and then you realize that they missed the tide because they were still loading so you have to spend a whole night and hoping sometime the next day it would go….. Even decades later, while transportation had improved, the logistics remained complicated. So for these days, if I am just going back to visit sea nomads I simply go into Singapore and then from Singapore there are boats that can take me, hydrofoils that can take you into the city areas in the archipelago and then from there you just have to try to find a fishing boat that can connect you to the other islands that you want to go to because all these islands in which I do field work, there are no commercial boats [that can take you]. In recent times transportation problems have eased, as a scholar of China indicated, but security concerns create logistical obstacles: It used to be I would have to take a bus from the city of Xian and then I’d go to [place] and from there would have to take a smaller kind of van and then you get off and from there I’d get on the back of a tractor and bounce around and sometimes I just walk and kind of catch a ride from somebody. It used to take me a day, sometimes two days and sometimes on the way, I’d have to sleep on blankets on board planks and it was technically illegal for foreigners to be in rural China because you have to be at a hotel where you could register but there is no place to register in the country. …. After 2010 to 2015 massive infrastructure investment in China what used to take us ten hours …. maybe sixteen hours, now takes about two or three. However, there still were security checks along the way that could interfere with the research. I never got searched but I was told to be prepared for it and keep my research data separate … They said in case you got, in case I got searched and then. And, then the assumption that I was a spy. In my own case, since I had never traveled, going 10,000 miles to Bangkok and then going up country was a logistical challenge. Travel agents could help with international travel but, once in Thailand, my wife and I had to figure out how to get from place to place—negotiating the bus system in an expanding Bangkok to find a research center, or riding a bicycle over mostly dirt roads to get to a distant village,

Combatting Obstacles 93

without being able to let anyone know in advance I was coming as there were no phones, or finding a medical facility to get a needed booster cholera shot. Logistical problems beyond simply getting there can pile up, as a frustrated bilingual researcher described for her research in Mexico. It was just an endless number of logistical problems … when you are doing international research, getting there, family, getting a computer into the country, getting a visa to stay longer than a tourist, you know, involves learning about the whole system of visas and pay somebody to stand in the line for them after standing in line for a couple of days. Some of my own logistical problems were self-imposed by my lack of knowledge of an unfamiliar place. In the Philippines, I separated from the evaluation team to visit on my own a ministry that I thought would have some relevant information for our project and with directions from the officials who had hired us I easily found the place (it didn’t have the information.) I was supposed to meet up with the team at another ministry office and mistakenly assumed the name of the ministry would be sufficiently well known for a taxi driver to get me there. That didn’t work and, to this day, I can’t remember how I found the offices of the other ministry. Even planning ways to get around the states by car requires thought, especially when research sites are scattered over multiple cities. And doing so can require coordinating schedules of busy people. I remember one triangular trip from home, 60 miles west of Chicago, to Cleveland, to Columbus, and then to Cincinnati, requiring coordinating four schedules—my own and three individuals who were part of the study. On another occasion, I had scheduled interviews in Philadelphia first, then Washington, D.C., followed by Columbia, Maryland, with the constraint that the person in Maryland was senior, very busy, and required me to schedule several months ahead of time. Things worked out with some planning as the person in Philadelphia had enjoyed a previous interview and readily accommodated to my schedule, and in D.C. there were several people I could interview, so I found the one whose schedule best coincided with my trip. Managing logistics requires flexibility, not only to adapt when an interviewee cancels, but also when unanticipated opportunities for further research occur. Minneapolis is almost 400 miles from my home, so I arrive the evening prior to my interview appointments and schedule meetings so that the interview on the last day ends midday and I can drive home when it is mostly light out (and I’m awake.). On one occasion, the morning interview on the last day was finished when the interviewee, now clearer on what my research was about, mentioned a meeting beginning at five that day that was germane for my project. I stuck around walking around the city for fun until five, attended the meeting, and at 6:30 or so started driving home. I had classes the next day so couldn’t stay overnight, to say nothing of the expense. I arrived home exhausted in the wee hours.

94  Combatting Obstacles

That long drive reminds me of an important practical point for handling exhaustion when you are traveling to do research or are doing intensive observations that fill up the entire day. Several of my research projects involved observing the conferences of organizations I was studying. Conference days are jam backed starting with breakfast sessions, panels throughout the day, group lobbying efforts, and receptions in the evenings, often with informed and interesting speeches all of which were relevant to my research question. That’s a tiring enough day. But, time is needed (usually out of sleep) to write down observations. Eventually, I figured out to bring a recorder and dictate notes to myself, something I can do far more quickly than writing down my observations. Later, as soon as I have more time (and sleep) I would transcribe my recordings. Nowadays, with security screening at airports, logistical concerns can occur in transporting equipment needed for the research that might look suspicious. Right after 9/11, I was flying to D.C. I took several recorders in case one fails (or I fail to turn it on) so I was carrying two recorders and a tangle of wires that permitted me to back up the recordings on my laptop. When x rays showed that mess of wires, security required me to assembly my apparatus and turn it on to show it wasn’t a bomb. Fortunately, more modern digital recorders have such a large capacity that immediate backup is no longer required and they are easily scanned with other carry on. Getting equipment into the research site proved problematic for one researcher. A criminologist was recording interviews at the state penitentiary and needed to get recording equipment (often unauthorized) into the facility. He described the need: to smuggle a tape recorder for your interviews … Herb: and you did smuggle? Yah yah, I did, and actually, for a while, one of the wardens let me bring it on occasion, he was a progressive warden … The point of that is he was sympathetic but more often than not I would have to smuggle one in, which wasn’t easy. Once in a while they stopped me, and I had to come up with some sort of an account to justify it. …. You went through a metal detector and then they patted down your body and it would seem hard to slip anything by them, but it was pretty loose in how they actually did it in the practice. It was a small tape recorder … I would just stick it in my sport coat, take it off pass it through the detectors where it won’t click and then put it on again. Because taking off your outer garment was routine and so that is how I got that in. Logistics are not always straightforward. You have to think about how to get to where you are going, what you are going to take with you, how closely to schedule interviews in case one runs over, what to do if someone cancels out (and you have spent time and money getting there), and how to adapt when some unexpected

Combatting Obstacles 95

opportunity arises. Will you be able to stay overnight? Can someone else take over for you at home for a day? If a flight is canceled, and you can’t make an appointment or get home when you expected, what are your alternatives? You can’t exactly plan on the unexpected, but you can try to build in flexibility just in case.

Conclusion Much of what is discussed in this chapter is hinted at in texts and methods classes. Clearly, you can’t do research if you don’t have access to sites, data bases, archives, or individuals. But, far more preparation is needed than is covered in courses on how to gain access to research sites and handle numerous, and apparently inevitable, logistical problems. Logistical, organizational, and management problems abound. Lacking a budget and released time from teaching there was no way of my studying east or west coast organizations. Even coordinating my schedule with those I wanted to interview in the Midwest was tricky. For survey researchers, assuring that interviewers are available to mesh with times when people will be home in a country stretching from the Atlantic coast to Hawaii requires careful planning. The goal is to anticipate problems as best you can, work out solutions in advance, and leave plenty of time to come up with options when glitches do occur. Also don’t be frustrated if everything does not work out right. Those you want to study might demur or leave the study long before you are finished. Equipment can fail (though bringing duplicates can help). Though many problems cannot be prevented, by anticipating them you can often work out ways of minimizing the harm they cause to your project.

7 OTHER OBSTACLES FACED IN COLLECTING DATA

Chapter 6 examined obstacles that occur in accessing sites, convincing indi­ viduals to participate along with the practical logistical problems of setting up interviews, getting to sites, and assuring needed equipment is at hand. This chapter continues the discussion of less frequent but still significant obstacles that can occur while doing research that can negatively impact data collection. To begin, after a project is underway, the external political, social, physical, or health environment might change handicapping data gathering. Who when designing a project would anticipate a pandemic that made travel difficult if not possible and face-to-face encounters potentially dangerous to one’s health? Or as happened to me in the Philippines an active volcano precluded visiting cer­ tain sites. Political regimes change and with them what research is permitted. Less dramatically, applied researchers can encounter problems when political or bureaucratic agencies seek to slant results. Push back, including lawsuits, can be stimulated by those who feel threatened by the potential findings of a project … While the word research conjures up a white-coated person in an isolated lab, or a survey researcher in a room with a bank of computers and telephones, safely hidden, some research projects pose physical dangers to the researchers and their staff. I’ve had guns pulled on me twice while doing research and spent months embedded in an area where violent antigovernment attacks were common. Most methods courses emphasize the imperative of research ethics—assuring that you protect those whom you are studying and that you report accurately the findings and research technique. The examples discussed are often of dramatic ethical abuses such as the horrible situation in Tuskegee when African American men with syphilis were not provided with antibiotics or when researchers had to go to jail rather than violate promises of confidentiality. Methods courses typically do not spend much time on the less newsworthy ethical dilemmas that occur during data collection. For example, what should have been an ethical DOI: 10.4324/9781003345909-7

Other Obstacles Faced in Collecting Data 97

response when in my research in Thailand an official whom I was studying gave my wife some gorgeous silk cloth that we knew he had paid for it with money he had stolen from the teachers’ fund he supervised? (After we left, he was arrested.) The remainder of this chapter will focus on a series of these less common but still important obstacles that occur as research is underway …

External Obstacles Research does not take place in a political or social vacuum. External circum­ stances can change and impact a project. It is not uncommon to launch a survey and while it is ongoing, events occur that could radically change people’s minds on the topic. Just imagine asking people about their attitudes toward people of the Moslem faith and while the survey was underway, the 9/11 attack occurred. Do you discard the initial pre-9/11 responses, treating the later ones as the real answer, or assume that the anger will eventually die down, and the answers would be closer later to the original answers? Is your data no longer meaningful? As I write, we are still in the covid-19 pandemic, something that clearly could not have been anticipated in a research design developed two years before. Faceto-face interviews are now problematic. Is a zoom interview the equivalent of sitting across from a person so they can see you take notes or hold up your pen when asked not to include something they just said? A scholar described constraints posed on her research in Costa Rica by the pandemic: the University … released a ban on in person interviews … so I am not allowed to interview in person, so I am going to have to do it by zoom or some other method to get my interviews done. … I’m in Costa Rica (her research site) but not allowed to contact people. Hopefully, the pandemic won’t be forever, but political changes occur all the time, either through elections or more violent turnovers. Political changes can pose serious obstacles to data gathering and, at times, pose personal danger. An expert on Peru had to switch her research topics and the site as “the conflict there was such that I needed to leave, just as most researchers from the U.S. and elsewhere needed to leave.” In her alternative site, her research was also curtailed through political instability. [The environment] had been very turbulent … U.S. sponsored contra war … People had voted the Sandinistas out because they were tired of the war …. largely a motivation for voting in the neo liberal government, but there was still a lot of discontent …. Nicaragua was suffering once again from the harsh structural adjustments…. And, what that meant was, when I was working in the early nineties in Nicaragua, it was a time when it was rather dangerous.

98  Other Obstacles Faced in Collecting Data

When tensions were great between the United States and Nicaragua, rules were set up by governments that impeded research. I very much needed my laptop, and I couldn’t take it with me. I had to ship it and it took a month to get it in. So, there were those kind of conditions. U.S. relations with Nicaragua, how it impacted my work. When there is violence, local officials might obscure what is happening so that American researchers do not report on it. I was interviewing a Thai official one night when another official appeared, shouting out that there had been a shoot­ ing between Thai police and tribal rebels. I asked for details of the fire fight and the official, realizing suddenly that I was there and understood, denied that there had been a shooting. With that kind of obfuscation, it is difficult to figure out what officials on other controversial matters were willing to talk about for fear the matters might be politically sensitive. The U.S. government opposition to the Cuban regime decidedly impacted how (or even if ) research was conducted in that country. An anthropologist interpreted this opposition from Uncle Sam in what she felt was intimidation. I go to Cuba and then … my taxes were audited and the fact that I had indicated in my taxes some travel research related expenses to Cuba …. and also remember the woman I shared a room with during that trip in Cuba …. was visited by the FBI after her trip to Cuba. Another researcher had to reformulate her doctoral project on the go as the political environment had dramatically changed. When the military took over the research she had just started on the country’s new capital, such studies were now forbidden.

Politics and Bureaucratic Games Can Create an Obstacle Textbook research procedures can be impacted by political pressures those in gov­ ernment bureaucracies feel. I experienced this during my year at the Department of Housing and Urban Development (DHUD). A company had received a contract to interview about one of the department’s programs. The plan submitted by the company was to interview in many states. With the modest size of the budget, that design meant that much of the limited funders were going to be spent on transportation reducing the money available for the interviews seriously limiting their number. I was assigned to supervise this project, though only after it was underway. I studied the design and was seriously worried that the dearth of interviews that would lead to less meaningful statistical data. I worked out an alternative design that would entail visiting far fewer states but doing many more inter­ views in each (technically cluster sampling) and, by doing so, dramatically reduce

Other Obstacles Faced in Collecting Data 99

sampling error. I presented the idea to the contractor and my boss, thinking I had increased the accuracy of the research at a reduced cost. My boss called a meeting where I explained the technicalities of what I was suggesting. (At that time, I was skilled in survey sampling.) My boss listened, indicated that he understood that I was technically right, but then rejected my idea saying we would do the way the contractor had proposed. I was puzzled. His explanation made clear how political bureaucratic concerns were valued more than methodological accuracy. The research office wanted to  have data from many states so that the department could present the state results, tacitly lobbying for program support, with a larger number of Senators. The need for political outreach dominated at the cost of the accu­ racy of the final report. My boss, probably correctly, understood that those in Congress would not care about the statistical accuracy so long as they received the figures for their states. In another incident, my boss assigned me to study documents describing projects paid for by the DHUD in a city he had chosen. I examined a huge file of documents, wrote the report, and submitted it to my boss, though wondering why the focus had been only on that particular city when the pro­ gram was nationwide. It turned out that my boss wanted the material to ingratiate himself with the political appointees of the newly elected administration (of the opposite party of the outgoing administration). Turns out that the mayor of the city I was told to study was the favorite of the President leaving office. My boss wanted the report so he could provide the new administration with data to attack its predecessor and by doing so ingratiate himself to his new political superiors. My research ended up as part of a political hatchet job, very much to my chagrin. An interviewee recounted another story about his experience also at DHUD that to him typified the difference in slant and purpose between academic research and that done in government agencies. He began describing I was at a meeting with representatives from various, different (research) offices … the person who organized it said, remember, we are here to make the Secretary look good. And I started laughing and everybody in the room stared at me and said what is so funny? I said I’m envisioning a collection of faculty … and somebody saying remember folks we are here to make the president of the university look good. I said, it is part of our job responsibilities to be critical of higher administration and the difference in culture was incredible between a government agency and an academic situation. There are decidedly differences between academic studies and those undertaken by government agencies, organizations, or businesses for applied purposes that can contradict the ideals of social science research.

100  Other Obstacles Faced in Collecting Data

Other considerations occur when researchers undertake projects to answer academic questions that the researchers believe will have relevance for policy. In this case, the wisdom is to work out an original design so that it would make sense to those in the policy world who are not trained in research methods. When one of interviewees recognized that his academic research would have policy implications, he made sure of “taking the extra step of translating it into policy recommendations and presenting them to local, state, and sometimes federal agencies,” in ways that they would understand, eschewing academic precision but accounting for political constraints that those in office faced. He explained that: I did a fair amount of [academic] work on energy efficiency. … we did an evaluation of the utility rate structure …. meant that you paid less the more you used. So, they encouraged consumption, rather than energy efficiency. And … that the electric companies …. were sending out information in their bills where it was technically impossible for the consumer to know how much they were paying … We wrote our papers, the journal articles come out. And, then we were engaged by the state Public Service Commission that regulates utilities and asked to apply the analysis which we did and found that the bills here were also inscrutable and made a series of recommenda­ tions. about (what) … the utility should do to change its billing practices to enable people to understand how much energy they were using and how much they could save by changing some of their behavior. The academic research was quickly published, however, to encourage those in office to understand and then implement the policy changes suggested by the research took years, as there was opposition, as expected, from the utility com­ panies. When providing research results to policy makers, researchers need to be prepared for political opposition, not based on the quality of their work, but on the push back from impacted interests. Another obstacle faced by researchers involved in the applied world is that there is tendency of officials to pull out of research reports findings that seem to support their predisposition, irrespective of what the reports actually say or the accuracy of the information. What should an informed researcher do? The per­ son who did studies on the utilities reflected on this broader question. I’ve run into many occasions where conclusions were reached by public offi­ cials with information provided to them [from politically slanted research reports] that was inaccurate … I would try to share with them the accurate data … I found that better than going in there and railing against the people who had been briefing them. … Policy work is demanding you have to be a little bit patient … So, the recommendations that [we made to change the educational funding system … were made ten years ago … so you know if you are going to get in this realm you have to have real stamina.

Other Obstacles Faced in Collecting Data 101

In general, those doing policy face a whole set of different obstacles than those who focus on more academic studies face.

Push Back and Attempted Censorship Several researchers reported unexpected push back while doing a projector or reporting the results, with opposition coming from businesses, elected officials, and, sad to say, from administrative officials at universities. One researcher, just out of graduate school, was in the early stages of a study on city budget processes when some (local) bankers contacted the President of the College … for which I worked and tried to stop me from doing the research … I don’t know what they were afraid of or what they thought I might find. They might have thought that I was a journalist who was poking around and somehow or other there might have been some shady loans or shady deals or maybe even some contact between the banks and organized crime. But it wasn’t anything that I was looking for. So, I continued to do the study. I was a little concerned about my job HJR: what did the President of the university do? He basically told me to stay clear [of the banks] … I said, I ‘m not inter­ ested in doing the banks, …. my actual research had nothing to with the banks, so I didn’t hear anything else after that but certainly that was an utterly unexpected pressure … and that was something that I had never been taught and never even thought about. …. I was so naive I didn’t expect to see anything overtly illegal. … Another case of push back was faced by two researchers confronting them with an ethical dilemma: My co-researcher [her husband] and I wrote a piece on New Haven … we had been part of a research project … examining the impact of the Community Development Block Grant program …. we interviewed a lot of people … we wrote this paper on New Haven and the rule being, [since] HUD financed the study with what then was 12 million dollars … they had to be the first reader of anything written and they didn’t have the right to censor it, but they had the right to comment … So, at any rate, they didn’t like this paper and we were on our annual vacation in Maine where … there was a note stuck in the door … call the Provost. I made the phone call and he said, this was a paper we were going to present at the annual meeting of the American Political Science Association, you cannot present this paper. And, I said, why not? and he said because the mayor of New Haven does not like it. And, for HUD of course their only clientele was mayors … and the Provost … was afraid that [DHUD] would

102  Other Obstacles Faced in Collecting Data

take away their 12 million dollars if we presented this paper … A paper presented at APSA was not going to make great waves in the world and nobody would even know about it. … We said, we are going to present the paper. And, basically, they said, well if you are going to present the paper, you can’t stay with this project which was in fact providing enough money to pay my son’s tuition in college. But I said we are going to do it so we did and we were separated from said project. Those working in governmental agencies sometimes are required to clear their writing with higher ups, a form of proactive push back. Some agencies han­ dle these matters well, without providing roadblocks for the researcher; others heavily censor unfavorable research. A sociologist described an event early in his agency career of his experience half way between the extremes. When I was at the Civil Rights Commission, I once published an article in the New York Times, and I got a letter from the staff director. He said he didn’t mind my doing it, but he said if I do this again, I should not identify myself with the U.S. Commission on Civil Rights. I thought initially I was going to get fired but it wasn’t anything like that. At times, push back is much rougher, as seen demonstrated in the response to the study about utility bills mentioned above. The study had concluded the utility should change its billing practices to enable people to understand how much energy they were using and how much they could save by changing some of their behavior. The author described: I got a call from the President’s office at the university saying that they had received an inquiry from one of our United States Senators who had been contacted by the utility to discuss the communists on the faculty and what was to be done about it. … The Senator called because he thought it was all hogwash and he was fine, so he wasn’t making the complaint, but he was saying he’s got this, these people who were out there, and that. This became a cause celebre in the papers. Push back can come from other academics, at times reflecting the tensions between qualitative and quantitative research methods. Rivalries between those in different disciplines can also create push back. A political scientist who had studied in multiple countries complained about such cross-disciplinary tensions. He described that cultural anthropologists rejected much of what he did indicating that: cultural anthropologists … were just horrified by the idea that something you learned in one country could explain something you learned in another.

Other Obstacles Faced in Collecting Data 103

Because their advantage was, their priesthood, was based on, we have been there and know the language and nobody else can know anything. The perspective of the cultural anthropologists was that each culture was unique. How could what was learned in one culture than be applied in a completely different setting and culture? Political scientists are more likely to seek out the commonalities in political processes rather than unique elements. Obstacles can occur because of marked differences in underlying perspectives of different academic disciplines. A social worker described a push back she felt when describing results from research done abroad versus work done in the states. There is also a question about the perception of your research back home. … If you go and look at a social security system in another country and ask questions … is it working?… If you were doing that research in the U.S. that would be considered research that is generalizable, I mean this is about social security. … If you go do that somewhere else, they consider it a case study … Academic squabbles based on different disciplinary perspectives do occur and can affect how your research will be received. Researchers are not always innocent when there is objection to their work as push back might occur because of faulty research. One of interviewees described that when she was a graduate student, one of her professors faced such a situation. He had distributed his … quantitative study of cities’ financial problems. The study ranked ordered a number of cities in terms of the extent of their problems … he widely circulated the report. A number of cities in the study were angry because he was causing an increase in their borrowing costs. They argued that the measures being used were not complete … they were in much better financial shape than the measures that he used. He hadn’t done the research in advance to find out what a good measure would have been … His study ranked some of the cities he looked at as better and some as worse, compared to each other, without asking how good the better ones were or how bad the worse ones were. The worst ones in his sample could still have been quite good, which is what the cities were arguing. He had caused them harm without knowing if they actually deserved it. HJR: What lesson for research did you take out of this situation? The lesson that I drew is that you very carefully pretest your measures … before you go ahead with a study, whether you want to measure them quantitatively or qualitatively. That is where you have to start because your study may have real world consequences.

104  Other Obstacles Faced in Collecting Data

Push back and censorship are highly visible and can be dangerous to the research­ ers especially in authoritarian regimes, where criticism of the leadership can result in arrests, imprisonment, and/or beatings. In such situations, researchers need not only look out for themselves, but also for their team members. This matter was of great concern to the American expert on China who worked with Chinese colleagues as part of a research team. He indicated that his rule of thumb was “First and foremost … the topics had to be something that would protect my Chinese colleagues.” Similarly, the sociologist who studied socialist workplaces had to be doubly cautious to protect the local academics who aided him in his work as the secret police were ever present.

Physical Danger When starting graduate school in the social sciences, the thought of physical danger never entered my mind. Maybe an interviewee would slam a door in my face, or a telephone respondent would hang up, or perhaps I might not pass qualifying exams or never have an article accepted, but physical dangers? I didn’t imagine any. I was wrong. Many in my small graduate school cohort ended up facing physical danger during their research. One was working in an area with a fun­ damentalist religious rebellion under way, another interviewed and observed in Vietnam at the height of the war, a third lived in one of the more dangerous shanty towns in South America. All were in physically dangerous areas. Me, too, as it turned out. One of the two areas I studied in Thailand had been on the path of the opium trade, and still had active criminal gangs. The second area was in the heart of a battle between mountain tribal people (whom the government had been cheat­ ing out of their land) and the government. Troop carriers passed by our house and a month after we left several government officials were assassinated in front of the house we had rented. The actual danger to my wife and myself was less than it appeared to be. As I mentioned before, I had interviewed the leader of the violent gang in the first locale, and having paid him respect, he made sure we were not threatened. In the second area, we were told by local farmers who supported the insurgents that we had been vetted by them, had been found clean, and were safe. We were lucky though the dangers were real. Sometimes, it is your very naivete that protects you. On one occasion, I had arrived early for an interview in one of Chicago’s poorest, virtually all African American, neighborhoods known for its active gangs. Out of respect to those I was going to interview, I wore a suit. As was my custom, before appearing at my interviewee’s office, I took an hour or so walk to explore the neighborhood seeing visible signs of the city’s neglect of the area. When I entered the office of the person I was going to interview, the receptionist asked me what I had been doing and I said, taking a walk to

Other Obstacles Faced in Collecting Data 105

look at housing and lack of street repair. She blanched, and then said, well, the locals must have thought a white guy wearing a suit walking around was either a cop or crazy. In another neighborhood, those whom I interviewed assured that the gang killings would not impact me, but they still insisted on escorting me to my car. By why put yourself in danger? Two reasons, one laudable, and one probably laughable. The first is social scientists do care about the problems people face and if doing research on them necessitates a certain amount of danger, so be it. The second reason was shared by the researcher who spent considerable time in Vietnam. He argued that the most dangerous research was often done early in our careers and “when you are young you think it is safe to do those things … It is other people who are going to die at that age, you think you are too smart for it to happen to you.” Research may seem exciting rather than dangerous. The anthropologist who studied the sea nomads indicated: So being young and looking for adventure it all sounded very exciting. And I think maybe it was good that I was naive because think if you think too hard and start to consider all the challenges, you might actually not have the courage to go into the field. So, you know being young, being naive it seemed fantastic to do something … to be able to break new ground. According to the criminologist I interviewed, prisons were fairly safe venues, although scary situations did arise. As an example, he described that after inter­ viewing inmates late at night: I walked out and there was a captain who was the cell block supervisor who was there and he came charging over to me (gruff voice) “what are doing in my cell house at this time? What are you, I’m going to write you up.” He was pissed and sudden (ly) here comes my friend Michael (a prisoner). Michael came charging down and he walked up to this guy and started poking his finger at the jailer’s chest and said, “this man is with me jailer and you show him respect, don’t mess with him.” … I was kind of say scared shitless oh God here we go we are both going to get our asses kicked, we both are going to go to jail. Well, we’re in jail, you know. And, it was bizarre, Herb, totally bizarre. This prisoner backed down a captain, a supervisor in front of other prisoners … I said, jeezus, shit, what the fuck did you do? we’re hosed and he started laughing. The jailer is taking too much money from us to say a goddamn word. The danger in this case was averted. Though mightily uncomfortable, it was helpful to the research as it gave insight into prison culture. Sometimes, you need to take risks to learn about a matter.

106  Other Obstacles Faced in Collecting Data

There may be lore, part of the local culture, that describes what happened to people who asked too many questions or that constituted too great a threat. An urbanist described: I did interviews with investors in foreclosed homes … in some of the highest crime neighborhoods in the city. …. One investor I wanted to interview, who had owned a bunch of properties in one neighborhood. … He was found murdered under a crawl space in one of his properties. So, it kind of hit home that I should be careful. The person who was looking at city financial affairs described: Well, I was afraid at one point, not terrified but certainly concerned … because there had been a journalist who had been poking around in the city a number of years earlier and she had been murdered and buried and her body had been found. … That was kind of considered to be warning that certain subjects you don’t approach. …there was organized crime that was operating in the city and that might effect some government contracts. …. So that was kind of in the back of my mind. It is not only you who are in danger, but also your team, and the people whom you are studying live with this danger every day. The danger you feel even briefly might make you more empathetic to those being studied as described by an urban researcher. I did a lot of this research in very poor ghetto neighborhoods and at times it was a little scary. When I was doing my thesis, I went at night to these meeting in public housing projects …. and going into these buildings where people were shooting up on the doorsteps and wonder if I’m going to come out of this building…besides whatever risk I ran personally, which was actually relatively rare, I was supervising research assistants who were going to these neighborhoods all the time. One of them told me he had gun pulled on him. Others were women who were inevitably hit upon. A South American expert described that in the neighborhood in which she lived, the houses all had armed guards and: especially in the north of the country … where there would be ambushes on the roads and if you were traveling on the roads if you were traveling you might be detained and so on by either side, but especially by the contra revolutionaries. And another South American expert described the precautions she did take: We were at risk to some extent when we visited (location in Sao Paolo) and other places because unfortunately these places are heavily gang governed and

Other Obstacles Faced in Collecting Data 107

the gangs are armed and they keep a close grasp on their territory. We had to ask permission to enter. You have a member of their gang accompany you and you have conversations monitored. They are armed, you can see the weapons and you know that they are violent. In that sense, there is an exposure to risk Short of actual violence, you may be threatened and asked to leave. I asked a political scientist who researched a controversial topic in a central American country whether he had confronted danger. He answered, Well I was chased out of town, does that count? Herb: Were you physically chased? No, this was threatened … I was told you need to leave or else.” Keep in mind that the questions you ask and what you are learning may have threat value and put you in danger. My classmate who did research in Vietnam responded to my question about being in danger. My life was threatened many times, especially the Koreans who knew I knew about some of their massacres. They put a bullet behind my head when I was in a jeep with two of my interviewers in back and one in front and the bullet went just behind my head to remind me. Most interviewees, now my age, admitted that they would no longer do the types of dangerous research that they had done when way younger.

Ethical Concerns as They Impact the Researcher Not all dangers are physical. Some are emotional or psychological. Being put in a bind where you have contradictory obligations can be stressful. Almost certainly in research methods courses, the instructor or the text, proba­ bly both, emphasized the importance of ethical research—not harming those being studied, respecting confidentiality, avoiding deception, or explaining it (as occurs in social psychological experiments that require brief deception). In universities, there are institutional review boards (IRB) that try to enforce ethical rules. IRBs are mandated by government funders to assure that research is conducted in ways that do not harm those being studied. IRBs were initially formed to monitor medical research where what is harmful is usually clear; you don’t inject a drug into an unwilling subject. Later, IRBs expanded their role to include the social sciences. IRBs tend to follow a few rules, often in a mechanical way. Those being studied must be informed of the study and agree to it. Thus, there should be no secret recording. If you promise anonymity to the people you are studying, you must comply, and demonstrate how you will protect the data from anyone else who might want to identify them. Generally, research with children or others who cannot grant informed consent faces numerous constraints from the IRBs.

108  Other Obstacles Faced in Collecting Data

However, in the social sciences, it is not always clear where the boundaries ought to be between ethical and unethical behavior. You may be working with a survey that is on an important topic, say, reluctance to get vaccinated. Some of your questions might make respondents uncomfortable. Is that harm sufficient to derail the project, despite the importance of the information you are collecting? Is it legitimate to mandate that students in a class be subjects of a research pro­ ject even if they didn’t explicitly agree to be part of it? There is a bit of coercion involved, if a grade in class is dependent on their participation. IRBs want to know what is going to be asked on a survey to assure it meets ethical standards, but what about questions in an open-ended depth interview in which topics covered emerge as the project progresses? Do you provide questions you are going to start with, and ignore the IRB as what you ask changes? Or, do you try to get approval every time that you shift focus to accommodate to examine topics raised by those whom you are studying? For academic projects, IRB approval is needed before data collection begins and waiting for that permission can significantly delay the research. If the IRB board has objections to what you are planning to do, time is needed either to convince the IRB that the objections are misplaced or to revamp what you are planning. For many researchers, IRB approval has become a standard step, sometimes seen as an obstacle, delaying or even preventing legitimate research. Every researcher has to jump through hoops to prevent the few who might abuse the process from hurting their research subjects. Sometimes, IRBs are disliked and are mocked. One respondent emoted: And, don’t get me started about what we learned about the IRB,. I had a friend at the University of (name) who was doing auto ethnography which is yourself as subject and he had to run it by the IRB human subjects protec­ tion and they made him go through the full protocol. He had to sign papers, “I understand that basically I will not hurt myself.” It was totally bizarre. Now, in fairness, he said the reason they made him do it was they were following the protocol. He knew it was stupid, they knew it was stupid, …. but they don’t want to put themselves at any risk given the craziness of the IRB … certifiers so they just made him jump through the hoops. In the midst of a project the design could change, not just the questions one asks. Data gathering could be delayed as further IRB approval would be needed. An individual who had been doing interview research with officials and business people, for which he had obtained IRB permission, but discovered once the project was well underway the need to survey the villagers who were impacted by what the business people were doing: the survey … was not something that I had intended to do. It was some­ thing that occurred to me to do once I was in the field. So, I had to go back and amend my IRB protocol to get permission to do a survey.

Other Obstacles Faced in Collecting Data 109

While acknowledging the legitimacy of a board enforcing ethical standards, researchers may still feel that doing research is made more difficult by such boards. Researchers are fully aware of the need to behave ethically but are frus­ trated as they find their work delayed by IRBs. As a social worker described, I often do research with a vulnerable population whether immigrants or people in poor communities or people displaced so it always very impor­ tant to maintain their anonymity and where they really are. I am facing that challenge at the moment because I haven’t yet got the IRB approval for my protocol. I will be interviewing immigrants and they (the IRB) like to make sure of this and that and this and that. Where we are storing the data and how I am going to secure it. I do appreciate the IRB but it does make research with a vulnerable population challenging. The effect of such controls may be both positive and negative. They assure that the research being done is ethical, and harmless to those being studied, but can discourage researchers from examining topics that would require a prolonged IRB review. In another project, this same social worker was seeking out women in an immigrant group who had given birth who also met two other criteria, their babies were under weight and the mothers had been in the country for a specified period of time. To get a large enough sample meeting the two criteria required access to eight different medical clinics and for each one, the IRB demanded filling out separately an entire IRB protocol. IRBs are not the only reminder of ethical obligations. Sometimes, the need to protect those being studied is made apparent by the people themselves. Several of those whom I interviewed reported that members of the organi­ zations they were studying wanted to cooperate but were afraid of the conse­ quences they would face in their organizations for doing so. These informants insisted on meeting the researchers far from their work sites. In a study com­ paring federal programs in several cities, one researcher remembered, “Corpus Christi, Texas was one of our cities. People would only talk with us if we met them someplace far from Corpus Christi in secret.” While in Washington, D.C., informants would only meet in restaurants at the opposite side of the city, far from their offices lest coworkers learn that they are helping a researcher examine a controversial matter. Respecting the concerns of those being studied is an ethical imperative but it can be frustrating for the researcher. One interviewee had promised anonym­ ity to those whom she studied, and accordingly wrote up the research without identifying the location that would have identified these individuals. The journal editor wrote back, demanding the location of the study. The researcher had a choice between publishing her work; she was a young scholar eager to make her mark or following through on her promise of anonymity. Ethics won, she did not publish the article, but it was stressful.

110  Other Obstacles Faced in Collecting Data

Even long after research has been completed, researchers do wonder if they had treated ethical dilemmas appropriately. Half-a-century later, I still reflect on a situation that I faced. The Thai bureaucrats I was studying differed greatly in their sympathy for the rural villagers. One of my informants, who became a friend, was a supporter of the villagers and an advocate of democracy in what was still a military dictatorship. I fear my obvious support of his beliefs and actions inadvertently encouraged him to be more aggressive in encouraging villager autonomy and by doing so getting himself into trouble with his superiors. I never thought of that at that time. Decades later, the anthropologist who studied sea nomads still worried about whether she had made a correct ethical decision: I also encountered difficulties in that some of (the nomads) would ask for a lot of things and I wouldn’t know whether they were trying to rip me off or whether they were really in need. And it is if you give one person then when the rest of the community finds out you are obliged to give everyone as well. It is really difficult to know and you feel bad not giving too because they are giving you so much but exactly is how much can you afford to sort of give. … One episode that has bothered me was in a family of sea nomads. The father took to drink;. these resettlements sometimes caused a lot of prob­ lems for them so some of them turned to alcohol and this what happened with the father of one of the groups of sea nomads and he sort of turned to alcohol … one time when he started to ask me could I buy the sugar, but he started to ask a lot and I was beginning to wonder whether to give it to him or not to give it to him then he won’t need to go fishing he won’t see a need to go fishing and then he will turn to drink. So, I decided to not (emphasis) after some time decided not to give him anything. And then after some time his son passed away, it was a young baby boy…what I didn’t know when he was asking he was because his baby boy was very, very ill and was dying and that’s why he couldn’t do his fishing expeditions. So … can you imagine how bad I felt and how bad I still feel. The criminologist, in doing a long-term prison study, described a situation that had conflicting ethical imperatives: I had a lot of ethical issues … This fellow I came very close to in prison. And he confided one day that another prisoner who I knew and got along very well with; there was a hit out on him and he told me what for, which gang was going to do it. And what should I do? I certainly wasn’t going to the authorities and he told me this in trust and I knew if it got out that would demolish the trust not just with him but word would get out and there would go my trust but I did go over and talked to the fellow that the hit was on and started asking him how

Other Obstacles Faced in Collecting Data 111

he is doing and what was going on and kind of led up to the subject and hey did you ever do anything where somebody is going to come after you. And, he said, yah, right now. There is a hit out on me. So he knew. And, this case, it was all moot because he had cancer and he died about a month later. Keeping the trust you have built up, keeping the research going, may entail doing some things that seem to you if not exactly unethical, at least problematic. Qualitative interviewers generally encourage and sometimes offer support to their interviewees. If the interviewees are doing something that you disapprove of, offering that kind of support can be stressful. How honest do you have to be? Do you need to tell interviewees your opinions? The person studying mining in South America worried about this issue years later. He was never sure he had behaved ethically. I found myself not lying, I never ever lied. But if I was trying to get access to somebody from the mining company and if I was trying to convince in the course of an interview somebody from the mining company that they should trust me, I would describe my research with an emphasis on. how … I was critiquing the solidarity organizations on the left. So, I would really drive home the fact I feel these organizations, these international organizations, are manipulating people and telling them what to think and it isn’t fair and there are some real development benefits to mining even though there are these social and environmental costs. And so, I would lay on thickly that side of my thinking and … sort of leave off that I had some real doubts about the utility of mining as a rural development strategy in that country. And so that did present me with some moral questions. I came away from that feeling gross and wondering where that kind of approach to interviewing, where that falls in terms of the ethics of the field. And that is a question that I really don’t still have a clear answer to. To put it more simply, where does one draw the line when seeking rapport and ingratiating oneself in a way that goes beyond what is ethical. It is difficult to figure out where to draw that line when one is in the middle of an interview. If you are interviewing or studying people whose behavior or ideas are fundamentally opposed to yours, think in advance about how far you are willing to go in expressing agreement or support. Anticipating where ethical dilemmas are likely to arise can be helpful, because it gives you time to work out strategies in advance and avoid the years of wondering; did I do that in an ethical manner? Could I, should I, have done it differently? Sometimes, there is a solution, if an imperfect one, to balancing competing goals. In my study of the zoning dispute, people were incredibly forthcoming, allowing me to accurately present a micro portrait of back-room dealings in local

112  Other Obstacles Faced in Collecting Data

politics along with unpleasant personal conflicts and other such sexy matters. I kept the location and all names confidential and used none in the article thinking that would protect all concerned. The article was readily accepted however the reviewers said that the case I had studied was one of a kind and well known in the planning field. My attempt to protect people by not identifying the community didn’t work. With the editor’s permission, I cut from my manuscript the material that I felt could create conflicts between the individuals studied. The redacted article was less informative, but it would have been unethical to publish some­ thing that could make those whom I studied angry with each other. It may be more common ethical problem that readers will recognize famous people, perhaps public officials, whom you have studied even though you have not named them. What do you do if you have promised anonymity? Does that promise mean you cannot use the material? Do you have to find others who could have provided the same information, so it is not so obvious with whom you have studied? I asked a political scientist who studied readily identifiable elites how he pro­ tected their identity. He answered that he often opted to not ask questions whose answers could only be known by only one person. Other times, he asked his questions to provide himself with a broader context as he thought about what he was studying but did not include the responses in his research report. Sometimes I don’t use the information if I see that it can put that person in danger. And, in addition to that, this is somebody you come back for more information at a later time. …There are facts you get that illumi­ nate what you are studying but that you don’t necessarily ask of people because if you include those facts, then people can say I know only two people who know this, this person and that person, and since this person is dead, it is that person who told this guy. You want to avoid that as much as possible. It may be frustrating to get information that helps explain the puzzle you are working on, but not be able to use it. But as this interviewee points out, keeping the those being studied safe also safeguards your future access. One interviewee suggested her way around this kind of problem. She typed up the interviews, and sent them back to her interviewees, asking them to mark off anything that would not be ok to quote and attribute to them. That way, she didn’t lose the whole interview, while learning specifically what topics were considered sensitive. When an individual explicitly tells you that something they are saying is off the record, it is an ethical imperative to maintain that privacy. But sometimes, your interviewee doesn’t say, don’t repeat this, or this is not for publication. This situation might occur when you are chatting informally with those whom you are studying.

Other Obstacles Faced in Collecting Data 113

I had gotten bored at a conference of an organization I was studying and took a walk. During the walk, I bumped into the head of that organization, and we walked together chatting. He shared with me an exchange he had with a senior legislator who usually supported the organization’s goals, but, in this case, was not doing so. He described to me the very unpleasant encounter which included a shouting match. What I heard in this informal conversation was germane to my research about how advocacy organizations interacted with legislators. But he hadn’t granted me permission to use this material. Did he intend for me to use it? Or, did he assume I wouldn’t because it might hurt the organization? Or, because we were just chatting during a walk? I didn’t know. I faced an ethical dilemma, especially since all my prior discussions with him were in his office with digital recorders going with full his full permission. Fortunately, the ethical problem was avoided as later the head of the organiza­ tion shared an abbreviated version of the story at a plenary session of the confer­ ence. So, I could use the story, at least in abbreviated form as the matter was now widely known. In this case the issue resolved itself, but informal conversations raise this question of whether it is ethical to use the material.

Ethical Problems with Applied Research Applied researchers have additional ethical problems. Problems occur if the researcher is pressured to come up with specified results, and, in doing so, dis­ torting what was found or worse yet becoming a pawn of those in powerful political or organizational positions. A personal story illustrates this trap. I was involved but only learned what was happening years later. I was young, junior in my career, though somehow, I had gained a reputation of being very concerned about ethics; probably by parroting back what I had taught in my research methods classes. I was working half time at a university research shop. One day a state politician in a senior position in his party arrived at the research shop and for reasons I did not figure out at the time, pulled me aside and grilled me for a good 15 minutes (he was a skillful interviewer). I was totally puzzled. Years later the director of the research institute explained what had occurred. The politician wanted the university research shop to do surveys that would slant results by a strategic wording of questions in ways that would favor the politician’s political party. Not wanting to offend the politician, as the university was state funded and the politician had clout, the director did not want to turn him down himself, but instead said Rubin might blow the whole deal so you should talk with him. He did, and I probably sounded like a rigid person on ethics and the deal never came to fruition, fortunately. The moral of the story is anticipate where ethical problems may occur, think out in advance what you will or won’t do, and let others know where your lines are drawn. And hopefully they will be on the side of ethics not expediency in doing research.

114  Other Obstacles Faced in Collecting Data

Conclusion A variety of problems can create obstacles in gathering research data; problems that researchers might not expect. Unforeseen external events can affect or derail data gathering. Coups occur, diseases make some sites dangerous, public opinion changes midstream while data gathering. Push back can occur from those concerned about what you might find. Third parties might not want your research results to see the light of day. University administrators might pressure academics to avoid topics that might offend funders. Other academics can attack your work because it doesn’t follow the same models that they take for granted. If you finish a research methods course and are not aware of ethical issues, you probably were asleep in class. But, what is usually covered in such courses are dramatic cases rather than the less exciting ones that occur more frequently but also are less clear. If you haven’t thought in advance where ethical problems might arise and figured out how to avoid them or deal with them, you might be unprepared when they happen and later feel unsure if you responded in an ethical way.

8 TRICKS OF THE TRADE

In the previous seven chapters, I’ve used examples provided by experienced social science researchers to extend, and sometimes modify, what is generally taught in social science methods courses. This final chapter brings together some of the tricks of the trade, that is, improvisations and data gathering gambits, that experienced researchers have developed over time. I’ve grouped these tricks of the trade into four categories presented below.

EXAMPLES OF TRICKS OF THE TRADE Facilitating survey research Enhancing in-depth interviewing Reaching out and working with contacts Anticipating the audience for your report

Facilitating Survey Research Those whom I interviewed for the most part followed standard textbook procedures in designing surveys; still from their experiences they worked out tricks of the trade for improving result from the surveys. For example, they emphasized the need to pay close attention to the length of the survey as described by a sociologist: You have the usual problems. People don’t want to participate, so there are refusals, and there are incomplete questionnaires. … you are always going to run into that, you know. And, of course, you try to come up with ways DOI: 10.4324/9781003345909-8

116  Tricks of the Trade

to minimize that, for example, keeping things not overly long in the survey. You may want to ask 100 questions, but you haven’t got time for that. You really have to hone it down and think about what are the most essential things to put into that survey, so it is going to take up twenty minutes, half an hour of somebody’s time. That can be challenging. The trick of the trade to increase participation rates is a willingness to focus on what is core to a survey, even if that means eliminating other interesting items. The interviewees shared other tricks to improve responses to surveys. One recommended to engage people quickly “make sure the first question is open ended so they get their story out, otherwise you are going to lose the interview as they fight to put their story in.” Another tactic was to assure that those who volunteer for surveys are getting something back for participating, perhaps just the knowledge that what they are saying could impact social policies or influence the thinking of public officials. Others recommended providing material benefits for participating. Especially in lower-income countries, researchers have worked out ways of providing benefits to those participating but dong so in ways that respected the sensitivities in the local culture. In a prior chapter, I described how one researcher attracted people to the survey site by providing them with a free meal, one prepared by a local self-employed person. Another researcher working in poorer areas of Indonesia found indirect ways of rewarding villagers through leaving behind supplies villagers would otherwise need to buy but doing so in a way that did not embarrass those he studied, again showing cultural sensitivity: I learned that what I needed was extra candles and a large number of cigarettes. The candles because you could only interview at night …. you can leave behind the unburned candle and that was something of worth to people in that social structure. You could also leave behind the unsmoked package of cigarettes. Researchers learn from experience to avoid jargon in survey wording, whether academic, legal, or medical. Academic researchers are likely to curious about how people respond to the implications of the theories in their academic fields. That’s fine. But as the interviewees indicated wording questions using academic jargon simply doesn’t work. You don’t ask interviewees whether they conform to group norms but instead determine what those norms might be among those being studied and then ask how strongly they agree or disagree with the importance of an illustration of the norm. Other tricks of the trade in doing surveys mainly involved setting up random samples when the conditions for doing so were far from optimal. This matter was covered in a previous chapter.

Tricks of the Trade 117

Enhancing In-Depth Interview Studies In the latter part of my career, my major data gathering tool was through depth interviewing. One of the simplest and effective tricks of the trade that I learned early on was to interview people more than once whenever possible. The first interview sets the stage, enables the interviewees to learn of my interests from the very questions I asked, and hopefully experience the interview as nonthreatening and fun. Almost always, the second interview provides deeper, more insightful responses. The interviewees are more aware of what the research is about and between interviews often thought of richer responses they wished that had initially given and now have an opportunity to do so. They take the time to clarify and elaborate points raised in the earlier interview, often doing so before I have posed any follow-up questions. Doing multiple interviews makes it easier in later discussions to deal with sensitive topics, as for example people of color whom I had interviewed before, during the second interview are far more willing to provide examples of racism as well as intergroup tensions. For me, the example that makes the value of the second interview crystal clear took place in the repeated interviews with a director of a community development corporation (CDC) about which I had learned from others that the city was investigating for potential corruption; I was curious. In the first interview by intent, I only mentioned that matter in passing not wanting to sound critical or threatening. Instead, during that interview, I focused only on the trials and tribulations in doing the development projects. On my return months later, I remained curious about the ongoing investigation, planned to ask about it but had no idea how to raise the subject. I need not have worried. My interviewee suspected that is why I had come back driving several hundred miles to do so. Without my asking, he raised the subject and provided his perspective that racism on part of the city was involved. His response had credibility as two other CDC directors in the same city, neither of whom were being investigated, had also indicated how racism on the part of some city officials affected the projects they were doing. This incident reinforces another trick of the trade. When possible, visit those whom you are studying rather than interviewing by phone, especially when those being studied are aware that you have traveled a distance just to talk with them. This courtesy will often elicit responses that simply do not occur on phone calls. On several occasions, in response to my willingness to travel my interviewee not only were quite forthcoming but, in addition, prepared fun “show and tell” tours relevant to my research. Other researchers who relied on in-depth interviewing described tricks of the trade they used when they wanted to raise questions for which the interviewee might be reluctant to answer or answer less honestly. The scholar whose work was in Vietnam (during the war) explained his approach: You don’t raise it exactly … you don’t act like you’re an outsider saying using a moralistic tone. … You sit down with somebody in the war zone and say,

118  Tricks of the Trade

who is doing the shakedown that is causing you trouble? That means you know that is happening. … You get a lot out of people if you know how to ask the question to make it sound natural, like of course this going on. Like in Vietnam, I would say how do you handle families whose son wants to come and visit when the son is of the Viet Cong? The village chief would explain, I would tell them if your son comes home, tell me in advance and promise he won’t make any trouble I won’t make any trouble either. After the end of the Vietnam war, on his return to Vietnam, he continued this approach to depth interviewing, showing interviewees that he was aware of what was going on and then asking for more details to flesh out his understanding: When I got to go back after the war and interviewed Viet Cong, the way they knew I was there for business and not to give bull shit is when I asked my first question, so how many man months does it take in your soil to fix a single B 52 crater? … And, how often did you have to change your hiding places for your weapons? Or how many people had to give you signals in the windows so you knew whether it was safe that night to come in the village? His approach of being indirect and showing prior knowledge while asking questions reminded me of one of my own tricks of the trade. I was trying to piece together a complicated story about why an organization took certain actions. I had already interviewed several people whose responses were superficial, mere clues that something was afoot. Now the trick of the trade. When interviewing others, I would drop hints of what prior interviews had said, doing so without violating confidentiality. My hints showed that I had knowledge of the matter freeing up the subsequent interviewees to provide full details on the issue. A closely related trick of the trade is useful when you are reasonably certain about something but need confirmation. In this case, I would formulate a question based on my knowledge of the matter but intentionally make a mistake in my description. The conversational partners invariably corrected my mistake and then would provide me with their corrected and, often, more nuanced version of the event. Experience teaches researchers how to avoid rejections of the request for an extended interview. A scholar who was studying budgetary processes at variety of federal agencies, needed the perspectives of top-level officials on the matter but feared they would ignore her request. She described her trick of the trade: I would often at the federal agencies start with the unions because the unions were more open, and they wanted to get their story across. So, for me I was their access to getting their story out. Then I was able to approach the upper levels saying that I have the union story. I want to get your side of it, do you want me to go out with just the union version? They did not

Tricks of the Trade 119

want me to go off knowing only the union version of the story. That is how I got both sides of the stories. The broader principle is that the order in which one does the interviewing matters, starting from those most willing to get their story out, and then go get the counter version of it. Preparation is vital before conducting depth interviews. When possible, I go to meetings also attended by those I want to study, both to learn their specialized vocabulary and become a familiar face. Or, when that is not possible I would find someone in the same field (not part of my study) and discuss ideas about which questions I should ask as well as get their take on whom they felt would be willing to talk with me. In addition, when documents are available, I read them prior to doing an interview. The social action organizations I studied each produced numerous written reports, some quite technical, on the areas of concern to the organization. Before interviewing, I made sure I had read (or at least skimmed) these reports to get a slant on the issues that the organizations were concerned about. I would begin the by asking questions about the reports grounding the conversation in a specific matter with which the interviewees were familiar and felt confident discussing. Talking about the reports also showed I cared enough to do background work while assuring that I would not get superficial answers that might be given to someone utterly unfamiliar with their work. Doing so set the tone for the entire interview of being a conversation between two informed individuals. A researcher studying how municipal budgets are impacted by the politics and bureaucratic processes with a city described the background work she needed to do for each case to become an effective interviewer. I had to inform myself as best as possible before any of these interviews. I would find archives and read old budgets; I would go to the library and read old newspaper stories. I wanted to learn what topics were sensitive, which ones the newspapers were following, and avoid those topics so I would not sound like a reporter looking for dirt, looking for a story. When I approached the budget director of one city to ask for an interview, I said the reason I’ve chosen you is because I know you have a very good budget and I want to know where it came from, what was the idea, whose idea was it, what difficulties did you encounter in producing this and how did you resolve them. When someone outside understands and appreciates your work, your problems, and your accomplishments, that creates a motivation to talk to you. The interview creates a kind of public recognition. Depth interviews often can be recorded though you must get permission to do so. Advice from my own experience, make sure the recorders are fully charged and remember to turn them. Because accidents happen (such as hitting the pause button instead of record), I learned to always use two recorders, assuming I would not screw up on both.

120  Tricks of the Trade

But what if recording, or even note taking, is not allowed? A person whose research dealt with sensitive topics explained her trick of the trade for carrying out conversations without taking notes: You have to develop your memory a lot (as) what I was looking for was considered sensitive … we met sometimes in remote locations. I would have to see someone in a restaurant some distance from city hall lest somebody else show up there and overhear us. That sometimes meant you were eating French fries … with your fingers and couldn’t take notes at the same time. But often times I had to promise … I wasn’t going to take notes so they couldn’t be quoted. … because if they were quoted it would create enemies, … So, I learned not to take notes, not to even have a piece of paper or a pen. … At my peak, I could reproduce two maybe three hours of interview material after the interview. HJR: How did you do that? Well, I had a list of my questions I knew what my questions were. So, what I  would do is immediately on leaving, sometimes I go to the rest room, sometimes I’d just leave the building. … then I would sit down and take my notes. I took alphabetic shorthand so that I could do this quickly and I’d write down the topics of the answers. I went back to my apartment and then I would fill out the rest of the answers as best I could from my raw notes. Then I would type them up. Then I would give them back to the interviewee and said, this is my understanding of what you said. Now I may not have understood everything you were saying, and you may have said some things that you wished you hadn’t or you could improve upon. Make any changes you wish. You can delete, you can add, make whatever changes you want. This, to me, was one of my best tactics, because what it did, among other things, was to show me the differences between what I heard and what the person wanted me to understand. So, things that I didn’t understand were explained but also the second layer of meaning was, what did the person eliminate? And that gave me an additional level of clues as to the meaning of what I was reading. …. And it also gave them a sense of protection. They could talk to me as openly as they wanted since they knew that I wasn’t going to run around and spew something they had said accidentally. They would have a chance to review and edit it. And, virtually all of the upper level administrators that I did this with took advantage of the opportunity and gave me back commented notes.

Reaching Out and Working with Contacts In Chapter 6, I described ways of finding sites and interviewees and mentioned the importance of using contacts. I’ll continue the discussion here with more detail, illustrating the various tricks of the trade in using personal contacts, to gain access to sites and encourage cooperation. The general sentiment on the matter was expressed by a social worker emphasizing that:

Tricks of the Trade 121

People know people and that is probably one of the things you should emphasize in your book. You know, whatever kind of research. … you just can’t go in like a cowboy. Contacts facilitate access to research sites that researchers on their own have difficulty might not be able to get permission to study. When he was a graduate student, a sociologist wanted to study worker behavior on the factory floor but had no luck in gaining access to industrial firms. I wanted to work in a factory, and I’d never worked in a factory anywhere. … and I tried and I tried and there was no chance, nobody would take me on board … so eventually I talked to a relative of mine who lived in South Chicago and he said, okay, my friend, he was a manager engineer in Allis Chalmers, in a diesel engine plant … So, he said why don’t you talk to our personnel manager. He will give you some ideas of how to get into a factory. So, I knocked on his door with a recommendation from my relative and he sat me down … Here I am, give me a job. And he looked at me, he wasn’t actually expecting me to actually ask for a job. And, he said, sure, sure I’ll give you a job. So that’s how I got my job at Allis Chalmers. He said, yeah, I could use somebody that sweeps up. So, I got the job That was a start of what led to a chain of research projects on factory workers undertaken in several countries. An unusual example of the role of contacts is seen in a study done by a sociologist who specialized in Central America. While in the Peace Corp, this young scholar met and got engaged to an educated woman in Guatemala. He returned to the states and, while doing his doctoral studies, befriended overseas students from Guatemala who in turn made him aware of the controversial mining industry in that country. He decided to study the mining industry and wondered how he could gain access to the professionals in the mining companies, people who are often reluctant to talk to researchers. He explained how his (beloved) contact helped him. My wife … got a master’s degree in environmental studies … in Guatemala city and … some of the people that she studied with, that she became friends with, were geologists … and these geologists went on … to found an environmental consulting company … they got to know folks from some of these (mining) companies and they were able to, because my wife’s colleagues had degrees in geology and could sort of talk that talk, there was a certain amount of rapport between that company and the mining company and so when it became time for me to try to get access I worked through my wife’s professional network. She set me up with her boss who was a geologist and then her boss … was able to convey that this wasn’t a gotcha piece of investigative

122  Tricks of the Trade

journalism but was scientific, … But it was a fluke of my relationships, it was a fluke of my social networks and of the particular historical moment that I was down there. I suspect similar tactics would work with friends without necessitating a marriage, though that case does show the romance of doing research. (Could not resist) Sometimes, you are seeking out a contact that in turn has the needed contacts in your research site. Though finding the initial contact can involve some effort (and in my case a lot of fun and cultural learning). In Thailand, I wanted to interview a Thai politician who had done doctoral research in the area I was studying, hoping he would both provide me with some background on the area and links to others in the area whose knowledge would be useful for my project. He now lived in a city an hour or so by bus from the area I was studying and in which I now lived. Well, there were no phones in that village, few in the city and I had no address for him. I took the bus to that city, and assuming the person was well known— he was a local politician—I asked individuals near the bus station, where he lived. Several pointed to the same area and I moseyed over there and again asked those I met in the street. That worked. I found is house; his mother was at home, but he wasn’t. I explained who I was, why I wanted to talk to him, and where I lived and returned to my village. A week or two later, my next-door neighbor told me the post mistress had told her that the bus driver had told her that the political figure wanted to see me at such and such a time at his house. That was enough for me. I traveled again and he was there awaiting to talk with me. He provided me with background on the area and mentioned names of those I should interview. Admittedly, that was an unusual way of making an appointment and connecting to a network; it certainly makes me appreciate cell phones nowadays but also the value of following a chain of contacts in locating those worth interviewing. Another person I interviewed for this book described how contacts help “to earn the trust (and gain access) to people who have good reasons not to always trust academics” especially in neighborhoods where people have had trying relations with outsiders. A white (color is relevant) researcher who wanted to work in a minority community described that I would get to know a particular individual or two who already had the trust of the community. So, they would be sort of my sponsors. So when I got involved in [an insurance case] it was the attorneys who contacted me and they had a … long history of filing lawsuits against the police and school board in discrimination. They already had a lot of credibility in the Black community. So, I think we researchers were successful because of them. Professional networks can also be useful, as a leading urban scholar described:

Tricks of the Trade 123

What has been very useful to me has been being part of the research committee … of an international sociological association … through it I’ve become friends with people around the world and that has enabled me to make contacts whenever I want to in cities, to do research, to stay with them and assist me and finding and getting interviews. At times, it is possible to request those whom you have met, better yet befriended, in a previous study to provide introductions. I had befriended the head of a D.C. trade association when I was study CDCs. My next study, as I’ve mentioned before, was to examine the national organizations that helped CDCs. Usually, I would set up a first interview to gain trust and then save the core questions for a second interview. But in this case, I was aware of how busy the heads of these national organizations were and feared I would probably not get a chance for a second interview. I needed a way to gain credibility prior to the first (and probably only) interview. I remembered that my friend who had been head of the CDC trade association knew all these individuals so I sked him to vouch for me. He made a few calls for me, opening the doors. Sometimes, luck plays a role in finding contacts. Some of those interviewed for this book described how past encounters and chance opportunities helped in gaining access to research sites. An urban scholar had worked in multiple countries, described her good fortune in making contacts: a lot of it is happenstance and a lot of it is my network with former students  … I’ve had former students in Australia and South Africa, in Vietnam, in India. You know, in lots of different places and that has enabled me … they will just hook me in. She continued, describing how an encounter with an English academic in the U.S. facilitated her later work in England. I knew a political scientist who had been in the United States for some time but had gone back to England and I was doing work in London, … I called him up and he introduced me to a guy named Roger … who became a principal in an urban consulting firm and then became the acting CEO of one of the boroughs … through him I was able to interview everybody including one of the Rothschilds. Just one thing would lead to another. Networks help in doing research and are well worth cultivating. Another researcher who studied federal agencies explained that you might have to create a network of your own to get the access needed. Networking is really important, getting the people through other people is important. If you can make friends with the person who is the administrative

124  Tricks of the Trade

assistant or secretary, you are halfway there. That kind of thing … If I didn’t know that from the beginning, I learned that quite quickly. Sometimes, the networks that help you are those between the people you want to interview, sometimes without your being aware. On several occasions, I was pleasantly surprised when the first interview with an individual was far more open, informative, and frank than I had reason to hope for. It turned out that I was interviewing people in the same field who knew one another and shared the fun of being interviewed with each other prior to my approaching these other individuals. I earned that they were looking forward to being interviewed because of what their colleagues had said about the experience. When studying people in one organization or in a closely linked field, there is a good chance that people whom you initially interview are connected. They will often share with others in their field or organization what the interview was about. That makes it imperative that you really be prepared for the first interview, not simply having good questions in mind, but being aware you are making an impression that subsequent interviewees will have heard about. Researchers with a series of publications, in the area in which they are continuing to research, might be known to those in a later study through their writings. I was interviewing a person for the first time and was pleased but pleasantly surprised at her immediate openness about the problems her organization faced. The discussion seemed more like old friends talking rather than a researcher and an interviewee. It turned out that prior to her taking her current position, she had taught a course on community organizing, using a book that I had written. I was known through what I had written. Probably more germane, the budget expert had written a well-received, introductory text on public budgeting. In some of her interviews in Washington, she noticed her book on the shelf of the interviewee. She quickly recognized that she was a known quantity to that person and that her views on budgeting were probably known to the interviewee. Her published book certainly facilitated getting the interview but could have slanted the responses as those she interviewed knew from the book the researcher had written the researcher’s broader take on the subject matter. Making contacts with higher-level federal bureaucrats takes some doing. How this was done was explained to me by a researcher whose interviewees included top staff in various bureaucratic agencies, including assistant secretaries (high-level political appointees). She explained the importance of using existing networks she had but then added her trick on building new networks that would facilitate gaining access with senior people. … Part of it is going through networks, finding somebody who knows somebody. So, I did a lot of networking to get into the bureaucracy to get into the agencies, sometimes attending a conference or a smaller meeting where I would look for somebody who had the right badge name for the

Tricks of the Trade 125

right agency and pull him aside for a little bit. I would introduce myself, talk about the research, and ask them for their advice on who they thought that I should best talk to. I would say to the person that I was looking for that so and so had recommended that I speak to you, told me that you were knowledgeable, and you might be willing to talk to me. So that was kind of my main strategy. But I used every connection, every connection that I could think of. If I knew a friend of mine had worked in an agency. I used that contact. I used professional associations. I used a lot of so-called informants, people who I knew who worked in an agency at one point who are now academics. I would ask them for advice, what should I avoid, what should I talk about, who is knowledgeable about this, who might be willing to talk, that sort of thing. The advantage of using a variety of networks is that it prevents you from relying solely on friends who might share the same experiences and expectations that you have or relying on groups of people who share the same interests, opinions, and prejudices as each other. For example, if you rely on union contacts, you might get only worker perspectives and then be put in contact only with executives who are sympathetic to the unions and miss those who are opposed. Bottom line is that we all recognize the importance of using networks and contacts in daily life and the trick of the trade is that this also holds in gaining access for doing in-depth interviews.

Anticipating the Audience for Your Report The academics whom I interviewed indicated the importance when designing a project to anticipate to whom the report will be presented. For academic studies, it makes sense to have an outlet, a journal, or a book publisher, in mind early on and then make sure your research deals with topics (and uses methodologies) relevant to those journals or publishers. Several of the interviewees, and myself, were unaware of this early in our careers and, in our innocence, sent off pieces based on qualitative methodologies to journals that preferred quantitative studies. That didn’t work. And, on one occasion, I failed to recognize that a journal I thought was appropriate was edited by those with strong ideological views and my article ran contrary to those views. Live and learn. When in doubt, it really doesn’t hurt to consult with editors on what types of articles they are seeking. For the most part, editors will respond, giving guidance, especially to those who are obviously early in their careers. When thinking of an audience in writing, it helps to imagine that those you are studying could be the audience and especially in applied research often are the audience. As one interviewee advised, In designing my research and thinking about presenting it, I always keep in mind that one potential audience would be the people I am studying.

126  Tricks of the Trade

In several recent projects, those whom I studied actually did read some of my reports. I circulated the reports to many whom I studied. The question I keep in mind is, would the people whom I studied recognize the world I am describing, and better yet, agree with my conclusions? I once circulated articles I wrote to those I studied, and when I later met them late at a professional meeting, they assured me I got them right. Not only did this validate my research findings, it maintained access to these same people or future studies if I continued to research similar topics. And not so incidentally made me feel good about what I had written. Reports prepared for the applied world can be quite different than those prepared for academe. A person who started out as an activist, doing research for a social change organization, and then moved on to a university and undertook academic studies reflected on this matter. He described the importance when writing for those in applied world of: being able to write for a very broad audience, being able to communicate numbers and sometimes quite complex numbers …. writing about it in a way that broadens its impact instead of writing for a quite narrow academic audience … it is very hard to write for public organizations … It is much more challenging than writing a journal article, in terms of the writing. An academic who later in her career ran an applied research center shared her trick of the trade in doing work for a client in contrast with writing for an academic audience. You really have to start by understanding …. when you are consulting with community groups, or public officials you have to learn what they know and how they know it and what it means to them and how they interpret their world. …You have to understand their unique experience in order for them to draw from their experience, for the matters [you report] to make sense to them. The trick of the trade here is to recognize that in designing a project when working for a client, you need to do a mini study of the potential audience for your report. A graduate school classmate of mine whose career alternated between academic work and senior positions with Uncle Sam described important differences in how information from research is communicated in government and in universities, something that he had to quickly learn when turning in his academic hat for a government suit and tie. Well, in government … you have to come up with a solution for the Cambodia civil war and you’ve got four minutes to present it to the President,

Tricks of the Trade 127

that’s different from having an hour and thirty minutes to get some concepts across to ten graduate students. It is the polar opposite, you have to be (a) logical (b) easy to understand (c) you got to have a real point (d) its got to be practical. They don’t care what Max Weber said in Politics as a Vocation. So, it is a different world in that respect, and you have to learn to adapt to it. The trick of the trade here is to know your audience and focus on the design so that the results of your research are relevant to that audience. A planner/economist who wrote prolifically in academic journals and was also an active consultant explained the different modes of communicating results to distinct audiences, and in doing so indicating the variety of data that the research should obtain to appeal to the different audiences: You have to be able to make an argument in three very different ways. The first thing … is where your academic skills help and that is, when framing an article for [academic] research, it is methodologically sound and is written for the five to twenty people in your subfield. …. that is, it is technically written … Then, the second is when I am doing public presentations, particularly to a business group or policy group, then what you are doing, you are telling that story with a beginning, middle and end …. So if I am doing macro economy, regional macro economics for manufacturing, it is a thirty page power point which is a whole bunch of numbers but they essentially they are wall paper to a story but it gives you credibility …. And, the third thing you have to do if you are talking to a political audience or a group of entrepreneurs, you talk in terms of anecdotes, because people who learn inductively aren’t going to follow it. So the way to think about it is, you’ve got an analytical audience …, an audience of deductive learners, and an audience of inductive learners. And, you have to be able to tell the story all three ways. The need to make these different types of presentation requires an initial design to obtain the data that speak to the distinct audience. Another researcher, also with extensive academic and consultancy experience, contrasted presenting a report to an agency with presenting to an academic audience. With a [academic] conference paper, a lot of people in the audience don’t have any idea about the agency or the research problem, you have to focus more on the problem, why it is important and explain the agency, the circumstances, why it is under threat or whatever. When you are reporting to the agency itself everybody knows so you have the most critical potential audience you could conceivably have. You have to be very careful not to go beyond your data and to make sure to limit yourself to exactly what you found and who in the agency told you. So, this is new information

128  Tricks of the Trade

for them, and it wasn’t something they could question because you are an outsider. It is a much tougher audience, much tougher. I once faced an awkward situation in communicating research results to practitioners while working at HUD (Department of Housing and Urban Development). My boss assigned me to explain a technical research report prepared by consultants on ongoing programs to the practitioners who day in, day out worked on that program. Translating the findings into practical terms that practitioners could use was new to me and was made more difficult because I was not the author of the original report and had no part in its design. I spent preparatory time informing myself of exactly what the practitioners did and structured my presentation to show point by point how the findings from the technical research directly spoke to their daily work. That is, I studied as well as I could the audience for my presentation. In my talk, I skipped much of the material in the report suitable for an academic audience rather than for those who did the work.

Conclusion This chapter brings together four topics—tricks of the trade in doing surveys, conducting depth interviews, making contacts, and preparing presentations. Some of these lessons are easily copied whenever they are relevant to a project, such as how to forge networks to reach out to the upper ranks of an organization, trying to schedule multiple interviews with each informant, or in the ways of broaching sensitive topics. More important though is that as your research skills grow, you not only can rely on tricks of the trade that others have learned, but also develop some of your own. You will gain confidence that while what is taught in research methods courses provides a starting point, in practice research requires matters to be tweaked, jiggled, and adapted to the reality in front of you. You develop confidence that you can improvise in ways that maintain the integrity and soundness of your research. Over the years you will work out a set of your own tricks of the trade.

INDEX

access gaining 32, 46–47, 63–84, 90; background work 86; denied 85; hanging around 61, 85; known quantity 86, 87–88, 124; through contacts 21, 85, 90, 123, 124; letters 88, 89; persistence asking 85–86; showing project value 86; showing respect 86–87; with vulnerable groups 122 archive research 69; 71

economic development project 55–56 econometric model 33

censorship 101, 102, 104 China project 51, 53, 56, 69, 74, 104 community development project (CDC) 51–52, 61, 85, 117, 123 contacts 120–121: aiding access 59, 89, 121–122; from professional networks 122–123, 124 contract research 30, 67, 125 cultural guide 60–61

helpful strangers 57, 58, 59–60, 61–62, 63

design 26, 39–41; accounting for how researcher is perceived 43–48; compatible with background 42; compatible with researcher’s skills 42–43; evolution of 55–56; impact of broader context 41, 54; impact of academic department 53; impact of gender 41; funder impact; modifications (not carved in stone) 41, 50–52, 54, 55, 56; specifying research limitations 40 documentary research 71, 81

fads and fashions 72–73 faulty research, problems with 103 favelas research (Brazil) 37, 74, 73 folk singer study 20–60 foreign research 73, 89–90 fun doing research 8, 22–23 funders and research 35, 48–50

ideology 28, 29 in depth interviews 21, 23, 43, 69, 71, 84, 112, 118; order of interviewees 118–119; preparation for 119; recording 119; repeat interviews117; without recording or notes 120 local government project 16, 36, 87 logistical problems 90–93; getting there 91–92; in documentary research 91; scheduling multiple interviews 93 luck 41, 57, 58 methodological improvisation 73–76 mistakes 4; learning from 78–80, 81; recovering from 80–82 obstacles 90; access denied 53, 88; covid 97; environmental changes 97–98; in international research 93; fatigue 94’

130

Index

political changes 98; push back 101, 102, 103, 106 personal consequences of research: life lessons 20–21; personal growth 9–11; political moderation 15–18; rites of passage 7; self discovery 7; tolerance increased 18–19 physical dangers 104, 107 police studies 19, 46 policy research 26–27, 99, 100, 126–127, 128 positionality theory 5 prison research 33–34, 61, 94, 105, 110 project ideas: academic theories, 28–29; from data gathering skills 66; guiding principles 25–26; from prior project 29–30, 85; personal experiences 27, 34, 35–36, 37; policy questions 31–21; unanticipated opportunities 31, 34 project specific skills 70 protecting data 81 public data sets, accessing 84 qualitative research, lack of training for 65, 68 reports 122, 125, 127–128 research design (see design)

research ethics 102–103, 107–108, 110–111; in applied research 11 sampling improvisation 1, 71, 82, 73, 98 sea nomad project 56, 91–92, 110 sensitive topic 79, 90, 109 serendipity 55–58 site selection 84–85 social skills 69–70 survey research, tricks of the trade 115–116 surveys, comparing over time 75–76 team research 76–78 Thailand research 44, 51, 73, 89, 92, 104, 122 tricks of the trade: qualitative research 117–119; survey research 115–116 trust, maintaining 45, 80, 110, 111, 122, 123 Vietnam research 107, 118 vulnerable population, researching 109 Washington, D.C. project 9, 13, 23, 54, 87 zoning project 57, 62