Small-Scale Research : Pragmatic Inquiry in Social Science and the Caring Professions [1 ed.] 9781446205396, 9780761968627

Citation preview

Peter T, Knight

Small­ Scale

Small-Scale Research

Copyrighted Material

Copyrighted Material

Small-Scale Research

Pragmatic Inquiry in Social Science and the Caring Professions

Peter T. Knight

SAGE Publications London. Thousand Oaks. New Delhi

Copyrighted Material

{,

Peter Knight 2002

First published 2002

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system. transmitted or utilized in any form or by any means. electronic. mechanical. photocopying, recording or otherwise. without permission in writing from the Publishers.

SAGE Publications Ltd 6 Bonhill Street London EC2A 4PU

SAGE Publications Inc 2455 Teller Road Thousand Oaks, California 91320

SAGE Publications India Pvt Ltd 32, M-Block Market Greater Kailash - I New Delhi 110048

British Library Cataloguing in Publication data

A

catalogue record for this book is available from

the British Library

ISBN 0 7619 6861

X

ISBN 0 7619 6862 8 (pbk)

Library of Congress control number available

Typeset by SIVA Math Setters, Chennai. India Printed in Great Britain by Biddies Ltd, Guildford. Surrey

Copyrighted Material

Con te n ts

List of Boxes

Vll

List of Figures

IX

List of Tables

x

Preface 1

2

3

4

Xl

Starting with Writing

1

Private writing Creating research questions Writing a literature review

1 5 11

Research as Claimsmaking

16

Research is more thinking than doing Claims that research may make Research standpoints Research forms Generalizing and learning from research A pragmatist account of social research

17 23 33 37 43 46

Face-to-face Inquiry Methods

49

Instrument structure and question formation Special concerns in face-to-face research Observation Interviews Nominal group technique Focus groups Memory work Experiments

54 54 56 61 69 70 71

Research at a Distance

72 80

Measurement, rating scales and lookalikes Questionnaires Diaries, logs and journals Q-sort Concept mapping Image-based research Documentary analysis

Copyrighted Material

80 87 95 97 98 101 1 04

vi

5

6

7

8

9

10

Sma l l -Scale Research

Unobtrusive methods Post-empirical research

1 08 1 09

Research Design: Bringing It All Together

114

A palette of methods Sampling, generalizations and research methods Multi-method design Quality

119 119 127 1 33

Complexity

144

The significance of complexity Complexity and organizations Complex people

146 151 156

Doing It

161

Being pragmatic Data capture Disclosure and harm

161 163 1 69

Sensemaking

173

Data analysis: levels, objectivity and continuity Numbers and data analysis Analysing qualitative data

1 73 1 76 1 82

Writing, Disseminating and Influencing

193

Public writing Disseminating The influences of social research

193 1 99 203

Reflecting

210

Hints for hard-pressed researchers A quality check

210 211

References

213

Index

220

Copyrighted Material

Boxes

Box 1 . 1

Alternatives t o traditional research reports

Box 2.1 Box 2.2 Box 2.3

Pattie's study Advice to Pattie on rescuing her study Extract from an evaluation agreement: principles guiding the Townside High School evaluation

Box 3 . 1 Box 3.2 Box 3.3 Box 3 .4 Box 3 .5 Box 3.6 Box 3.7 Box 3.8 Box 3.9 Box 3.10

Box 4.1 Box 4.2 Box 4.3 Box 4.4 Box 4.5 Box 4.6 Box 4.7 Box 4.8

The safety o f Prozac An ethnographic study among the Bush Cree An ethnographic study of African American girls in an elite high school Participant research and body culture within fitness gyms Semi-structured interviewing and the responsibilities of counsellor supervisors Semi-structured interviewing, body modification, fashion and identity Conversational interviewing, crack dealing, gender and arrest avoidance Nominal group technique and planning a communication education programme for older people An experimental study of reading speech and hearing print An experimental study of the representation of facial expression of emotion Attitude scales and a study of authoritarian and paranormal beliefs A fragment of a draft Likert scale exploring undergraduates' self-theories An extract from a grid used in rating the professional standing of occupations A questionnaire survey of drink driving and deterrence A questionnaire survey of the effect of the prison environment on inmate drug taking behaviour Q-sort in an evaluation of a drop-in youth centre Q-sort in organizational research Image-based research into the role of environmental complexity in the well-being of the elderly

Copyrighted Material

4 19 21 41 53 57 58 58 64 65 65 70 76 77

82 83 84 91 91 97 99 102

viii

Box 4.9 Box 4.10 Box 4.11 Box 4.12 Box 5.1 Box 5.2 Box 5.3 Box 5.4 Box 5.5 Box 5.6

S m a l l -Scale Research

Deconstructive discourse analysis: feminist deconstruction of heterosexual orgasm Discourse analysis of retrospective accounts of drunken behaviour Analysis of secondary school prospectuses Analysis of media representations of child abuse Mixed methods: using interviews to validate a questionnaire Mixed methods: the termination of an established needle exchange programme Mixed methods: interviews and newspaper analysis Mixed methods: literature, interviews and the need for more beside An internal validity prompt sheet Reliability in survey interviews: guidelines for interviewers

105 106 107 108

129 131 131 132 135 139

Box 7.1 Box 7.2 Box 7.3

Pragmatic responses to nine common mishaps Collecting data A critique of insider research

162 168 1 70

Box 8.1

Statistical techniques and four qualities of data

1 75

Box 9.1 Box 9.2

Planning a report: a connected outline Ways of improving the chances that your work will have an impact on practice

195

Copyrighted Material

207

Figures

Figure 2.1 Figure 2.2 Figure 2.3

Convergence and divergence in human experience Typical stages in action research inquiries Research methods and generalizations

Figure 5 . 1

Elements i n a multistage research design

123

Figure 6.1

A degree programme represented as an activity system

153

Figure 8 . 1 Figure 8.2

Figure 9 . 1

Qualitative data analysis: creating and applying coding categories Variations in the complexity of qualitative data analysis Research purposes and influences o n practice

Copyrighted Material

26 39 47

1 85 186 204

Tabl es

Table 1 . 1 Table 1 .2 Table 1 . 3

Selecting, specifying and connecting Small-scale inquiries into the assessment of nurse competence Elements o f a literature review

8 14

Table 2.1 Table 2.2

Three tendencies in research thinking and practice Some characteristics of three research forms

28 38

Table 3 . 1 Table 3 .2

Levels o f participation i n ethnographic studies Characteristics of structured, semi-structured and lightly structured questionnaires and interviews

57

6

63

Table 4.1 Table 4.2 Table 4.3

Questionnaires and interviews A q-sort template for 1 00 prompt cards Prompts for reflective inquiry

89 99 113

Table 5.1 Table 5.2

A palette of research methods Multiple methods, diverse findings

115 1 29

Table 7.1

Data capture methods and costs

1 65

Table 9 . 1

Criticisms o f social research and some responses to them

206

Copyrighted Material

Preface

In the twentieth century the amount of research into social thought and action grew relentlessly and rapidly. Scientific management and rational decision-making depended upon it, as did cost-benefit analyses, environ­ mental and social audits, and evaluations. Social policies were suppos­ edly informed by research, which also 'benchmarked' the best practices that then became the goals for all to pursue; and who could disagree with the common-sense idea that social problems could best be tackled once they had been understood thanks to penetrating inquiries? Sociologists, psychologists and historians obviously had a research interest in the social but so too did those interested in marketing, management and organizations, and others concerned with fields such as health, deviance and welfare. And new questions - about gender, everyday cultures, sexu­ alities, discourses and the media, for example - j oined older ones. Not surprisingly the number of researchers grew, helped by the massive expansion in post-war higher education. So too did the number of research­ literate practitioners because many occupations were caught up in a con­ spicuous consumption of credentials which led to a first degree becoming an entry qualification and a master's degree a passport to promotion. In order to establish their academic credentials these bachelor 's and mas­ ter 's degrees in education, management, criminology and public health (for example) exposed students to research findings, as you would expect, and increasingly expected students themselves to do research. As social research spread across the English-speaking nations, collect­ ing more and more initiates in the process, so too it changed. Earlier in the century it was mainly seen as a pursuit of truth (or, according to some, of truths) that would succeed as long as the right methods were rigorously applied in many large studies. Social research was an elite activity at the beginning of the century and an open one by the end. Beliefs in the tri­ umph of the method were supplanted by beliefs that anyone could inquire into social thought and practices and that anything that could be used as a method of inquiry could be used as a method of inquiry. Nor were massive attacks on problems seen as the only, the best, or even use­ ful ways of trying to understand social practices that were increasingly understood to be distinctively marked by particular circumstances, people, beliefs and thoughts. At the same time microelectronics made it possible for one person with a computer, a Walkman and a photocopier to do alone what once needed a team. Small-scale research became legitimate and possible. By 'small-scale research' I mainly mean systematic inquiries that involve one person, little

Copyrighted Material

XII

S m a l l -Scale Research

or no funding and a fairly short period of time to complete the inquiry and the report, but I also include as small-scale researchers two hard­ pressed students with a bit of money for travel and coffee and a project to get done in almost no time at all. Small-scale researchers like them make up the majority - probably the great majority - of the people who at any one time are doing research projects. Nevertheless, small-scale inquiry can still be seen as the runt, a puny form of the real thing - probably harmless but hardly something to be taken seriously. This book says otherwise, claiming that it is of value to the inquirer herself or himself (and that is not a trivial matter); that it may be of value to the research participants; and that it can be presented as a valuable contribution to the wider worlds of practice, policy and theory. The big problem with small-scale research is not that it is small scale: it is the naive belief that good research depends on finding the 'killer ' method and then making more effort to get more information to be analysed more thoroughly. If small-scale research has a problem it is because method and effort routinely displace thinking, sensemaking and claimsmaking. Books are organized by authors, although skilled readers use the index to stamp out their own pathways, as you are advised to. But the author still organizes the book by putting one thing before another, forcing a lin­ ear pattern onto things that might be very different. Despite my best efforts this book still suggests that researchers do this, then that, then that, then 'abracadabra', it's finished. But social research is not like building a h ouse: foundations, walls, roof. It is more like a nest of snakes with sepa­ rate bodies looping back on themselves and tangled one around another, all moving and alive. As Chapter 6 argues, it is complex. A recurrent theme is that doing social research is about making prag­ matic j udgements about what look to be the best ways of getting a grip on your research questions in specific settings. What seems to make sense in one research setting might be less attractive in another; research methods that work for one investigator may not satisfy others or may disappoint that investigator when used in a different inquiry or on a different site; and different investigators may reasonably differ about the questions to ask in the same setting or agree on the questions but differ about the understandings they construct through their research. Despite the emphasis on sensemaking, claimsmaking and complexity, this book is full of advice on methods, designs and analytical techniques. For people who need to get straight into that material, the extended Contents is a good place to start, quickly followed by reference to the Index. Tables and figures, such as Tables 5.1 and 7.1 and Figures 2.3 and 8.1, are intended to be useful summaries of much of the information about research methods, and although the boxes mainly contain illustra­ tive material, some, such as Boxes 7.1 and 9.1, are also intended as conve­ nient reference points. The two lists in the final, brief chapter can be read as a recapitulation of the book's main themes.

Copyrighted Material

Preface

xiii

Three people I write with have commented o n parts of drafts o f this book. I am grateful to Lee Harvey (University of Central England), Jo Tait (The Open University) and Paul Trowler (Lancaster University) for what I have gained by working with them and for their comments on what I have tried to do here.

Copyrighted Material

Copyrighted Material

1 Starti n g w i th Writi n g

As you would expect, in this book there are chapters o n research methods - what they are and how to use them - and on data analysis. Chapters 3, 4 and 8 deal with those topics. However, you might not expect the headline message to be that these chapters do not contain the key to doing good research. Thinking and writing matter more: ideally, they are part of the same process. What matters is how you adopt and adapt evaluation and inquiry methods so that you can offer some answers to worthwhile research questions. This means that the methods and the overall research design (Chapter 5) need to be fit for the purpose of trying to make the claim that the understandings you develop should be seen as a plausible and careful set of answers to those research questions. That, in turn, means that methods should be mindfully chosen, because otherwise it would be pure chance should they turn out to be fit for your research purpose. This chapter pursues those ideas by exploring the interplay of writing and thinking from the beginning of the small-scale inquiry. This is really a chapter about writing as a part of that research process, about private writing, which is treated very much as a form of thinking about what the research is for and about the ways in which research questions are identified and discarded or developed. Chapter 9 is about the public writing involved in reporting the responses you construct to the research questions that emerge from this early thinking and writing.

Private Writing

Although reading and writing need some technical knowledge, such as knowledge of phonics, letters and spelling, experts agree that both are, first and foremost, thinking processes. Researchers should start writing from the moment that they see a glimmer of an inquiry because it helps them to think and to capture their thinking. In both cases this written thinking is intended as a private act but researchers who are relaxed about others seeing their raw, naked writing benefit a lot by sharing their private writing with critical friends. It is easiest to see the value of private writing as a record of the inquiry process. Even small-scale projects can be quite complex and develop in

Copyrighted Material

2

S ma l l -Scale Research

unexpected ways. When researchers come to produce a public account of their inquiry and claims it can be surprisingly hard to remember what happened and why. This is why so many of them keep research journals, diaries or logs, sometimes handwritten in a notebook, often as a file of papers and increasingly as a folder in a word-processing program. Journals, which are started as early as possible, when the study is being planned, will usually include: 1 2

3 4 5

6 7 8

9

Messages to yourself about why you are doing what you are doing. Jottings about ideas that you may be able to use as you try and make sense of what your inquiry is showing. These insights will be valuable when you analyse your data (Chapter 8). Jottings about claims you might be able to make - for example, a note that what you see in an HIV education programme shows perfectly how policies are always changed when they are implemented. Notes of what you did (which ought to have an explanation of why you did them firmly attached). Copies of draft instruments (such as questionnaires, Likert scales, tests, q-sorts) and friends' comments on your proposed designs and instruments. Other documents and images will be filed with the main set of research information. Reflections about how you, the researcher, are influencing the research findings and on the significance of that influence. Lists of things to do. Lists of things to avoid ('I must stop getting involved in helping out and stand aside and be a non-participant researcher because when I'm a participant I stop researching'). Neat sentences, draft paragraphs, notes of headings or themes that you might use in your public presentation of your inquiry.

These nine categories do overlap a lot and most investigators only use some of them. Nor is a research diary something that has to be completed every day, although ethnographic researchers, who participate in a social setting and immerse themselves in its cultures, are fanatical journal keep­ ers because their fieldnotes usually are their research evidence. What is indisputable is that researchers who do not keep some form of j ournal create two nasty problems for themselves: at any point when they are thinking about the inquiry (and thinking ought to saturate the whole pro­ ject) they are cut off from their earlier reasoning and insights; and when writing a public account of the project they find that they cannot properly explain why the inquiry was designed the way it was, why Likert scales were preferred to a q-sort and what was wrong with the first draft inter­ view schedule (which has been lost in any case). Private writing is important as a way of creating a research memory but its other use is no less significant. This can be best explained by extending the idea that research is primarily about thinking and saying that it pivots

Copyrighted Material

Start i n g with Writing

3

on sense making and claimsmaking. By sensemaking (a term taken from Weick, 1 995) I mean creating understandings of the research topic, situation and questions. Claimsmaking is about connecting those under­ standings to ideas that are discussed in academic communities, amongst practitioners or in policy-making circles. It entails arguing that your inquiry is significant because it is reasonable to believe that it can con­ tribute to thought or practice in the ways that you indicate. Chapter 2 has more to say about claimsmaking and Chapter 8 treats data analysis as acts of sensemaking. Chapter 6 sets out some further ideas on the ways in which sense and claims can be constructed. As far as this chapter is con­ cerned the point is that researchers ought to be continually pondering how they are to go about making sense, about the sorts of claims they might make and how those claims might be sustained. Writing helps many people in at least two ways. It requires concentrated attention that may be missing from other, more fragile, ways of thinking. The other contribution is writing's challenge to say things precisely and to make connections between ideas. It is a challenge which ought to sharpen think­ ing and make it possible to develop the more complex understandings that come from spinning webs of ideas. The great advantage of writing-to-think that is proper, joined-up and grammatical is that it can be edited at a later date and dropped directly into public reports and research papers. Private writing alternatives can be j ust as valuable. Three that I use are: •

•

•

Jotting ideas on Post-its or scraps of paper and putting them in folders organized by theme. Newspaper cuttings and references to publica­ tions that I ought to read go in there too. This book started with 13 folders (one for each chapter I expected to write) whose contents were rearranged several times as I reconsidered how to organize it. Mind mapping'" key ideas and themes. This is more an organizing than a writing tool. Having read through a folder, I jot the main ideas that I want to use on a blank page and rearrange them in successive versions so that similar ideas are put next to each other, boundaries are drawn (and redrawn) around them to show that these are coherent sets of ideas, and lines are added to link clusters and show how they relate to each other. See Chapter 4 and Buzan (1974) for examples and a complete explanation. Summary notes. Piles of the notes I have made on books and articles need to be compressed, which involves trying to get the main ideas, the best examples and apposite quotations on to, say, three sides of paper. Different coloured marker pens are then used to highlight things on these three pages that are on similar themes. A fresh set of notes is then made with care taken to ensure that I make it clear to myself what ideas and arguments I am using to organize each section and to connect sections to each other. I depend upon these 'connected plans' (see Chapter 9) when writing my first full drafts.

Copyrighted Material

4

Smal l-Sca l e Research

This writing does not have to be formal and polished because it is being treated as a way of supporting thinking and it is the quality of thinking that is important. This is writing as a way of working out what you want to say. Many people do small-scale research and tell interesting stories but it's often not at all clear what they want readers to learn or understand from it. It's not clear how the research findings fit with what they're try­ ing to say and that seems to be because they are not really very clear what they really do want to say. So, it would be very harmful if a writing block, which many people have, obstructed this private writing. That means learning to have an informal view of writing so that there is much less chance that fear of writing will inhibit thought. Following Boice's ( 1 992) finding that the new lecturers who published the most were those who wrote at least a little most days, my advice to small-scale researchers is to write something - formal writing, mind maps"; summary notes or jottings - often. He also found that those who deferred writing until they had good, clear stretches of time were the least productive.

Box 1 . 1 A l ternatives to trad i t i o na l research reports

Visual representations, including posters, photography, collage, video, exhibits of work Novels, dramatic works and other creative writing Fiction Biography and autobiography Presentations by researchers (sometimes including informants) Artefacts Innovative practices, programmes and products

In emphasizing the value of informal writing I am not minimizing the importance of good public writing. It is the main way of making claims about the significance of investigations, although alternative ways of representing research findings find favour in some research communities. Box 1 . 1 lists some of them. It will become clearer in Chapter 9 that plenti­ ful private writing is a part of good public writing. Briefly, the point is that for most people polished writing is the product of several drafts and that redrafting is best if critical friends advise on improvement. In other words, the first three (or five, or n) versions will get junked. Seen like this, anyone who invests a lot of time in getting a first draft as perfect as they can make it is wasting a lot of time because nothing is more certain than that friends will see ways of improving it that had eluded the writer. Efficient writers understand that the sooner they can get an intelligible but unfinished draft to friends the better. It means they have few hang­ ups about sharing incomplete and imperfect work because they recognize that no matter how long they spend alone trying to get it right, others will

Copyrighted Material

Starti n g w i t h Writing

5

see ways in which it could be better. They share their private writing as part of the process of making it into public writing, which means they are not precious about their writing and that they have learned to trust friends to be critical to their drafts. Creating Research Questions

If writing is a way of sensemaking and claimsmaking, and if they are at the heart of social research, then it is important early in an inquiry to be clear about the inquiry's purposes - about the research questions. They define the types of understandings to be sought and the sorts of claims that might be made. When investigators have a topic of interest but do not themselves know what their research questions are then, unless their pur­ pose really is just to see how things are, their private writing and thinking will be difficult because they lack focus. Imagine a case study of a hospital ward. It would certainly be possible to go into the ward, watch, listen and ask. There are circumstances when this sort of unfocused inquiry is useful, especially when inquirers are ven­ turing into new worlds, in the early stages of a goal-free evaluation or as pilot work for a larger study. As the inquiry progresses, researchers begin to identify key questions amongst the welter of commonplace observa­ tions, such as that nurses are busy, the food is awful and humour is important. This approach, sometimes called 'progressive focusing', is one way for investigators to identify organizing questions that direct their research gaze and which help to identify the sorts of claims they might make about the significance of their inquiries. Suppose that a researcher had been visiting a relative in hospital and thought that a hospital ward must be a good site for a small-scale study for an undergraduate dissertation. In my experience it is quite common for students to say that they want to study an area such as this because it has caught their interest, or because they are doing a course on, say, racism and would like to find out more. Of course students should choose research themes that interest them but unless someone, such as a super­ visor, can compel them to identify some feasible research questions within the theme of life on the wards or racism, then they are headed for a lot of grief. Worse, they may be blase and not realize that an inquiry, whether it is large scale or small, is not done by collecting information willy-nilly. Quite simply, there is too much going on for it to be possible for anyone to capture everything from every possible perspective, let alone to make sense of it and represent it to an audience. Research questions are needed to select, specify and connect, as Table 1 .1 indicates: •

In order to make a study manageable, it is necessary to narrow it down by selecting an aspect or two for study. Novice researchers - some experienced ones as well - always choose studies that are too

Copyrighted Material

S ma l l -Scale Research

6

Table 1 . 1

Selecting, Specifying and Connecting

Su pervi sor's response

'My research is go ing to be about . . .' A study of old people's l iving environments

Topic too big. Focus unclear. No i ndication of what the va l ue of the study m i ght be, or of who m i ght be i n terested by it. See a l so Box 4.8

An action research project to reduce drug taking in prisons

Too gener