A research handbook for patient and public involvement researchers 9781526136527

Citation preview

A research handbook for patient & public involvement researchers

ISBN 978-1-5261-3653-4

A research handbook

for patient & public involvement researchers 9 781526 136534

BEE (RESEARCH) COVER.indd All Pages

11/05/2018 16:16

A research

handbook for patient and public involvement researchers

BEE (RESEARCH) PRINT.indd 1

11/05/2018 16:14

Copyright © Manchester University Press 2018 While copyright in the volume as a whole is vested in Manchester University Press, copyright in individual chapters belongs to their respective authors. An electronic version of this book is also available under a Creative Commons (CC-BY-NC-ND) licence, which permits non-commercial use, distribution and reproduction provided the editor(s), chapter author(s) and Manchester University Press are fully cited. Published by Manchester University Press Altrincham Street, Manchester M1 7JA www.manchesteruniversitypress.co.uk

British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library

ISBN 978 1 5261 3653 4 paperback ISBN 978 1 5261 3652 7 open access

First published 2018

The publisher has no responsibility for the persistence or accuracy of URLs for any external or third-party internet websites referred to in this book, and does not guarantee that any content on such websites is, or will remain, accurate or appropriate.

The EQUIP project is funded by the National Institute for Health Research’s Programme Grants for Applied Research Programme (Grant Reference Number RP-PG-1210-12007) and Greater Manchester Mental Health NHS Foundation Trust (formerly Manchester Mental Health and Social Care Trust (MMHSCT)).

Graphic Design www.greg-whitehead.com Illustration www.mistermunro.co.uk

BEE (RESEARCH) PRINT.indd 2

11/05/2018 16:14

A research

handbook for patient and public involvement researchers

Edited by Penny Bee, Helen Brooks, Patrick Callaghan and Karina Lovell

BEE (RESEARCH) PRINT.indd 3

11/05/2018 16:14

Foreword Patient and public involvement in research is a requirement of most major health research funders. User-led research and involvement activities are important in shaping and determining research questions, assessing research proposals and guiding and informing research processes. All of these tasks require specific areas of knowledge and skills that can be difficult for members of the general public to acquire. People who get involved in health research as experts from experience now have a text book to support their research involvement journey. This book will be useful for many, including students at college or university, people working in health research and members of the public getting involved in developing and delivering research studies. I hope this book will encourage members of the public to become involved in health research, and build confidence in their own contributions, because we need them to be involved in research from the very beginning. The material presented in this book derives from a face-to-face methods course developed for public and patient representatives working on the EQUIP study. It is a book written in partnership between the study academics and the mental health service users or carers who worked with them as advisors and research assistants. The EQUIP team did not start out with the idea of writing a book. It was the experience of working together over five years that led to this joint venture. If my own experience is anything to go by, patient and public involvement in research is often a reciprocal journey. Academic members of the EQUIP team learnt how to integrate expertise from experience into their work, and the service user and carer representatives positively shaped their research in new and unplanned ways.

4

BEE (RESEARCH) PRINT.indd 4

11/05/2018 16:14

A Research Handbook for Patient and Public Involvement Researchers

And what a brilliant output we have all been given. A very comprehensive book with a topic list, covering all the basics needed for large-scale health research projects: systematic reviews; research design and analysis using both qualitative and quantitative approaches; health economics; research ethics; impact and dissemination. The book is well written and interesting, with a mix of research practices clearly outlined, and an insight into how public and patient representatives can be involved in them and shape decisions. We learn how a carer worked with the team to author a peer review paper and a service user co-delivered a training intervention. We learn how a service user and carer advisory panel influenced study outcome measurement and how service users and carers become involved in focus group data collection and dissemination. I run a charity that champions the involvement of experts by experience in mental health services research projects. We work with people to create lived experience advisory panels and to support peer researchers delivering user-led projects or collaborative studies. This book is going on the reading list for all our staff, and for the service user and carer advisors we work with, as part of their induction process. Thank you to all the EQUIP team for putting time and effort into this co-produced project and for sharing your partnership with us.

Dr Vanessa Pinfold Co-founder and Research Director, McPin Foundation

5

BEE (RESEARCH) PRINT.indd 5

11/05/2018 16:14

Abbreviations BDI Beck depression inventory CBT Cognitive behavioural therapy

FGA First generation antipsychotic medication ICER Incremental cost-effectiveness ratio

PICo Population, interest, context PPI Patient and public involvement

NHS National health service

PREM Patient reported experience measure

CONSORT Consolidated standards of reporting trials

NICE National institute for health and clinical excellence

PROM Patient reported outcome measure

COREQ Consolidated criteria for the reporting of qualitative research

NIHR National Institute of Health Research

QALY Quality-adjusted life year

CD-RISC Connor-Davidson resilience scale

EDI Eating disorder inventory EQUIP Enhancing the quality of user involvement in mental health care planning ESRC Economic and social research council

OECD Organisation of economic co-operation and development PANSS Positive and negative syndrome scale PICO Population, intervention, comparison, outcomes

REC Research ethics committee RCT Randomised controlled trial SAP Statistical analysis plan SD Standard deviation SGA Second generation antipsychotic medication SOAS-R Staff observation aggression scale - revised SUCAG Service user and carer advisory group WTPT Willingness to pay threshold

6

BEE (RESEARCH) PRINT.indd 6

11/05/2018 16:14

This book was developed during a five-year research programme funded by the UK’s National Institute for Health Research (NIHR). This study aimed to improve service user and carer involvement in care planning in mental health services. The study was called Enhancing the Quality of User Involved Care Planning in Mental Health Services (EQUIP).

Its aim is to help other public and patient representatives increase their understanding and skills in research methods. The EQUIP programme used a range of different research methods to achieve its goals, and you will read more about these as you progress through this book. More detail on the EQUIP programme is provided on page 8.

As part of our work on EQUIP, we developed The EQUIP programme involved patient and and delivered a successful research methods public representatives with lived experience course for service users and carers. The aim of mental health services, but all of the of this course was to help these individuals research methods that we discuss are used engage with our research and research team in both physical and mental health research. and to work together in true partnership. Whatever your background, or health experiences, this handbook could be helpful. This book has arisen out of our partnership, and has been co-written with our service users and carers.

A Research Handbook for Patient and Public Involvement Researchers

How this book came about

How this book is presented Each chapter will provide a brief overview and outline key learning objectives before moving onto the main body of the chapter. All chapters end with a reflective exercise, to help you check what you have learnt. Where helpful, there are also some suggested sources of additional reading. You will find stories from some of the PPI representatives who worked on EQUIP scattered throughout. These stories reflect on our representatives’ own experiences of being involved in the different types of research we discuss.

Andy, Lindsey, Lauren, Debbie and Joe attended our first research methods course. Andy, Lindsey, Lauren joined the EQUIP team as grant co-applicants and researchers. Along with Garry, Debbie and Joe became members of our Service User and Advisory Panel. Andy, Lauren, Debbie and Garry have lived experience of mental health difficulties, and of using mental health services. Lindsey is a carer for her son, who lives with psychosis. Joe has worked as a mentor with AnxietyUK. We hope you find their stories interesting.

7

BEE (RESEARCH) PRINT.indd 7

11/05/2018 16:14

An overview of the EQUIP study The EQUIP study aimed to improve service user and carer involvement in care planning in mental health services. We co-developed with service users and carers a training package for mental health professionals so that they would be better equipped to involve users and carers in their care. Service users and carers helped to design, shape and conduct the EQUIP study and you can learn more about their experiences in Chapter 1.

During the EQUIP programme, we worked with service users and carers to develop a new instrument, a patient-reported outcome measure, to measure the extent to which people were involved in their own care planning. You can read more about the importance of patient-reported outcome measures and how you might design and test them in Chapter 6.

We explored the organisational changes that needed to be made by Community Mental Health Teams and the wider healthcare The content of the training was ‘evidencesystem to improve user and carer involved based.’ This means that it was built upon care planning. We did this by talking to detailed knowledge of the care-planning different people and listening to their process, why service user and carer different views and perspectives. You can involvement in care planning may not have happened in the past, and what might be the learn more about this research approach in Chapters 7 and 8. best way of making sure it happens in the future. You will learn more about how to find We conducted all our work according to the and use research knowledge and evidence in principles of ethical research, and these are Chapter 2. discussed more fully in Chapter 9. Training was delivered by service users and Finally, we used lots of different ways to carers, researchers and health professionals tell users, carers, health professionals and to a range of mental health workers managers about our research findings, including doctors, nurses, social workers especially how user and carer involvement and occupational therapists working in in care planning could be improved. You Community Mental Health Teams. We can learn more about different ways to examined whether our new training course disseminate research in the final chapter of led to health professionals involving users this book, Chapter 10. and carers in their care. We also looked at how this training influenced health service delivery costs. To do this we used a specific research design, called a randomized controlled trial. You can learn more about this and other research designs in Chapter 3. You can learn more about the type of data we can collect in a trial and how to analyse this in Chapters 4 and 5. 8

BEE (RESEARCH) PRINT.indd 8

11/05/2018 16:14

A Research Handbook for Patient and Public Involvement Researchers

Chapter 1: Patient and Public Involvement (PPI) and the research process Andrew C Grundy

Chapter overview This chapter defines and introduces the different stages of the research process: from identifying a problem, to reviewing the literature; then developing a research question; designing a study; obtaining funding and ethical approval; recruiting participants; collecting and analysing data; and reporting and disseminating findings. This chapter will outline how users of health services, their carers and family members, and other members of the public can be involved in these different research stages, and demonstrate the impact that this involvement can have. Examples of different ways of involving and engaging public members in research studies are drawn from the Enhancing the Quality of User-Involved Care Planning in Mental Health Services (EQUIP) research programme.

Learning objectives By the end of this chapter you should be able to: 1. Understand the different stages of the research process 2. Understand the impact of Patient and Public Involvement in research 3. Understand the different ways you could be involved

9

BEE (RESEARCH) PRINT.indd 9

11/05/2018 16:14

Introduction The Frascati Manual provides an internationally recognised definition of research. It defines research as:

“

creative work undertaken on a systematic basis in order to increase the stock of knowledge including knowledge of man, culture and society, and the use of this stock of knowledge to devise new applications.

”

(OECD, 2002, pp30)

Inviting members of the public to offer a lay, non-specialist perspective on the design and conduct of research studies is typically referred to as ‘Patient and Public Involvement’ (PPI). PPI is a term used to denote meaningful involvement in the design and conduct of a research study. It does not mean a person is involved in a research study as a participant. At the very least, a research study should be able to evidence consultation with service users, and at best, collaboration and partnership with them as an equal and valuable part of the research team. Meaningful involvement is not always easy to achieve. One of the most important factors influencing the outcome of involvement is the perspective of different team members, and the different skills, assumptions, values and priorities that each of them brings. Acknowledging and working with these different perspectives is precisely what makes PPI so valuable, but it can also be what limits its success. Effective PPI requires that equal respect is afforded to academic and patient and public researchers, that the perspectives of both parties are equally valued, and that the team as a whole develops and maintains a shared language and goal. It is also important that PPI opportunities are advertised as widely as possible to ensure that they are accessible to a broad range of people from different backgrounds. Adequate training should be provided to ensure that people can be involved in research in a meaningful way (e.g. research methods and how clinical services are organised and commissioned).

10

BEE (RESEARCH) PRINT.indd 10

11/05/2018 16:14

A Research Handbook for Patient and Public Involvement Researchers

Figure 1 Levels of service user involvement Leadership

Since the mid-1990s, increasing emphasis has been placed on the importance of PPI. The desire to strengthen the involvement and engagement of service users, carers and members of the public in research has been driven by:

Partnership Consultation

a. a strong moral argument that any publicly funded research that aims to benefit health status or health services should be shaped and informed by the people it will affect (Hanley, 2012)

Informing

b. accumulating evidence of the benefits of patient and public involvement in research (Staley, 2015) c. recognition that service users and carers, by virtue of their lived experience, can bring a wealth of experiential knowledge and expertise to the design and conduct of research studies (Faulkner, 1997; Repper, 2008). Different roles and opportunities for patient and public members to participate in research have emerged. Some people may wish to act as consultants, advising on multiple projects during the early phases of research commissioning and design. Others may choose to engage in one specific project, joining a project advisory panel who will guide and advise a research team throughout the life of that project. PPI advisory panels (sometimes called a Service User and Carer Advisory Groups (SUCAG) or Expert User Groups) provide an independent viewpoint on research progress, advising on research procedures and challenges as they arise and assisting with dissemination.

11

BEE (RESEARCH) PRINT.indd 11

11/05/2018 16:14

Service users and carers may also choose to be researchers. Unlike advisory panel members, these individuals work as trained, integral members of the research team, contributing to the design and conduct of the study, and in some cases, its funding application. Service users and carers can be named as principal investigators or study co-applicants. As principal investigators they might take the lead in managing, designing and carrying out a study, or in forming a collaborative team, in what is sometimes called user-led or usercontrolled research. The EQUIP study was conducted by a mix of researchers from different backgrounds. The different contributions that service users and carers made to the EQUIP research programme are shown in Figure 1. As you can see, service users and carers worked in many different roles and had a range of different experiences, and you will learn more about their personal stories throughout the book. First though, let’s take a journey through the research process and look more closely at where and when PPI opportunities can arise.

Beginning the research process The beginning of the research process involves identifying a problem, reviewing the current literature to clarify what is already known and developing a research question to resolve remaining uncertainties or fill knowledge gaps. This is rarely a straightforward process.

12

BEE (RESEARCH) PRINT.indd 12

11/05/2018 16:14

A Research Handbook for Patient and Public Involvement Researchers

Reviewing the literature on a particular topic can help to identify relevant papers quickly, enabling researchers to build upon, rather than duplicate, existing work. It can help to narrow a broad problem down to a specific issue, assess its importance and develop an appropriate and meaningful research question. We will learn more about how to conduct a literature review in Chapter 2. It is good practice to involve stakeholders in the review process (Rees and Oliver, 2012). In research, the term ‘stakeholder’ is often used to refer to those individuals, groups or communities who have an interest in, and are likely to be affected by, the conduct and findings of a research project. In mental health research for example, important stakeholders can include service users, carers, wider family members, mental health professionals, service managers and commissioners. Working with stakeholders to define the appropriate focus for a literature review, and identifying and prioritising the research questions that might arise from it, is therefore an important step in making sure any future studies have relevance and applicability to health services (Arksey and O’Malley, 2005). We know that people who use mental health services, carers and professionals have different views about effective care, with professionals often prioritising a clinical model of care, and service users emphasising a social model of care (Rose, 2003). Similarly, they may also have different research priorities.

Service users want research that makes a noticeable difference to their care experiences, both personally and generally (Beresford, 2005). More importantly, they want research that leads to positive improvements in the whole of people’s lives, not just in the design and delivery of mental health services (Faulkner and Layzell, 2000). Given these priorities, it’s crucial that service users, carers and public members are consulted, or even better, are asked to collaborate in the early stages of the research process, helping to prioritise research ideas and to frame research questions. Involving service users should lead to questions that are more relevant and meaningful to participants. Where service users are not part of the actual study team, this kind of involvement can be achieved through holding focus groups, discussions or local and national stakeholder events.

13

BEE (RESEARCH) PRINT.indd 13

11/05/2018 16:14

Designing the study Having developed a research question, it is important to decide which methods might be the best to answer it. Research generally falls into two types: quantitative research and qualitative research. Quantitative generates numerical data, often through the use of large studies, using methods such as questionnaires and surveys. We will learn more about how to collect and analyse quantitative data in Chapters 3 and 4. Qualitative research explores attitudes, behaviour and experiences through methods such as interviews, focus groups or observation, and we will learn more about this in Chapter 7 and 8. Once the type of research is set, the study needs to be designed in detail. Collaborating with service users and carers in the design of a research study allows researchers to understand how best to approach potential research participants, why people might drop out of research studies (Goward et al., 2006), why an intervention might work from the user/carer perspective (Allam et al., 2004), what people might find most useful about different interventions and what might be the most appropriate outcomes to measure (Faulkner, 1997). When evaluating if, and how well, a new intervention works for instance, it is often necessary to ask people to report treatment ‘outcomes.’ Popular outcomes for mental health interventions might include scales that measures symptoms, recovery, hope or daily activities and functioning. Treatment costs or the need to use other services might also be measured. Interestingly, the most common clinical measures are often the ones that service users like the least, because they do not tap into the priorities of service users themselves (Crawford, 2011). Service users want measures that can capture both the negative and also the positive effects of treatments, and are often willing to complete longer questionnaires to ensure that this is possible (Kabir and Wykes, 2010). The act of completing questionnaires for service users and carers can in itself be challenging. Collaborating with service users and carers to select and prioritise outcome measures for quantitative research studies is therefore incredibly important, and can help to minimise the number of questionnaires or questions that participants miss out or refuse to answer. It can also help to get feedback on the length of time needed to complete any questionnaires.

14

BEE (RESEARCH) PRINT.indd 14

11/05/2018 16:14

A Research Handbook for Patient and Public Involvement Researchers

Increasingly, service user involvement is also being sought in the design of new outcome measures, for example in selecting possible questions, prioritising questions and/or reducing the number of questions included in a scale, and commenting on ease of response and the emotional impact of its wording (Wiering et al., 2016). Is it too distressing or demoralising for example? Questionnaire development can be a complex and time-consuming process, and we will learn more about this in Chapter 6. Similarly, when it comes to designing focus group or interview topic guides, service users often ask different questions to non-service users (Rose et al., 2004). Gillard et al. (2010) compared ‘academic-researcher’ and ‘service user-researcher’ questions and found that the latter were more concerned with ‘how things felt’ rather than ‘what happened next’. They may also ask questions in a different way, using different phrases and words. It is therefore crucial to involve service users, carers and public members in this stage of the research process.

Funding Depending on their size and purpose, research studies can be expensive. New research proposals will therefore usually be submitted to a funding body. Funders will look to see if the proposed research study is important (from the funder’s point of view); that the proposed methods will answer the research question; that the study represents good value for money; that it be conducted safely and in line with ethical guidelines (see chapter 9); and that the research team are the right people to do the work (Aldridge and Derrington, 2012).

15

BEE (RESEARCH) PRINT.indd 15

11/05/2018 16:14

Funders will also want to see that the proposal is well structured and is written simply and clearly, including a summary of the proposed research which is accessible and understandable to members of the public (Aldridge and Derrington, 2012). Service user, carer and public member involvement has an obvious role to play in this, and most funding bodies now mandate PPI in the development of research proposals and grant applications. Many funders also seek to actively include service user, carer and public members in the appraisal of funding applications, both as peer reviewers and as panel members participating in the meetings where funding decisions are finally made.

Figure 2

What are funders looking for in a research application? Aldridge and Derrington, 2012 ✔ The research is important (from funder’s point of view) ✔ The research will answer the question ✔ The proposal represents good value for money ✔ The proposal is ethically sound ✔ The research team are the right people to do the research ✔ A well-structured and well written application ✔ A clearly written proposal ✔ Includes a lay summary ✔ Incorporates patient and public involvement

16

BEE (RESEARCH) PRINT.indd 16

11/05/2018 16:14

All research studies, with the exception of service evaluations and audits, need to be approved by a Research Ethics Committee (REC) before they can begin. A detailed protocol, which outlines how researchers will deal with any ethical issues (e.g. confidentiality, informed consent etc.), is submitted electronically to the REC and reviewed by a multi-disciplinary team. Researchers may need to attend an REC, if invited, to discuss their application. Additional permissions to carry out research in specific organisations may also be required. We will learn more about research ethics and governance in Chapter 9. Informed consent is an important principle of ethical research. This means that all potential participants must be allowed to choose to take part in a research study, without fear of losing care, or worrying about what might happened if they don’t. To make sure this decision is an informed one, all potential research participants must be given clear and accurate information about why the study is being conducted and what participation would involve. Service user involvement can help to ensure this information is presented clearly and provides all the details that people might want to know. It can ensure that consent is truly informed by making sure that the right information is accessible (Allam et al., 2004) and that potentially offensive, dismissive or misleading statements are avoided (Rose, 2003). Service users may have different perspectives on what might cause distress and how that should be managed (Nicholls et al., 2003).

A Research Handbook for Patient and Public Involvement Researchers

Ethics

17

BEE (RESEARCH) PRINT.indd 17

11/05/2018 16:14

Figure 3

PPI involvement in the design of an ethically appropriate study ✔ Making information assessable to enable truly informed consent (Allam et al., 2004) ✔ Identify potential offensive, dismissive or misleading questions (Rose, 2003) ✔ Insight into what might cause distress and how to manage this appropriately (Nicholls et al., 2003) ✔ Feedback on proposal and all supporting documentation (e.g. participant information sheets, consent forms, adverts etc.)

Once a study has all the necessary approvals, it can begin to recruit research participants. The precise recruitment strategy that is used will have been outlined in the ethics submission. It could, for example, include recruiting in person at clinics, or via poster display, or via social media. Patient and public representatives who sit on advisory panels can often advise on or act as a conduit to service user networks, potentially increasing access to people who may or may not be in contact with statutory health services. In some instances, the research topic will be sensitive, and in these cases, trained service user researchers can play a valuable role in study recruitment. People from ‘seldom heard’ or marginalised groups may be more willing to participate in a project involving someone they know (Fleischmann and Wigmore, 2000; Ennis and Wykes, 2013).

18

BEE (RESEARCH) PRINT.indd 18

11/05/2018 16:14

Figure 4

A Research Handbook for Patient and Public Involvement Researchers

Exactly how data is collected will depend upon the methods chosen to answer the research question. As an integral part of a multi-disciplinary research team, service users, carers and public members can be co-investigators with an active role in collecting data (Hanley, 2012). For example, they could help assist someone in the completion of a questionnaire, or they could facilitate a focus group or face-to-face interview. Often this can enhance the richness and relevance of the data that is obtained. Participants may also choose to share a different type or level of information with someone who has had a similar life story or experience compared to somebody who has not.

Analysing data The move from collecting to analysing data is rarely a linear process, and exactly how and when researchers begin their analysis will largely depend upon their underlying approach. Quantitative studies will involve data-input and some kind of statistical analysis (see Chapter 4). Qualitative studies may involve identifying themes (or codes) from interview and/or focus group transcripts or observational field notes (see Chapter 8). There is evidence to suggest that service users, carers and public members interrogate qualitative data differently, asking different questions of it and interpreting qualitative data in ways that reflect their priorities (Gillard et al., 2010). These interpretations can be fruitfully pooled with other (non-service user) perspectives to provide a more holistic and meaningful analysis.

19

BEE (RESEARCH) PRINT.indd 19

11/05/2018 16:14

Reporting findings Researchers often talk about ‘dissemination’ or ‘knowledge transfer.’ These terms are used to refer to the mechanisms and strategies by which other groups and communities become aware of, obtain and/or subsequently make use of new research findings (Freemantle and Watt, 1994). Service users may have very different dissemination priorities to academicresearchers, whose main emphasis is often upon publication in peer-reviewed journals or presentations at academic and professional conferences. Service users, carers and public members may publish or present the research in their local communities or groups, ensuring wider reach and understanding of the study and its findings. Service users play an important role in framing research findings, in deciding what implications they may have for practice (Hanley, 2012) and in preparing accessible summaries of projects for dissemination that explain the results in a clear and jargon-free way. Increasingly, academic and professional journals are recognising the value of this input and seeking articles led by or co-authored with service users and carers.

20

BEE (RESEARCH) PRINT.indd 20

11/05/2018 16:14

If you are a patient or a carer, somebody who has used health services in the past or a general member of the public, then you will most likely have a unique viewpoint on health service delivery and research, and a valuable contribution that you can make. Getting involved in research can be extremely rewarding but it is not always easy to do. Barriers include a lack of awareness about opportunities to get involved in research, language barriers, physical or emotional health, a lack of confidence, the behaviour or attitudes of researchers, or inappropriate timing/location of meetings. Research teams need to make sure that people are and can be involved in a meaningful way. This means making sure that PPI members are properly and regularly supported, and that they are fully recognised for their time and knowledge contributions, as important and integral members of the research team. The EQUIP team has found that offering service users and carers a short course in Research Methods and Design can help to facilitate collaborative relationships and can give members the confidence to play their role in the multi-disciplinary research team.

BEE (RESEARCH) PRINT.indd 21

A Research Handbook for Patient and Public Involvement Researchers

Supporting and equipping people to be involved in research

11/05/2018 16:14

PPI stories from EQUIP Next, Andrew and Garry (members of the EQUIP team) reflect on their experiences of being involved in the EQUIP programme of research.

Andrew’s story Following the first research methods training course I was invited to be a coapplicant on the EQUIP research programme. This meant that I became a member of the research team for the whole of the EQUIP programme, and shared responsibility for the design, management and conduct of the EQUIP studies. One of my roles was to promote the studies across Nottinghamshire, as this was one of our main research sites. It’s been a real privilege to network with local service user and carer groups, who have invited me back time and time again to talk about our work. I’ve really appreciated their help and support! I received some practical training in qualitative research methods, and thoroughly enjoyed co-facilitating focus groups and conducting interviews in the

EQUIP programme. It was wonderful to take a lead on the analysis of the service user data, to help write up our findings, and to assist in developing an animation to outline our new model of care planning involvement. It was also really interesting to see how new questionnaires are developed and tested – and it’s been great to be part of producing a new measure of service user involvement in care planning, and an audit tool that services can use to improve their care. Co-delivering the EQUIP training intervention took up a lot of my time, but I absolutely loved it. I was involved in recruiting mental health teams to the trial, in co-facilitating their two days of training and in providing subsequent clinical supervision. It’s been a privilege to help Trusts consider how best to implement our training, working with service users, carers and clinicians so that they can go on to deliver the training themselves. It’s been a joy to be able to present at events and conferences, I’ve loved telling people what we’ve been finding out. I’ve had a few mental health ‘blips’ along the way, but I feel that with the right training and mentorship I’ve been able to use my lived experience and expertise in a constructive way.

22

BEE (RESEARCH) PRINT.indd 22

11/05/2018 16:14

During the time I’ve been involved in EQUIP, I’ve really valued the meetings that have taken place. As a group we’ve dedicated time to looking at the process of care planning. In the focus groups that I have observed, service users have been very open and very honest, sharing experiences in a safe and supportive setting.

programme and has been published as important research findings. I think focus groups are very good; by helping people to share their feelings and experiences, we can travel a long way towards improving the care that we receive.

We asked people to talk about their positive experiences, what has worked well for them and also what could be improved. We gathered a great deal of information that has been to the EQUIP

A Research Handbook for Patient and Public Involvement Researchers

Garry’s story

Reflective exercise opportunities are there for members of • What the public to get involved in health services research?

there any stages of the research process • Are that are of particular interest to you? Why? • What skills can you offer a research team? might the chapters in this book help you • How and what other training might you need?

23

BEE (RESEARCH) PRINT.indd 23

11/05/2018 16:14

References and further reading 1) Aldridge, J. and Derrington, A. M. (2012) The Research Funding Toolkit: How to Plan and Write Successful Grant Applications. London: Sage.

13) Hanley B. (2012) Involving the public in NHS, public health, and social care research: Briefing notes for researchers. Eastleigh: INVOLVE.

2) Allam S., Blyth S., Newman, A. and Repper, J. (2004) Researching collaboratively with service users, Journal of Psychiatric and Mental Health Nursing 11:3, 368-373.

14) Kabir, T. and Wykes T. (2010) Measures of outcomes that are valued by service users., Chapter 1 in G. Thornicroft and M. Tansella (Eds) Mental Health Outcome Measures. RCPsych 3rd edition.

3) Arksey, H. and O’Malley, L. (2005) Scoping studies: Towards a methodological framework, International Journal of Social Research Methodology: Theory & Practice 8:1, 19-32.

15) Nicholls, V., Wright S., Waters, R. and Wells, S. (2003) Surviving user led research: Reflections on supporting user led research projects. London: Mental Health Foundation.

4) Beresford, P. (2005) Theory and practice of user involvement in research: making the connection with public policy and practice. Chapter 1 in L. Lowes and I. Hulatt (Eds.) Involving service users in health and social care research. London: Routledge.

16) Organisation for Economic Cooperation and Development [OECD] (2015) The Measurement of Scientific and Technological and Innovation Activities, The Frascati Manual: Guidelines for Collecting and Reporting Data on Research and Experimental Development. OECD Publishing: Paris.

5) Crawford, M.J., Robotham D., Thana, L, Patterson, S., Weaver, T., Barber, R., Wykes, T. and Rose, D. (2011) Selecting outcome measures in mental health: the views of service users. Journal of Mental Health 20:4, 336-46. 6) Ennis, L., and Wykes, T. (2013) Impact of patient involvement in mental health research: longitudinal study. British Journal of Psychiatry 203:5, 381-386.

17) Rees, R. and Oliver, S. (2012) Stakeholder perspectives and participation in reviews., Chapter 2 in D. Gough, S. Oliver and J. Thomas, An Introduction to Systematic Reviews. London: Sage. 18) Repper, J. (2008) Carers as Researchers: Lessons from the Partnerships in Carer Assessment Project’. In M. Nolan, E. Hanson, G. Grant, and J. Keady (Eds) User Participation in Health and Social Care Research, Open University Press.

7) Faulkner, A., (1997) Knowing our own minds: Users views of Alternative and Complementary Treatments in Mental Health. London: Mental Health Foundation. 19) Rose D. (2003) Collaborative research between users and professionals: peaks and pitfalls. Psychiatric 8) Faulkner, A. and Layzell, S. (2000) Strategies for Bulletin 27, 404-406. Living: A report of user-led research into people’s strategies for living with mental distress. 20) Rose D., Fleischmann, P. and Wykes T. (2004) London: Mental Health Foundation. Consumers’ perspectives on ECT: A qualitative analysis. Journal of Mental Health 13:3, 285-294. 9) Fleischmann P. and Wigmore J. (2000) Nowhere else to go: Increasing choice and control within supported 21) Staley, K. (2012) An evaluation of service user housing for homeless people with mental health involvement in studies adopted by the Mental Health problems. London: Single Homeless Project. Research Network. London, MHRN. 10) Freemantle, N. and Watt, I. (1994) Dissemination: implementing the findings of research. Health Information and Libraries Journal 11:2, 133-137.

22) Thornicroft, G., Rose, D., Huxley, P., Dale, G. and Wykes, T. (2002) What are the research priorities of mental health service users? Journal of Mental Health 11:1, 1-5.

11) Gillard, S., Borschmann, R., Turner, K., GoodrichPurnell, N., Lovell, K. and Chambers, M. (2010) ‘What 23) Wiering, B., de Boer, D., and Delnoij, D. (2016) Patient involvement in the development of patient-reported difference does it make?’ Finding evidence of the outcome measures: a scoping review. impact of mental health service user researchers on research into the experiences of detained psychiatric Health Expectations 20, 11-23. patients. Health Expectations 13:2, 185-194. 12) Goward, P., Repper, J., Appleton, L. and Hagan, T. (2006) Crossing boundaries: identifying and meeting the mental health needs of gypsy travellers. Journal of Mental Health 15:3, 315-327.

24

BEE (RESEARCH) PRINT.indd 24

11/05/2018 16:14

A Research Handbook for Patient and Public Involvement Researchers

Chapter 2: Introduction to systematic reviews Kelly Rushton and Owen Price

Chapter overview Health professionals must make sure that the treatments they provide are safe and have been proven to work. This means they must be able to access and understand all the available research relating to treatments that they use. There can be hundreds of research studies conducted on each treatment, so it is often not practical for busy health professionals to gather all of the evidence themselves. Deciding on a treatment on the basis of a single study or a select number of studies is not recommended. Individual studies may have been poorly conducted and have misleading results, or the findings of the study may conflict with other studies that were not accessed by the health professional. This is why it is important that researchers gather all of the available evidence relating to each treatment, analyse and combine that evidence, and make it available to healthcare professionals in a comprehensible way. This allows professionals, and their patients, to make accurate judgements about a treatment’s risks and benefits. The method used by researchers to ensure that this process of identifying, analysing and combining the findings of multiple studies is sufficiently rigorous, is called a systematic review. This chapter will detail the steps involved in a well-conducted systematic review.

25

BEE (RESEARCH) PRINT.indd 25

11/05/2018 16:14

Learning objectives By the end of this chapter you should be able to: 1. Understand what systematic reviews are 2. Understand the stages involved in undertaking a systematic review 3. Understand why systematic reviews are important

Introduction Traditionally, systematic reviews have combined evidence from treatment trials and have answered ‘yes/no’ questions such as ‘Does this treatment work?’, ‘Is it safe?’ This involves combining ‘quantitative’ data (data related to numbers) across a large number of participants and studies and calculating what the average benefit (or risk) of a treatment might be. More recently, systematic reviews have also combined findings from ‘qualitative’ studies. Qualitative studies explore people’s views, experiences and beliefs in an in-depth way, using, for example, individual or group interviews. This type of evidence is important. For example, understanding the way service users feel about a service or treatment can provide important indications of how likely they are to continue to use it. Systematic reviews based on qualitative data can answer important questions about how, why and when a service or treatment might work, rather than ‘does it work.’ The information this provides can be used to change treatments and services to better meet the needs of service users. Again, a more complete understanding of views and experiences can be generated from combining the findings of many qualitative studies than from relying on the findings of one study alone.

BEE (RESEARCH) PRINT.indd 26

11/05/2018 16:14

Irrespective of whether you are conducting a systematic review involving qualitative or quantitative evidence, there are a series of steps that must be followed to ensure that your review is properly conducted. These are as follows: Data synthesis (the process used to combine and summarise all of the available data).

Defining the question (what is not already known and what will the review provide an answer to?).

1 Searching the literature (how and where studies will be identified).

2

3

Data extraction (how results will be transferred from the original studies to a review spreadsheet, so that the results of all the included studies can be viewed together).

4

A Research Handbook for Patient and Public Involvement Researchers

Conducting a review

5

Quality assessment (to judge how well the original studies were conducted, and to identify any particular strengths or weaknesses that might have influenced their results).

The following sections will describe these steps in more detail. Importantly, how each of these stages is conducted should always be agreed and written up in a review protocol before starting the systematic review. The review protocol should provide detail on all the stages of the proposed review. This helps to prevent researchers making on-the-spot decisions. It reduces error or ‘bias’ in the findings and helps others to judge how well the review was completed i.e. did it address its original aims and objectives and follow the right methods?

27

BEE (RESEARCH) PRINT.indd 27

11/05/2018 16:14

Defining the review question Developing a good review question is perhaps the most important step in conducting a systematic review. The question will inform the search methods, guide researchers in deciding which studies should be included and excluded and determine whether or not the review produces findings that are meaningful and useful to health services and their users. It is really important that the review question is well-constructed. The review question must have potential to generate new knowledge and understanding. Put simply, is the review worth doing? Is the question worth answering? The review question must be precise enough to ensure that the review can be completed. Systematic reviews are normally undertaken by a team of people, but can sometimes still take one or two years to complete. A review question that would need hundreds of thousands of studies to answer it would not be feasible to complete. There are some easy ways to ensure that a proper review question is developed. ‘Does it work’ (or effectiveness) questions use a technique called PICO (Population, Intervention, Comparison, Outcomes). This ensures the following categories are specified in the review question:

• • • •

Population: the patient or service user group that the review is interested in e.g. children, working-age adults, people living with psychosis Intervention: the treatment or therapy that the review will evaluate e.g. counselling, anti-psychotic medicines, care-planning Comparison: the treatment or therapy that the intervention will be compared with e.g. education, usual care or ‘no treatment’ Outcomes: the outcomes that will be used to evaluate the intervention’s effect e.g. mental health symptoms, quality of life, patient satisfaction

28

BEE (RESEARCH) PRINT.indd 28

11/05/2018 16:14

Figure 5 Developing a review question using PICO

Pop

ulat

ion

on

ention

Interv

Ou

is ar

tco

p

om

C

In working-age adults with anxiety, is Cognitive Behavioural Therapy more effective than Citalopram in reducing anxiety symptoms

A Research Handbook for Patient and Public Involvement Researchers

Figure 5 provides an example of how these four categories can be used to develop a focused research question.

me

However, not all review questions will be addressing ‘does it work’ or ‘effectiveness’ questions. Reviews that are interested in understanding people’s experiences will be searching for qualitative evidence. These types of review use a different ‘PICO ‘technique to structure their questions: PICo (Population, Interest, Context). This ensures that the following categories are specified in the review question:

• • •

Population: the patient or service user group you are interested in e.g. adults with bi-polar disorder Interest: the activity or issue you are interested in e.g. care-planning Context: the setting of interest e.g. community mental health teams

29

BEE (RESEARCH) PRINT.indd 29

11/05/2018 16:14

Figure 6 provides an example of how these three categories can be used to develop a focused research question for a review of qualitative evidence.

Figure 6 Developing a review question using PICo (review of qualitative evidence)

Po

n

tio

a pul

What do people with anxiety report about their experiences of receiving Cognitive Behavioural Therapy for anxiety in the community?

st

Co

re nte

I

nt

ex

t

Searching the literature Once you have your question it is time to see what studies are already out there. To search the literature in a rigorous way you must:

• •

Know which electronic databases you are going to search (i.e. identify your data sources) Have a systematic method for searching the databases to make sure no relevant studies are missed (i.e. develop a search strategy)

Searching databases will generate many studies that are potentially relevant to your review question. You will need to sift through these studies to identify those that are and discard those that aren’t. To ensure that this is done objectively, without bias, each study is assessed against pre-agreed criteria. These are called the review’s inclusion and exclusion criteria. Remember, the arrangements for all of these processes should be pre-agreed and written up in your review protocol before your systematic review begins. Changing the protocol after the review begins is possible but is not desirable.

30

BEE (RESEARCH) PRINT.indd 30

11/05/2018 16:14

Systematic reviews are normally conducted by searching online databases. These databases store nearly all of the research conducted in healthcare and their records stretch back many decades. Figure 7 shows some of the databases that are typically used for systematic reviews. You will notice that some of these are quite general and keep records related to many different areas of health research (e.g. MEDline). Others are more subjectspecific. PsycInfo, for example, only includes records of mental health and psychological research. Which databases you choose will depend partly on your review question. As a general rule, it is desirable to include all the databases that you think might hold relevant records. Although this will make your review more time-consuming, it will reduce the likelihood of you missing relevant studies and biasing your review. All of the databases in Figure 7 can be accessed via a simple Google search but they require expensive subscriptions. Most people access them using a University or hospital subscription, so you may need to ask these organisations for a username and password. Although using these databases is not difficult once you’ve learnt how to use them, they can seem fiddly and frustrating at first. Each has its own quirks in terms of how they work, so it is advisable to enlist the help of someone who is experienced in using them. Hospital and/or University librarians are often happy to book you in for an individual session on request.

A Research Handbook for Patient and Public Involvement Researchers

Selecting the data sources - where to search

Figure 7 Examples of electronic databases

As a Minimum • Cochrane Library (all sources) • Medline • Embase

Nursing-Specific • British Nursing Index • CINAHL

N.B. Which databases you choose depends on your review question but you will need to be able to justify your choices of databases, including any your choose not to use.

SubjectSpecific • Psycinfo

31

BEE (RESEARCH) PRINT.indd 31

11/05/2018 16:14

Developing a search strategy Developing an effective search strategy is vital, because it is this which ensures all relevant studies are included in the review. The term ‘search strategy’ refers to the terms that are entered into the electronic databases to retrieve studies. It also refers to how these terms are combined to ensure that only studies relevant to the review question are generated by the databases you use. This really is important - if your search generates 20,000 results, you might not have the time check them for relevance. The aim is to capture all relevant results without generating lots of irrelevant results. A useful technique is to structure your search terms around the PICO categories of your research question. Using the PICO question in Figure 5 (In working-age adults with anxiety, is Cognitive Behavioural Therapy more effective than Citalopram in reducing anxiety symptoms?), you can see that for each PICO component (Population, Intervention, Comparison, Outcome) there are many possible ways of referring to these. For example, Cognitive Behavioural Therapy is often referred to by the acronym ‘CBT’ and Citalopram can be referred to by its trade name ‘Cipramil,’ or a more general term such as ‘medication.’ Of course, different research studies will opt for different terms, so it is important that your search strategy can capture these differences. Figure 8 shows how search terms can be organised around the PICO categories (Population, Intervention, Comparison, Outcomes) using all possible term variants for each category. Of course, inputting all your search terms into the databases would simply generate a long list of the studies relevant to each term. Really, you want the database to only generate a list of studies that include all of the terms from all your PICO categories (e.g. Population, Intervention, Comparison, Outcome). This will minimise the number of results your search generates, without missing relevant studies. To do this, you have to let the database know which terms are variants within each PICO category, and how the different categories should be combined to generate the best set of results. This is achieved using ‘boolean operators.’ Boolean operators are simply the words ‘AND’ or ‘OR.’ You can enter these after each search word to let the database know whether the word you have used is a variant of other terms in the same category (by using ‘OR’) or whether you are combining different categories of search terms (by using ‘AND.)’ Look again at Figure 8 to see how AND and OR have been used to distinguish search term variants and PICO categories.

32

BEE (RESEARCH) PRINT.indd 32

11/05/2018 16:14

Population search terms

Intervention search terms

Comparison search terms

Outcome search terms

Working age OR 18-65 OR Adult

Cognitive Behavioural Therapy OR CBT

Citalopram OR Cipramil OR Celexa

Anxiety OR Worry OR Panic

A Research Handbook for Patient and Public Involvement Researchers

Figure 8 Search terms generated using PICO technique

Inclusion and exclusion criteria Once all your databases have been searched, you will have a long list of studies that are potentially relevant. These need to be sifted to check their relevance and to decide whether or not they will be included in the review. To ensure that this process of sifting and checking is unbiased, it is important that pre-agreed criteria are followed. These criteria are called your inclusion and exclusion criteria. To minimise bias, it is important that two researchers independently check each study against these criteria and compare whether or not they think it should be kept in or out of the review. The easiest way to develop inclusion and exclusion criteria for a review is to use the PICO framework. Table 1 provides an example of inclusion and exclusion criteria developed for the example review question in Figure 5.

33

BEE (RESEARCH) PRINT.indd 33

11/05/2018 16:14

Table 1 Inclusion and exclusion criteria generated using PICO technique PICO component

Inclusion criteria

Exclusion criteria

Population

Participants are working age adults (18-65 years)

Participants are older adults (65+) Participants are children and/or adolescents (